Shadow Sciences Group, a boutique exposure intelligence and risk advisory firm, warns that the disappearance of human judgment from automated workflows has left fraud controls with nowhere to attach — and that reversibility, not importance, is the right test for where a person must remain in the loop.
The Argument
The standard advice before sending a wire transfer is to phone the recipient first, on a number you looked up yourself rather than one printed in the email. It is good advice. It also depends entirely on somebody being there to place the call.
The firm's August series “Who Decides” traced how judgment became something people hand off, moving from labor to research and eventually to the conclusion itself. Its argument now is that the handoff did not stop there. It carried on into the action.
“A recommendation still leaves somebody in the loop, even a distracted one. There is a click. That click is where our controls live. It is where you catch that the account number changed. When the system acts instead of suggesting, the click is gone. Nothing got defeated. The control just has nowhere left to attach.”
Kia Hakimi, Partner, Shadow Sciences Group LLC
What Six Governments Said
In April, Australia’s Cyber Security Centre published guidance on agentic AI together with CISA, the NSA, Canada’s Cyber Centre, and the national cyber agencies of New Zealand and Britain. Two of its recommendations bear directly on the firm’s argument.
The first asks organizations to sort automated actions by impact, likelihood, and reversibility before deciding how to guard them, and the document closes by ranking reversibility ahead of efficiency gains.
“That is the whole test. Not importance. Reversibility. Nobody needs to review a calendar invite. If the wrong answer wires money overseas or records a document against your title, somebody has to be standing there.”
Kia Hakimi, Partner, Shadow Sciences Group LLC
The second recommendation states that decisions about when a human must approve something belong to the people who design and operate the system, and cannot be handed to the system itself.
“Six governments sat down and wrote that sentence, which tells you it was already happening. Your reviewer is looking at the exceptions that some other system decided were worth showing them. So the judgment about whether the judgment held up got delegated too. At some point you are not really supervising it. It is just telling you that everything went fine.”
Kia Hakimi, Partner, Shadow Sciences Group LLC
The Cost of What Cannot Be Reversed
FBI figures put business email compromise at $3.047 billion across 24,768 complaints last year, second only to investment fraud, with roughly 86 percent of the money moving by wire or ACH, where recovery is least likely. Of that total the bureau attributed more than $30 million specifically to AI-enabled schemes — the first year the figure could be measured at all.
The Firm’s Position
Shadow Sciences is not arguing against automation, which it calls neither achievable nor desirable to avoid. Its position matches the advisory’s: automate the things you could undo and keep a person on anything you could not.
“Who Decides” is available at shadowsciences.com/insights/who-decides.