Human review isn't enough: manage AI risk task by task
Analysts followed the AI agent in 86% of choices. Putting a human at the end isn't a control. How to give AI autonomy per task, with measurement, without losing the reins.

“Don’t worry, there’s a human reviewing it.” That sentence has become the official sedative of every AI agent project. It shows up in the vendor’s pitch, in the committee minutes and in the reply to the legal team.
Last week, a study showed why it isn’t enough. And, on the same day, Oracle announced that ChatGPT will carry out actions inside the ERP and IDC said that companies without an off switch will pay more. AI is getting hands, and our protection is still a checkmark at the end of the process.
My thesis: AI autonomy isn’t a company-wide decision, it’s a per-task decision. And what unlocks that autonomy should be measurement, not the impression of whoever uses it.
What Anthropic’s study exposed
Anthropic published a controlled experiment with 50 professional analysts using an AI agent to evaluate a program at a fictional farming cooperative. In four choices of method, each with two defensible options, a coin flip secretly decided which way the agent leaned.
The analysts went with the agent’s option 86% of the time, when chance would give about 50%. Among those who had already picked an option, 60% switched when the agent disagreed, although that number comes from only 38 cases. And 77% repeated the agent’s arguments in their own explanations, but only 15% gave it credit.
The limits matter, and the authors themselves point them out. The preference was deliberately reinforced, it was a single task with a single model, and the analysts didn’t know the farming sector. In everyday work, the effect may be smaller.
There’s also a data point in AI’s favor. Of the 50 who used the agent, 6% made serious method errors. Among the 11 who worked without it, 6 people did, or 55%, but that group had less experience and the authors warn that the comparison doesn’t prove causation.
Putting both ends together, my reading is this: AI tends to reduce execution errors and, at the same time, steers the choice of path. Review at the end catches a wrong calculation. It doesn’t catch criteria chosen by someone else. The authors say it another way: a reviewer is only an independent check if the judgment is really their own.
Meanwhile, many companies have already let the agent loose
On the other side, some don’t even have review. A RealVNC survey of IT decision-makers in the United States, the United Kingdom and Canada says almost three in four companies let agents act without prior approval. Only 26.4% can revoke an agent’s access. Among those that let agents act without approval, 75.5% have no way of doing so. RealVNC sells remote access and the press release doesn’t say how many companies it surveyed, so read it as a signal, not a picture of the market.
A survey with a small, self-selected sample from VentureBeat, with 108 people in July, points in the same direction: 49% had a customer-visible incident after an AI feature passed internal testing. And only 28% of the group that already allows changes without approval automatically checks whether the answer is right.
The market, interestingly, doesn’t ask for either extreme. G2 analyzed 3,222 reviews from companies with more than a thousand employees. Among the 1,746 who answered about autonomy, 60% prefer autonomy with defined limits, 11% want a human on every action and only 7% want full autonomy.
A human on every action doesn’t scale either. In the same analysis, G2 cites its own research in which 31% of companies spend six or more hours a week supervising agents. Whoever reviews everything ends up signing without reading.
Red, yellow and green
The best model I’ve found to get out of this false dilemma comes from Richard Stockley, in the book Intent-Driven Development, which is distributed for free. He talks about “risk dials”, with three positions:
- Red: the person reviews and approves before moving on. What the agent delivers is a suggestion.
- Yellow: the agent proceeds and the person reviews a sample, and can undo.
- Green: the agent acts on its own and the person is notified. Only after proven trust.
Three of Stockley’s ideas seem the most important to me. Everyone starts on red. The one who sets the dial is the company, not the AI or the vendor. And some things never leave red: strategy, ethical limits, security, compliance and final approval of what goes to production.
I’ll add one thing because of Anthropic’s study: red isn’t a synonym for safe. If the person approves what the AI proposed without having thought it through first, the dial is on red and the control is on green.
The gradual approach also has support in usage data. In an Anthropic study of the company’s programming agent, new users turned on automatic approval in about 20% of sessions, and the most seasoned ones in more than 40%. But the experienced users interrupted more: about 9% of turns, versus 5% for newcomers. They went from approving every step to watching and stepping in. It’s a technical audience, but the logic holds: good autonomy isn’t letting go, it’s changing how you keep track.
What the manager should do
1. List what the AI does, task by task. For each one, answer three questions: can it be undone? Who is affected if it goes wrong? Does it involve money, personal data or a legal obligation? Summarizing a meeting and approving credit can’t live on the same dial.
2. Start on red and write down what never leaves it. That list fits on one page and should have an owner. Decisions that affect customers deserve extra attention: article 20 of the LGPD (Brazil’s data protection law) gives people the right to request review of decisions made solely through automated processing. If you don’t yet know where you’re exposed, the free self-assessment takes 2 minutes.
3. Define the criteria for changing the dial before you start. Stockley suggests a reference: full review below 85% fidelity to intent, sampling between 85% and 95% over twenty or more deliveries, and monitoring only above 95% over fifty or more. Adjust it to your reality, but have a number. And agree that the dial can also go back.
4. On judgment calls, get the opinion before the AI’s. The person responsible writes down their own position in two lines, and only then consults the agent. Set up the agent to show the alternatives, with pros and cons, instead of a ready-made answer. Anthropic’s authors suggest a similar path.
5. Test the shutdown. IDC predicts that, by 2028, large companies without an off switch will pay 25% more for liability insurance and will have twice as many audit findings. It’s a prediction, not a fact, but the test is cheap: once a quarter, run a simulation. Who shuts it down, how fast, and what happens to tasks left halfway?
6. Ask the vendor who controls the dial. When hiring an agent, ask what level it comes with out of the box, whether it can be changed by type of action and whether every action is logged. Oracle says approvals and controls remain in place, but the announcement doesn’t explain how. That’s the right question for any vendor.
Where experience comes in
In 29 years of programming, I’ve never seen an important system go to production without someone deciding what could fail and what couldn’t. AI agents don’t change that conversation. They just make the answer more urgent, because they execute before anyone finishes reading.
That’s the work we do at IWA7: look task by task, separate what can run on its own from what needs people, and measure before letting go. Not to slow AI down, but so the company can accelerate knowing where the brake is.
Originally published on IWA7 Insights, IWA7 Digital’s blog, which is now part of Escovando Bytes.


