Building effective human oversight into agentic AI
Most of the agentic AI systems I've seen proposed over the last year come with a reassurance attached, which is that a person will stay in the loop. A human approves the payment the agent has set up, signs off the report it has drafted, or checks the decision before it reaches a customer. It's a sensible instinct, and for a lot of regulated work it isn't optional. Under the EU AI Act, anyone deploying a high-risk system will need to make sure the people overseeing it are competent, trained and able to act, and in the UK the Data (Use and Access) Act has widened the room for automated decisions, provided safeguards are in place, including a way for people to obtain human intervention.
What gets much less attention, in my experience, is whether that oversight can still happen once agents are in place, and whether anyone could show afterwards that it did.
A lot of the review in today's workflows happens almost by accident, at the point where one person passes work to another. The manager reads the draft before it goes out, the second team looks at the case when it lands with them, and the payment sits in a queue until someone signs it off. Nobody designed most of those moments as controls, but they catch a good deal. Agents are very good at joining up steps that used to be handled by different people, and when they do, those moments tend to go too, usually without anyone having decided they should. In some set-ups the person and the agent are working on the same piece of work together, so there isn't a separate point of review to speak of.
techUK's industry brief on scaling agentic AI recommends building observability in from day one, and that's right. But it answers a different question. Observability tells you what the agent did, and it doesn't tell you whether a person examined it, whether they had what they needed to catch a problem, or whether they could have stopped it. A record that someone pressed approve, or accepted a suggested change, looks exactly the same whether they read the work carefully or barely glanced at it.
A colleague and I recently proposed a framework to the European Commission on what organisations would need to record for human oversight of AI-supported work to be demonstrated, and what an inspector could reasonably conclude from those records. One of the things it sets out is that the phrase "a human reviews it" can hide several quite different situations. Sometimes the check exists in the procedure and nothing shows it was carried out. Sometimes it happens, but the reviewer isn't really in a position to spot a problem, because they don't have the context, the time, or the standing to send the work back. Sometimes it's done properly on some of the work and not the rest, and sometimes the person doing it isn't the one the procedure says should be. Each of those calls for a different fix, and a policy that says review takes place doesn't tell you which one you have.
For organisations deploying agents, I think a few practical things follow from this.
The first is to decide where a person needs to check the work before the workflow is automated, rather than discovering afterwards that the checks have gone. Mapping the process before deploying agents, which techUK's brief also recommends, is the natural point to do it, because it's when you can see where work currently changes hands and which of those moments are doing real work.
The second is that where review has to be shown as a distinct step, the workflow needs two things built into it, somewhere in the process where the review can genuinely happen, and something that records whether it happened. Neither is much use without the other, and neither can be added cheaply once the system is live.
The third is to record more than the approval. Whether the reviewer had the information and the time they needed, and what share of the work the check actually reached compared with what it should have, are what turn a log entry into evidence that oversight happened.
And the fourth is to be honest about the steps that don't need a person at all. Some work is better handed over completely, and keeping a nominal approval on it costs time while adding very little. Taking the check away deliberately, and saying so, is a better position to be in than keeping one that nobody can show is doing anything, and that only adds to approval fatigue.
This doesn't need new technology. It needs the oversight to be designed in alongside the agent, rather than assumed to survive it. I think the organisations that do this well will find they can give agents more responsibility over time, because they'll be able to show where people still matter in the work, and that the people were there.
