The Agent is the Wrong Unit of Governance
Agentic AI is redefining what it means to ‘use’ a system. These tools no longer wait to be prompted; they plan, decide, and act with limited human oversight, and increasingly hand work to one another to run end-to-end processes.
Enterprise AI has arrived in waves. Much of it thus far has been discrete and task-bound, a model to score a document, for instance, deployed inside workflows under human review. The capabilities now emerging are fundamentally different. Agents are beginning to coordinate with other agents and complete multi-step processes. Value is therefore shifting from the individual task to the workflow, and ultimately to a system that both advises and acts.
The opportunity extends across sectors. In financial services, agents could coordinate activities across customer servicing, transaction processing and exception handling. In utilities, they could orchestrate customer and operational workflows, while in public services they could coordinate multi-step processes across systems. The common thread is that work which once needed a person at every step can increasingly be delegated to systems capable of sequencing those steps themselves. For a UK economy focused on productivity, this represents a substantial opportunity.
The next stage is not simply the emergence of more capable individual agents, but networks of specialised agents working across systems, functions and, increasingly, organisational boundaries. A customer-facing agent may initiate a request, another may interpret policy, another may execute a transaction, and another may monitor exceptions.
Simultaneously, this is the point raising more complexities and questions. When an agent acts, someone still must answer for what it did. The instinctive answer is to govern each agent, to give it an owner, scope its permissions, and keep a human in the loop. While this is an appropriate response, it is increasingly difficult to sustain at scale. The Bank of England reported that firms themselves think the traditional model-validation approach “wouldn’t be sustainable in its current form” as agentic systems spread, and that the human-in-the-loop concept is “challenged by the rise of agentic AI.” When a person cannot review every action, monitoring the agent no longer provides the same degree of control.
This points towards a broader shift in risk management: from validating individual models and reviewing individual actions towards testing, monitoring, and setting guardrails around the outcomes of the wider AI system. For agentic AI, this becomes vital as multiple agents combine to deliver a singular outcome. The agent may therefore be the wrong unit of governance. The more relevant unit is the service the agents exist to deliver.
UK financial services already has the vocabulary for this. The FCA’s operational resilience regime asks firms to ‘identify their important business services, set an impact tolerance for each, the point past which disruption causes intolerable harm, and show they can stay within it under severe but plausible conditions, including by understanding the third parties their services lean on’. Although designed for financial services, the underlying principle offers a useful model for other sectors running critical operations; an agent is simply one more dependency inside a service someone relies on.
Reframed that way, the focus shifts from whether an agent is ‘well-behaved’, to which service it resides in, and how much disruption that service is able to absorb. It shifts to who owns the outcome across the whole chain, not just the agent that acted; to whether we could reconstruct, afterwards, what each agent did and in what order; and to what happens to the service if a model or provider beneath the agent changes or fails.
This suggests a shift in governance architecture. Model governance and agent-level controls remain necessary, yet neither is sufficient when multiple agents collectively deliver a business outcome. A service-level layer is needed above them, connecting accountability, resilience, permissions, dependencies and evidence to the outcome that is produced.
UK regulatory thinking already points in this direction. In his June keynote, the FCA’s chief executive called agentic systems a “profound step change” while insisting that accountability for regulated activities and their outcomes must stay clear, whatever the degree of automation, and that firms should strengthen governance over their third-party dependencies. Accountability does not travel down the chain to the agent. It stays with the organisation, attached to the service.
This is the discipline turning the productivity promise into one that is durable. Agentic commerce makes it concrete: an agent negotiating a purchase pulls in the customer, the firm and the model provider at once. Responsibility becomes fragmented when assessed agent by agent, but is easier to establish at the service level, where an evidence trail can be maintained.
The organisations that scale agentic AI successfully will be those that embed governance into the system itself. They must be able to state clearly which service an agent supports, what it is permitted to do, who owns the outcome, and what evidence exists for every action it takes.
