Skip to content

How to scale AI responsibly in government

27 October 20263 min read
Guest Insights
How to scale AI responsibly in government

AI is moving quickly from experimentation into real public services. The challenge now is not whether government can use AI, but whether we can build the governance, assurance and organisational maturity needed to scale it responsibly.

What stops an AI prototype from becoming a production service in government?

Increasingly, I don't think the the answer is technology.

We can already build systems that summarise complex information, extract structured data from documents, assist caseworkers, generate content and increasingly take actions across other systems.

The harder question is whether our ability to assess, govern and assure those systems is developing at the same pace. Government now has a growing body of guidance on responsible AI, meaningful human control, lifecycle management and quality assurance. The challenge is turning those principles into something teams can apply repeatedly, proportionately and confidently.

We've been here before

I've worked in technology long enough to have seen several waves of transformational
technology.

Expert systems promised to capture human expertise in software. The web changed how organisations interacted with users. Mobile created new services and new questions about security, accessibility and governance.

Each felt fundamentally different at the time, but the organisational pattern was familiar. Technology moves faster than our ability to understand how to use it well. Teams experiment, standards emerge and governance catches up.

AI is following the same pattern, only much faster. The risk is responding with increasingly elaborate approval processes that make organisations feel safer without necessarily making the technology safer. What we need instead is better assessment maturity: understanding the use case, identifying the risks that matter and applying proportionate controls.

Governance maturity matters more than governance volume

There is already plenty of AI governance. The challenge is turning it into confident decisions about real systems, not creating another framework. A system that summarises internal documents for a civil servant is very different from one that recommends access to a public service or triggers actions across operational systems.

They should not face the same assurance process.

Mature governance means understanding consequence, autonomy, reversibility, data sensitivity and the opportunity for human intervention, then applying controls proportionately. And simply putting a "human in the loop" does not make a high-risk system acceptable.

What matters is whether the system remains meaningfully under human control.

Assure the components, then assure the system

The challenge is how we think about AI assurance itself.

An AI-enabled service is rarely one thing. It may include a foundation model, prompts, retrieval, government data, APIs, agent tools, identity and access controls, policy rules, guardrails and human approval points. Assessing that as a single ‘AI system’ quickly becomes difficult. A more scalable approach is to create an assurance chain.

We should be able to assess individual components against clearly understood requirements. What is this model approved to do? What information can this retrieval component access? What actions can this agent perform? What evaluation has a particular guardrail passed? What permissions does this tool actually have? Where components are reused, much of that evidence should be reusable too.

That does not mean certifying a model once and assuming every system built with it is safe. Composition matters. A low-risk model connected to a powerful tool with excessive permissions can create a high-risk service very quickly.

So component assurance needs to sit alongside end-to-end evaluation. That gives us a potentially much more scalable model: known components, known constraints, reusable evidence and service-level assurance of how those components interact. It also moves us away from one-off approval towards continuous assurance. Models change.

Data changes. suppliers release updates and user behaviour changes.

An AI service that was safe when it launched is not automatically safe six months later.

Keep the human in the lead

The language we use around human oversight matters too. We often talk about keeping a "human in the loop". But that can imply that every AI action needs somebody sitting there approving it. Sometimes that is exactly the right control. Sometimes it is inefficient theatre.

I prefer a phrase Dr Tommaso Spinelli used during our recent AI lunch and learn event: keep the human in the lead. A human can remain in the lead without manually approving every individual step. People should define the objectives. They should decide what the system is allowed to do, establish the boundaries within which it operates and determine which decisions require escalation or approval.

They should be able to understand what happened, intervene when required and ultimately stop the system.

For a consequential decision, that may mean explicit human approval. For a lower-risk administrative process, it may mean sampling, monitoring, thresholds and exception handling instead. The important thing is that accountability has not quietly migrated from
people into the technology.

That distinction becomes even more important as we move towards agentic systems. The interesting question isn't whether an agent can act autonomously. Technically, increasingly it can.

The question is what autonomy we’re prepared to give it, under what conditions and with what evidence that those controls work?

Responsible AI is how we get to scale

Responsible AI is sometimes framed as the counterweight to innovation. I think the opposite is true. Good governance should help organisations move faster: clear risk classifications, reusable assurance and known boundaries for experimentation should reduce unnecessary friction while focusing scrutiny where it matters.

That is especially important in government, where a visible failure can damage confidence well beyond a single service. The goal is not risk-free AI. That does not exist. And human-led systems are not risk-free either. The goal is to understand the risks well enough to make deliberate decisions about them.

Many of the disciplines are familiar: understand the problem, know your data, establish accountability, test what you build and monitor it in the real world. The organisations that scale AI successfully will be those that get good at deciding where AI should be used, what evidence is enough to trust it and how much control it should be allowed to exercise.

That is the maturity we need to build.