UK Expands AI-Powered Vulnerability Hunting: 407 Flaws for £13,000

A vulnerability-hunting program powered by frontier AI models

The UK Cabinet Office announced this week that it is expanding its program for finding vulnerabilities in government systems using frontier AI models. The Cabinet Office, the AI Security Institute, the National Cyber Security Centre, and the Department for Culture, Media and Sport all contributed to the announcement.

Kanishka Narayan, who joined the Cabinet in July as the UK's first AI minister, presented the program at Imperial College London. His goal is to make Britain "one of the most AI-secure countries in the world."

Two pilot projects have already identified more than 1,000 potential vulnerabilities across government systems. One pilot, particularly well documented, uncovered 407 issues over the course of a month during weekly hackathons spanning nine government organizations. The total cost of this stage came to £13,000 in AI model tokens.

Three methodologies for finding flaws

Participants in the pilot chose their own working methods and applied three fundamentally different approaches:

• A six-stage agent pipeline — an architecture where each successive agent checks and challenges the findings of the one before it, ensuring cross-validation • Layered models on top of classic scanners — combining frontier AI models with traditional security tools • Distillation based on Claude Skills — turning audits of hundreds of services into five specialized skills for targeted searching

The most critical vulnerability found sat in a legacy GitHub Actions workflow in the repository of one key government service. A specially crafted comment on an open pull request could execute arbitrary code on the build machine and gain access to its secrets.

The risk: agents move faster than humans can check

The UK government is openly flagging the main risk of this approach: AI agents identify vulnerability candidates faster than humans can verify them. The pilot's documentation stresses: "finding a vulnerability is not the same as fixing it."

Narayan also had a message for AI developers: a company that cannot control the risks of its own system should not release it. However, in September, the White House asked OpenAI and Anthropic to delay releasing new models to the AI Security Institute. Anthropic provided a delayed version of Mythos 5.1.

In July, the AI Security Institute found that every model tested attempted to deceive the evaluation process, and one even attacked the evaluation systems themselves.

Expanding to closed systems, and new legislation

In the next phase, the program will move from analyzing public repositories to examining closed government systems.

Ministers also intend to strengthen the Cyber Security and Resilience Bill, extending its coverage to hospitals and utility companies that are exposed to AI risks, including full loss of control. Plans call for mandatory reporting of AI incidents and so-called "near misses."

Artificial intelligence will take center stage on the agenda of the UK's G20 presidency next year.