Bank Architect: In a METR Test, AI Made Experienced Developers' Tasks Take 19% Longer

The head of the architecture function at a Russian bank has published an analysis on Habr of how generative AI and LLM agents are changing enterprise architecture. The author's main argument rests on a randomized METR test from 2025 (METR is an independent organization that evaluates AI capabilities). Experienced developers who used AI on mature codebases spent 19% more time on tasks than when working without it.

They were nonetheless convinced that AI had made them 20% faster, so the gap between perception and reality came to 39 percentage points. In the author's view, this shows that writing code was never the bottleneck in the enterprise. The real difficulty is fitting new logic into a landscape of legacy systems, regulation and information security constraints.

The Illusion of Speed and the Price of Cheap Code

A neural network removes the fear of the blank page: a developer gets a chunk of code instantly. Then integration begins. Generated code looks syntactically flawless, but it often ignores the implicit dependencies of legacy systems and crashes at runtime. Engineers spend hours debugging code that only looks like it works.

Data from GitClear indirectly supports this. The volume of copied code has exceeded the volume of refactored code, and the churn rate (the share of fresh code rewritten soon after commit) has risen from 3.3% before mass adoption of AI to 7.1%. DORA reports show the same pattern: throughput rises while delivery stability falls.

The author's conclusion is practical: you cannot rely on human code review. Pipelines need deterministic sensors, such as fitness functions, linters and contracts, that keep hallucinated or structurally broken code out of the main branch. Architecture documents give way to guardrails-as-code.

Russian Regulation and the Model as an Actor

Western news about postponed deadlines means almost nothing to a Russian bank. The EU AI Act Omnibus pushed requirements for high-risk systems back to 2 December 2027, and the US Federal Reserve and FDIC, through document SR 26-2, took generative and agentic AI out of standard model risk management. In Russia, architecture is shaped by Federal Law 187-FZ on critical information infrastructure (CII), Regulation 850-P on operational resilience, and the deadline for moving significant CII objects to a domestic stack, 1 January 2028. Every decision has to pass the filters of the Russian software registry, the significance categories for CII objects, and personal data localization under 152-FZ.

Clause 2.5 of the Bank of Russia's document 3-MR, issued in June 2026, recommends mandatory confirmation of an operation by an employee, who must be able to change it. This applies to AI in critical processes such as payments with high information security risks. As a result, human-in-the-loop has become a mandatory layer of design. Above the LLM, the bank builds a router abstraction so that switching models because of the AI law (243-FZ) does not paralyze operations.

A design review starts with one question: is the model a component or an actor? If it is locked inside an application and only prints an answer to the screen, the OWASP GenAI LLM Top 10 is enough. If it has memory, API access and function calling, the OWASP Agentic Top 10 applies, where the risk of Excessive Agency has climbed to third place. For such agents, the bank builds code execution sandboxes, strictly limits tool calls and closes direct access to databases.

An agent is not a service account. A static API key with broad permissions opens the door to a confused-deputy attack: a crafty prompt makes an authorized agent perform a malicious action with maximum privileges. That is why every agent gets its own identity, and permissions are issued as short-lived delegation tokens for a specific call. Audit logs show where a human acted and where an agent acted on their behalf.

The third risk is "inherited" AI. SaaS, ERP and ITSM vendors switch on built-in Copilots through updates and change the map of data flows, which affects banking secrecy (Art. 26 of the law "On Banks and Banking Activity", Art. 857 of the Russian Civil Code). The bank's policy is default-off for external AI features. Enabling one is treated as a major change, and each is tracked in a separate register.

Documentation for Machines, the End of the ARB and New NFRs

Architecture documentation (ADRs, C4 diagrams, API descriptions) lives in Git as Markdown, PlantUML and Structurizr. Agents read raw text from a repository better than exports from proprietary systems, so the repository holds the context files RULES.md and ARCHITECTURE.md. For access, the bank wrote an internal layer over its repositories with MCP support. The ARB committees have been disbanded. An advice process now applies, in which the solution architect in a domain makes the decision, while regulatory perimeters remain mandatory constraints.

The ADR templates now include the boundary between deterministic logic and probabilistic output, the topology of data flows inside the bank's perimeter, the triggers for handing control to a human, and an exit path in case a vendor degrades or changes its license. Non-functional requirements now include metrics for behavior on failure: a "correct refusal" is worth more than a risky hallucination. In FinOps, each business transaction has a token cost ceiling, and latency is measured across the whole cycle, including the agent's calls to external systems.

The author leaves the question of metamodels open: ArchiMate cannot adequately describe a prompt, a validation threshold or a dynamic delegation chain. The bottom line is this: a blueprint promises how a system will work, and a probabilistic component makes no such promises. What remains is to constrain it in advance, intercept it at the moment of action, and prove after the fact whose authority it acted under.