Anthropic Shows How It Verifies Code That Claude Code Writes 90% Of
When an agent writes about 90% of the code, reading every diff by hand is not feasible, so the Claude Code team builds verification as a separate system. The main tool is end-to-end tests: the agent runs the application itself and walks through a user scenario. In an internal test, a separate verifier agent using a checklist raised the task success rate by 10 points. The takeaway: give the agent a way to prove its result, and you can run dozens of instances in parallel.
AI-processed from Hacker News AI; edited by Hamidun News
If agents write nearly all the code, the bottleneck shifts from generation to verification. Claude writes about 90% of the code in the Claude Code team, and Anthropic has described how it controls quality at that volume. Without reliable verification, you can run only one or two agents, because you have to check everything yourself. With it, you can run dozens or even hundreds in parallel.
A figure from internal testing: when generating structured files, a separate verifier agent checked Claude's work against a rubric of criteria. This raised the share of successfully completed tasks by 10 points compared with the usual prompt-and-response loop. The gains were largest on the hardest tasks.
Three Types of Tests Worth Investing In
If you pick only one, the team recommends end-to-end tests. They describe the real user path: add an item to the cart, place the order, see the confirmation page. Such a test is hard to fool: the feature either works as a whole or it does not. A unit test will report "Timeout after 30 seconds," while an end-to-end test can show a screenshot with a description of what broke. Unit tests are faster and more stable, but they can pass while the function is still broken, because neither the agent nor the test sees the task as a whole.
The second type is property-based tests. You state an assertion that must never be violated, and a library generates thousands of inputs trying to refute it. Examples: encoding data and then decoding it returns the original value; an account balance never goes negative; running a migration a second time produces the same result as the first; optimized code matches a simple version that is known to be correct. These tests catch edge cases the agent would not have thought of, because those inputs are not generated with the same bias as the original code.
The third type is differential tests. The old and new versions of the system run on identical inputs, and their results are compared. Any divergence is either a bug or a change that must be explicitly approved. The method is useful during rewrites, when behavior must stay the same. Inputs should be random or sampled from real traffic, insignificant differences such as whitespace should be normalized, and acceptable differences should be kept in an explicit list.
A separate warning: flaky tests teach the agent to rerun the check instead of fixing the code. Slow tests lead people to run them less often and make root-cause analysis harder.
How This Works Inside the Team
Verification means running the change. To do this, the team creates a verify skill in the repository's .claude/skills/ folder, close to the relevant code. It describes how to bring up the application or simulator, and Claude uses it before committing. For the web, they connect a Chrome extension; for mobile, an MCP for iOS or Android; for desktop, computer use; for the backend, a locally run service.
A human is called on a failure signal: if verification still fails after three rounds of fixes, Claude stops and hands the task to people. Code owners write the checks, and when a reviewer is invoked, the check explains the cause so that no one has to work it out from the diff.
There is a 39-item reliability checklist. Each item has a trigger readable from the diff, the check itself, and links to incidents. Claude reads the diff and applies only the matching items, usually 3–10.
The key protective rule: a change must not alter the check that judges it. Code Review reads REVIEW.md from the base branch, not from the pull request. A review result is valid only for a specific commit.
A new commit gate is rolled out in stages in off, observe, and enforce modes. In observe, it only logs what it would have blocked, so false positives can be analyzed. In enforce, it blocks a commit that lacks /code-review or /verify, but after two blocks in a session it lets the next one through so the agent does not get stuck. Every commit is tagged with its verification method, including "none."
Checking the Checkers
The team tracks how many merged changes were reverted, how many were rejected by a maintainer, and compares these against a clean sample of the same size. To evaluate a reviewer, they assemble a set based on past incidents: the repository is restored to the state at review time, and the reviewer answers in one sentence what the bug is. Points are awarded only for a correct description. The tuning set is kept separate from the test set, and recall is measured separately for raw findings, after filtering, and at the end, to see whether already-found bugs are being lost. Findings are labeled Important, Nit, or Pre-existing.
New rules come from two sources: recurring review comments and reverts (at least two PRs, three agents try to refute the rule, and a human approves it), and re-running three bug-finding agents on a PR with a known incident. Claude may not skip, add delays to, or rerun flaky tests until they turn green: if the cause is not found, the test is deleted with a note on why.
What to Take Away
This is a description of one team's practice, not independent research: the +10 point figure comes from internal tests. But the set of techniques is general: make verification fast and stable, protect it from edits with a verifiable change, evaluate it on real bugs, and regularly prune its rules. Your work then shifts from reading every diff to defining what "done" means and checking the evidence.
- Nov 21, 2026Why an AI Agent Writes Slop: Ex-CTO of City Drive Explains How to Prepare a Repository
- Nov 21, 2026Anthropic Releases Claude Duet: An Agent That Reads Your Email and Slack and Finds Tasks on Its Own
- Nov 21, 2026Экс-исследователь Коксон заявил, что Anthropic может переобучить Mythos на доле вычислений
- Nov 21, 2026Nvidia reports without Huang: options price in a 7% stock move
Key Questions Answered
What are the key details on Three Types of Tests Worth Investing In?
If you pick only one, the team recommends end-to-end tests. They describe the real user path: add an item to the cart, place the order, see the confirmation page. Such a test is hard to fool: the feature either works as a whole or it does not. A unit test will report "Timeout after 30 seconds," while an end-to-end test can show a screenshot with a description of what broke.
How This Works Inside the Team?
Verification means running the change. To do this, the team creates a verify skill in the repository's .claude/skills/ folder, close to the relevant code. It describes how to bring up the application or simulator, and Claude uses it before committing. For the web, they connect a Chrome extension; for mobile, an MCP for iOS or Android; for desktop, computer use; for the backend, a locally run service.
What are the key details on Checking the Checkers?
The team tracks how many merged changes were reverted, how many were rejected by a maintainer, and compares these against a clean sample of the same size. To evaluate a reviewer, they assemble a set based on past incidents: the repository is restored to the state at review time, and the reviewer answers in one sentence what the bug is. Points are awarded only for a correct description.
Build & Ship Production AI Agents Without Boilerplate Code
Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.