Claude Code wrote the search in 3 days, but the Jmix team needed 3 weeks to accept the code

Agents write code fast, but a human still has to read it, understand it, and take responsibility for it. The team behind the open-source Jmix platform describes this from their own experience. They wrote a new search and index for their AI assistant in three days together with Claude Code. It took almost three weeks before someone on the team was ready to answer for the result.

Two timers instead of one

Before answering, the assistant itself searches a vector database for documentation fragments. The team added class cards from Javadoc to that database and split the documentation into short, self-contained examples. Quality is checked by an evaluation loop: 74 control questions whose answers are scored by a judge model.

Claude Fable 5 wrote the first version of the code, and GPT-5.6 Sol independently re-checked individual pieces. The result was a diff of 122 files and almost 15,000 lines. The authors had planned for two to three days of review. They saw every change, and they could ask the agent about any decision. But skimming a diff and understanding it well enough to answer for it without the agent turned out to be two different things.

The team describes this as two timers. The first runs until working code exists, and it stopped after three days. The second runs until someone on the team can fix the system from memory. The authors call this moment code adoption; in essence, it is ownership. The signs: a person can trace the data path from input to result, explain the invariants, predict what a change will affect, and alter a contract knowing what will need re-checking. A green build does not guarantee this, because it answers only the questions defined in advance. For example, has anyone checked that if a failure hits mid-reindexing, the old fragments stay in place?

Where the three weeks went

Reading speed explains only part of it. According to SmartBear and Cisco, a single review session should cover 200–400 lines in 60–90 minutes. Of the 15,000 lines, about 3,000 are HTML fixtures that do not need to be read. That leaves roughly 12,000, or 30 sessions and 30–45 hours of reading. That is about one work week, and tests are read faster. The authors admit the estimate is rough and the study is almost twenty years old.

Three things consumed the rest. First, the agent changed code "along the way." On the third day of review, support for several Jmix versions had vanished from the code.

The agent changed the /api/search response format on day one, so the old contract had to be restored and the new format moved to /api/v2/search. The team also removed screenshots and dumps totaling 3.5 MB and Playwright logs from the repository.

A series of "Restore … as in main" commits ran for almost a week. Second, the agent explains correct and incorrect things with equal confidence, so its explanations had to be checked against the code and confirmed with tests.

Third, late changes arrived: the pass threshold became a separate field of the run, the judge's rules were tightened, and a shortened form of a DataManager query was added to the reference set. Everything the team had already understood about what the score meant had to be re-verified.

From the commit history: 44 commits in the first three days, then 53 over a week and a half of review, and another 47 in the last nine days.

The industry data gives this context. According to DORA 2024, a 25% rise in AI use was associated with better documentation and faster review, but delivery throughput fell by about 1.5% and stability by 7.2%. That is a correlation from a survey, not an experiment. In DORA 2025, the link to throughput turned positive, while the link to stability stayed negative. In a METR experiment, experienced developers using AI spent 19% more time on tasks but believed they had sped up by 20%. The tools there were from early 2025, and a repeat measurement in February 2026 turned out to be statistically unreliable.

What the team does now

The main idea is to distinguish three boundaries. Delivery defines what lands in main as a whole. An agent iteration brings the work to a stable state that can be brought up and checked. The code-adoption point shows which piece a human already maintains alone. Previously, the team combined all three in one big pull request, and the second timer started only after the first.

Now the work is cut into three slices: the corpus lifecycle, search with the API, and the evaluation loop. A human checks and accepts each slice before the next one begins. The slices go in a stack of branches, and one final pull request goes to main, where only the seams are checked. For example, that after a failed reindexing, search still returns the old generation of data.

Six practices: describe the contract of the whole system before the first session; cut the work into stable states and leave a note between sessions; ask the agent for confirmation instead of a report; hand verification to a separate agent with fresh context; bring in a human after each slice; and finish with a full integration acceptance. Similar ideas appear in Anthropic's experiments with long-running agent tasks and in Claude Code's recommendations for checking a diff with a subagent.

Limitations

The authors are upfront: this is not yet a formal procedure, and they have no measurements comparable to the three weeks of that branch. An open question remains whether the separate checker helped because of its fresh perspective or simply because it was a different model. An accepted slice is not frozen either: the meaning of the score had to be revisited on the last day. The practical takeaway for teams is simple: count not only the time to generate code but also the time until someone is ready to answer for it, and build acceptance points into the plan before the agent's first session.