Agent loses data because of table size, not the number of turns: 92% correct turns vs 46%

A typical agent is simple: a tool returns a table, the model receives it in context and retells it to the user, and answers follow-ups like "show only departures after 9 p.m." from memory.

It looks harmless, but this is exactly where the data gets corrupted. The search computed everything correctly, the JSON is valid, the schema matches, and yet the final table the user sees has a different price, a wrong date or a missing flight. The authors of a recent arXiv paper measured this on a real production agent.

By "accuracy" in the title they mean the share of dialogue turns in which every field of every row matched the reference. The main result: the drop in accuracy is determined by table size, not by how many turns have passed. If you build agents, this directly affects your architecture and your token bill.

What was compared

The researchers used AwardWise, a multi-agent assistant for finding award flights bookable with miles, built on Google Agent Development Kit. The two versions of the system differ only in how data is delivered. The model (Gemini 2.5 Flash), the temperature of 0.1, top_p of 0.5 and the related instructions are identical.

The first version, SYNTHESIS, follows the standard framework pattern. The full list of flights goes into the planner's context, and the planner builds the table itself and answers follow-up questions itself.

The second, BYPASS, is the one running in the production system. Records sit in the session state and never pass through the model's generation. The model only translates the user's request into filter arguments, ordinary deterministic code runs the filter, and the rows reach the interface through a separate typed channel.

The test covered 24 dialogues, with 339 scored turns for each variant. Provider responses were frozen in advance: 408 snapshots across 36 routes, with the largest result set containing 510 rows. The criterion is strict: a turn counts as correct only if the fields match the reference.

What the results show

BYPASS was fully correct in 92.3% of turns and SYNTHESIS in 45.7%, a gap of +46.6 percentage points. Across three model families, the gap stayed in the range from +46.6 to +55.8 points.

The researchers then worked out what drives the failure. They varied table size by snapshot date, by prefixes of the same result set, and with a "Latin square" design that separates size from position in the dialogue. In every case SYNTHESIS collapses as the table grows, while BYPASS stays flat. The collapse threshold depends on model strength: for GPT-4o-mini it is below 10 rows, and for Gemini 2.5 Pro it is higher. But the bypass holds its level on all four models tested: 140 of 144 runs. The authors put it this way: a stronger model buys a higher threshold, while architecture buys immunity.

Swapping Flash for Gemini 2.5 Pro on the same dialogues raises retelling accuracy from 46% to 72%, but each turn costs roughly 5 times more. That is an expensive way to rent accuracy instead of not losing it in the first place.

Costs, errors and limitations

With BYPASS, the cost and latency of a turn do not depend on table size: about $0.012. With retelling, costs grow along with the context. On result sets of more than 100 rows, almost no answer is correct, and each correct answer costs roughly 200 times more than on the bypass path.

A pre-registered test did not confirm the hypothesis that errors accelerate with each turn. According to the authors, the cause is the method: with such a high failure rate, there is nothing to compare against. What is visible, though, is that errors propagate. A corrupted table becomes the input for the next turn. In BYPASS, each turn is recomputed from the stored state, so there is no such chain, and the null result of 0/25 only means that no counterexamples were found.

The limitations matter too. This is one system in one domain, the main measurements were made on Gemini 2.5 Flash, and the dialogue scenarios were written from the sessions of one experienced award-travel practitioner. The gain is not broken down by factor: state storage, the deterministic filter and the side channel were evaluated together.

The practical takeaway: side channels such as LangGraph or AG-UI have usually been adopted for the sake of a predictable interface. This work shows that they also protect the data. If a tool returns dozens of rows, keep them in the session state, hand the filtering to code, and leave the model only the job of translating the request into arguments.