Researchers prove that bloating AGENTS.md is harmful: the file has a size limit

The preprint arXiv 2610.11007 treats an agent's context file, whether AGENTS.md or an equivalent, as an optimization problem. At the start of each session, an LLM agent loads the entire file, and every token in it is paid for again in every subsequent round. The authors prove an upper bound on the optimal file size, and the bound does not depend on the number of candidate instructions available. No matter how many rules a team accumulates, they help up to a certain size, and beyond it the file only does harm.

In practice, these files usually grow by appending. The agent makes a mistake, and someone adds a line. Six months later the file runs to hundreds of lines, and nobody knows which of them work.

The model: a file as a shelf with limited attention

The authors frame curation as a capacitated assortment problem, that is, choosing an assortment under limited capacity. Instructions take up tokens, and the model's attention is finite. Adding a new instruction never raises the probability that the others are followed; it can only lower it. Each instruction kept in the file also carries a per-session maintenance cost.

These assumptions yield two results:

• An optimal file has a size ceiling that does not depend on the number of candidates.
• The strategy of "adding everything that is useful on its own" can lose to the optimal subset in net benefit by an arbitrarily large margin.
• A token budget caps the losses even if the token price was underestimated.

The last point is a practical one. The exact price of a token in an agent loop is hard to calculate, but a hard limit on file size protects against a gross error.

Censored feedback

The second part of the paper concerns learning from past sessions, and this is where a bias in the data arises. The benefit and harm of loaded instructions can be observed. A missing instruction leaves a trace only when its absence led to an error. If a rule is absent and everything goes smoothly, nothing can be learned about that rule. This is censoring.

An uncomfortable conclusion follows. If you delete instructions that the agent ignores, you will inevitably delete useful ones too. The absence of a visible effect in the logs does not prove that a rule is useless.

The authors also determine how much evidence must be collected before a new instruction is added. They also bound the regret for the case where human reviewers can check only a limited number of edits per period. This reflects a real workflow: an automated curator proposes changes, and a human physically cannot review all of them.

What the experiments showed

The abstract describes the empirical part only briefly. Irrelevant rules taken from real context files reduce a language model's instruction compliance. This is consistent with the theory, but the abstract gives no concrete numbers. Which models were tested and how far accuracy dropped will become clear only from the full text. The results are still preliminary: this is the first version of the preprint, without peer review.

How to apply it now

For teams that maintain AGENTS.md, CLAUDE.md or similar files, the paper suggests several rules:

• Set a hard limit on file size and discuss any excess explicitly.
• Add a new rule after several repeated failures, not after the first one.
• Do not delete a rule just because the agent "does not react" to it: first check whether it prevents silent errors.
• Periodically rebuild the file from a subset of the best rules and compare it with the current one on real tasks.
• Pass automated edits through review in small batches, taking the limited attention of reviewers into account.

This is a theoretical paper, and its conclusions rest on the model's assumptions, for example that a new instruction does not raise compliance with the others. Still, it offers a formal argument against the habit of endlessly appending to an agent's file.