Agent With a 16K Window Holds 70.2% on SWE-bench Verified by Resetting Context Every Step

Everyone is used to the idea that an agent working on a long task needs a huge context window: the more fits, the better. A new paper, "You don't need long context" (Belcak, Molchanov and co-authors), argues the opposite: a short window is enough if you build the harness correctly.

What the authors proposed

The idea is simple. Instead of accumulating conversation history until it runs out, the harness periodically resets it and starts a new session. Only a structured working summary is carried over. Large files are not loaded into context in full: the agent searches for what it needs using Bash utilities such as grep and reads in limited windows.

There are two modes. End-of-Conversation Compaction (EoCC) compresses the conversation once a token threshold is reached. Single-Action Compaction (SAC) is more radical and resets the context after every action.

Imagine a developer fixing a bug in a large repository. They don't reread the entire codebase at every step. They keep notes: what they've already checked, what is broken, what they tried in vain. It's the same here, except the harness keeps the notes.

What the results were

On long-horizon benchmarks and tasks with large documents, a short window preserves accuracy and sometimes even improves it. The cost of GPU decoding drops by up to 7 times when moving from 256K to 16K tokens. Even in the harsh SAC mode, Claude Opus 4.5 solves 70.2% of SWE-bench Verified tasks.

It's also nice that the authors described the summary format. It is a six-part handoff brief: the goal, verified facts kept separate from hypotheses, changes made, checks run, dead ends, and a plan for next steps. Separating facts from hypotheses guards against a situation where a guess, after being retold, turns into an "established truth."

A skill for Claude Code

The paper comes with no code, so Zhemal Khamidun of gonzo-obzory asked Claude to build a skill that reproduces the approach. It lives in the short-context-agent repository. The skill teaches the agent not to load large files into context but to search them and read in windows, and to maintain that same brief on long multi-step tasks.

Context resets are still handled by Claude Code: auto-compaction at a threshold, or /compact and /clear by hand. The plugin's hook puts the brief back, and the agent continues from the same place. This yields the EoCC mode. SAC mode can't be reproduced in Claude Code; it exists only in the reference harness in the repository, alongside EoCC and a baseline without compaction. The author honestly warns that he did not test it carefully, bugs are possible, and PRs are welcome.

Limitations

First: a context reset breaks the prompt cache. Every new session starts with a cold prefix, so you may not save any money even if the GPU savings on decoding are real. The 7x gain applies to decoding cost, not to your bill from the provider.

Second: weak open models may collapse. If a model is verbose, it burns through the hard window budget before finishing the task. Results on Opus 4.5 cannot be automatically transferred to any model.

Third: the skill is unofficial and rough; it is not the authors' verified implementation.

What to do in practice

If you are building agents, check three things. Don't make the agent read large files in full; let it search and read in windows. Set up an explicit summary that separates facts from hypotheses. And calculate the real cost including cache resets, not just the window size. For self-hosted models the idea is even more interesting: training and running a model with a 16K–50K window is noticeably cheaper than chasing a million tokens.