Live Nov 18 · Cohort 3Nov 18, 19:00 MSK — free hands-on AI agents live workshop · Live Cohort 3 starts Nov 23Join Nov 18 Live →Cohort 3 (Nov 23)
ReleaseInsight · On RadarHabr AI→ original

99 Claude Code Compactions Analyzed: 35% of the Details the Agent Needed Never Made It Into Context

When Claude Code runs out of context, the model retells the conversation, and the agent carries on from that retelling. The author analyzed 99 compactions in his own logs. Of the 506 details (paths, links, hashes) the agent returned to, 179 (35%) were not in the context it was handed. His first estimate of 72% losses turned out to be a script error, and agreements survived almost every time. Ask the summary to list paths and links, and keep prohibitions in project rules.

AI-processed from Habr AI; edited by Hamidun News
99 Claude Code Compactions Analyzed: 35% of the Details the Agent Needed Never Made It Into Context
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
Agreements243 of 265 survived in full, 19 partially, 3 were lost
Commit hashes7 of 66 lost; all 14 merge request numbers were found
Paths, links, task numbers172 of 426 not found (about 40%)

When Claude Code runs out of context, the model retells the whole conversation, and from then on the agent works from that retelling. This is what compaction means. Imagine you spent an hour editing src/config.

rs and agreed: "push only after my approval." After compaction, the retelling should preserve both the file and the prohibition. A developer at Lamantin AI, who builds kaeru (memory for agents), checked his own logs to see what actually gets lost.

The result: of 506 details the agent returned to after compaction, 179 (35%) could not be found in the context it was given. But the main conclusion is different: the first, far scarier numbers turned out to be a counting error. For a developer who works with coding agents every day, this is a practical guide to what to ask of the summary and what to keep outside the conversation.

What the agent gets after compaction

According to Claude Code logs, the agent has three sources after compaction: the summary (the goal, what was done, which files were touched, where work stopped), a few of the most recent messages verbatim, and recently read files that Claude Code attaches afterward. If a detail is in none of them, the agent has to look it up again, reconstruct it from indirect clues, or pull it from external memory, if one is connected.

The data: 231 Claude Code sessions from April to early November, plus Codex sessions from November. After removing duplicates, there were 67 compactions in Claude Code and 32 in Codex, 99 in total. Duplicates (38 were found) appear when a resumed session copies the history together with the old compaction boundaries. Of the 67 Claude Code compactions, 64 were automatic, and the median context size before them was about 973 thousand tokens. In Codex the summary is encrypted, so the author checked its contents only for Claude Code.

How 72% losses turned into 35%

The first count was crude: take every path, link, hash and number that appeared both before and after compaction, and check whether it was in the summary and the preserved messages. 955 of 1 330 were not found, or 72%. But that pile included junk: a path from a stack trace could flash by in command output both before and after compaction even though the agent never needed it.

And some of the "losses" were sitting right in the summary. A telling case: after compaction the agent re-read a config file by its full path, and the script logged it as a loss. But the summary contained the line "Edited src/config.

rs: added CommandSpec struct." The mistake was not the agent's but the regular expression's, which skipped short paths.

The author then narrowed the question: count only details that appeared before compaction and that the agent itself wrote afterward, in a command or a reply. That left 506 details:

  • 300 (59%) were found in the summary or the preserved messages;
  • 27 (5%) were in the attached files;
  • 179 (35%) were not found in the context it was given.

The gap between types is large. All 14 merge request numbers were found, 7 of 66 commit hashes were lost, and paths, links and task numbers were not found in 172 of 426 cases. The author speculates that the model readily writes a short number into the retelling but replaces a long path with words like "edited the config," though he did not test this guess.

An important caveat: 35% does not mean "the agent forgot." A detail the agent forgot and never mentioned again does not show up in this count. It is the share of details the agent returned to but the script could not find in the context. Of the 179 details, 82 first appeared after compaction in Bash, PowerShell or Read output, one appeared in a kaeru response, and the remaining 96 the agent wrote with no visible source in the log.

Agreements and a "violation" that never happened

String matching does not work for meaning, so for 60 compactions the author extracted decisions from the last few exchanges ("push only after my approval," "don't touch this file"), which gave 298. A language model labeled them in two independent passes, and only the matching ratings went into the final tally. Of 265 decisions, 243 survived in full, 19 partially, and 3 were lost, for example "log Friday's hours." The data does not say what happens to an agreement made a hundred calls before compaction.

Of the 152 decisions where compliance was visible, four were labeled "violated." The most striking case: after compaction the agent appeared to run a script without permission. But the log contained the author's "go," said before compaction yet after a request to wait, and the first 30 calls of the new run did not include it. Once again, the checking method failed, not the agent.

A side finding: the "check memory" reminder in the kaeru installer fired on session start, resume and clear, but not after compaction. The fix is already in the main branch. For 14 of the 179 details, kaeru had successful records from before compaction; for one of them (hash 12ce9e1), the author confirmed the note was in memory at the time of compaction. Whether it would have helped the agent was not tested: that needs a separate experiment with identical tasks with and without memory.

What to do in practice

  • Ask the summary to list paths, links and commits. When running /compact manually, add: "list the file paths, links and commits we worked with."
  • Keep hard prohibitions outside the conversation: in project rules or in a hook. "Don't push without confirmation" survives compaction, but this is more reliable.
  • Add compact to the SessionStart hook matcher (startup|resume|clear|compact) so the agent gets a reminder to check memory after compaction.
  • Before blaming the agent, check the summary and the order of messages before compaction.

Limitations: these are one developer's logs, models and client versions changed along the way, paths were compared by their last two segments, and the decisions were labeled by a model. The share of repeated calls after compaction did not rise: in Claude Code it was 5.1% after versus 6.2% before, in Codex 1.5% versus 4.9%. This check cannot catch a repeat of old work with different arguments. To find compactions in your own logs, search for compact_boundary in the JSONL files in ~/.claude/projects; in Codex, look for a compacted entry with a replacement_history field.

Story

Key Questions Answered

What the agent gets after compaction?

According to Claude Code logs, the agent has three sources after compaction: the summary (the goal, what was done, which files were touched, where work stopped), a few of the most recent messages verbatim, and recently read files that Claude Code attaches afterward. If a detail is in none of them, the agent has to look it up again, reconstruct it from indirect clues, or pull it from external memory, if one is connected.

How 72% losses turned into 35%?

The first count was crude: take every path, link, hash and number that appeared both before and after compaction, and check whether it was in the summary and the preserved messages. 955 of 1 330 were not found, or 72%. But that pile included junk: a path from a stack trace could flash by in command output both before and after compaction even though the agent never needed it. And some of the "losses" were sitting right in the summary.

What are the key details on Agreements and a "violation" that never happened?

String matching does not work for meaning, so for 60 compactions the author extracted decisions from the last few exchanges ("push only after my approval," "don't touch this file"), which gave 298. A language model labeled them in two independent passes, and only the matching ratings went into the final tally. Of 265 decisions, 243 survived in full, 19 partially, and 3 were lost, for example "log Friday's hours." The data does not say what happens to an agreement made a hundred calls before compaction.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Hands-On AI Engineering · Hamidun Academy

Build & Ship Production AI Agents Without Boilerplate Code

Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.

Start 72h Free Trial →Vibe-Coding Intensive
What do you think?
Loading comments…