Context Rot

Context rot is the degradation of a language model's effective reasoning and retrieval accuracy as its context window fills with text, causing it to underweight or lose track of information—especially content placed in the middle of long contexts.

Context rot is an informal but widely used term describing a cluster of failure modes that emerge when language models process very long input sequences. The most rigorously studied manifestation—documented in Liu et al.'s 2023 paper "Lost in the Middle"—showed that retrieval tasks exhibit a characteristic U-shaped performance curve: models recall information placed at the very beginning and very end of a context window reasonably well, but systematically underperform on facts located in the middle. This effect persisted even as context windows expanded from roughly 4,000 tokens to 128,000 tokens and beyond.

The root cause is architectural. Transformer attention can theoretically attend to any position, but in practice learned attention patterns concentrate on recently seen tokens and on tokens near the beginning, which receive strong positional signals during training. Information buried in the middle competes with a much larger noise signal from surrounding content. The quadratic scaling of standard attention with sequence length creates further practical pressures that efficient approximations do not fully resolve at very long contexts.

The term extends beyond positional retrieval to include broader coherence degradation: models following long system prompts with many instructions tend to forget or contradict earlier constraints by the end of a session; agentic systems accumulate errors in tool-call transcripts that misdirect subsequent steps; and long coding sessions in which many files are loaded into context see the model conflating variable names or function signatures from different files.

By 2026, leading model providers including Anthropic, Google DeepMind, and OpenAI had invested substantially in mitigations: improved positional encodings such as RoPE scaling and YaRN, training explicitly on longer sequences, and retrieval-augmented generation as an architectural workaround that keeps the active context short. Benchmarks such as RULER continued to reveal meaningful performance gaps relative to short-context settings, and practical prompt-engineering guides routinely advise placing the most critical instructions at the beginning and end of a prompt.