Self-Reflective Program Search: Apple Improves How LLMs Handle Long Contexts

Apple ML Research has published a study examining Self-Reflective Program Search, a method for running language models on long contexts. The study shows that recursively decomposing a context into manageable subqueries can significantly improve performance on this critical aspect of modern LLMs.

Why long contexts remain a problem

Extended context windows have become a standard feature of modern language models: Claude, GPT and Gemini promise to handle from 100K to 1M tokens. In practice, though, even with such huge windows, models often fail at the task. They lose information from the beginning or end of the context, link details incorrectly, and omit important facts when reasoning.

Researchers and users know the problem: a model can see the whole text but cannot reliably extract the information it needs and use it in its answer. This produces the so-called "lost in the middle" effect and other artifacts that show up most on truly long contexts.

Recursive

Language Models: breaking the complex into the simple

Apple proposes a solution built on the idea of Recursive Language Models (RLMs): instead of processing a long context whole, the model recursively splits the task into a chain of subqueries. This works like an automatic program for interacting with the context: the model itself decides which subqueries are needed, in what order to ask them, and how to assemble the results into a final answer.

The key idea of Self-Reflective Program Search is that there are many ways to decompose one complex task into subqueries, and the results differ greatly. Apple's study shows that some decompositions work much better than others. The search algorithm reflects on itself: it analyzes the results of earlier subqueries and adjusts its strategy on the fly.

Why recursive search works better

This approach solves several problems at once:

• Reduces cognitive load — the model works with smaller chunks of information instead of processing 100K tokens at a time
• Preserves information — explicitly tracking subqueries and results reduces the information loss that occurs in direct processing
• Allows the model to revise its decision — self-reflection helps the model fix errors and refine its answer after the first pass
• Optimizes the inference process — the model does not waste resources on unnecessary computation and directs attention to the relevant parts of the context

Apple's study shows that this approach is especially effective on long contexts (100K+), where traditional methods start to falter.

What this means

The results from Apple ML Research point to a promising direction for future language models. Instead of racing toward ever larger context windows, developers can invest in smarter ways of working with long contexts and still get better results. This may mean that in 2026–2027 we will see LLMs that not only see more information but can actually use it. For users, this means more reliable answers on tasks that require analyzing large volumes of information: summarizing long documents, searching archives, analyzing code, and working with scientific papers.