Self-Reflective Program Search: Apple improved LLM performance with long contexts

Apple ML Research has published a study examining the Self-Reflective Program Search method for language models working with long contexts. The research demonstrates that recursive decomposition of context into manageable sub-queries can significantly improve the efficiency of this critical aspect of modern LLM performance.

Why Long Contexts Remain a Problem

Extended context windows have become a standard feature of modern language models: Claude, GPT, and Gemini promise processing from 100K to 1M tokens. However, in practice, even with such massive windows, models often struggle with the task. They lose information from the beginning or end of the context, incorrectly link details, and omit important facts during reasoning.

The problem is well-known to researchers and users: a model can see all the text but fails to reliably extract the necessary information and use it in its response. This creates the so-called "lost in the middle" effect and other artifacts that are especially apparent on truly long contexts.

Recursive Language Models: Breaking Down Complexity

Apple proposes a solution based on the concept of Recursive Language Models (RLMs): instead of processing a long context as a whole, the model recursively breaks the task into a sequence of sub-queries. This resembles an automated program for interacting with context — the model itself determines which sub-queries are needed, in what order to pose them, and how to assemble the results into a final answer.

The key idea behind Self-Reflective Program Search is that many ways exist to decompose a single complex task into sub-queries, and the results vary significantly. Apple's research shows that some decomposition strategies work much better than others. The search algorithm is self-reflective — it analyzes the results of previous sub-queries and adjusts its strategy on the fly.

Why Recursive Search Works Better

This approach solves several problems simultaneously:

• Reduces cognitive load — the model works with smaller chunks of information instead of processing 100K tokens at once
• Preserves information — explicit tracking of sub-queries and results reduces information loss that occurs during direct processing
• Enables solution refinement — self-reflection helps the model correct errors and refine the answer after the first pass
• Optimizes the inference process — the model does not waste resources on unnecessary computations but directs attention to relevant parts of the context

Apple's research shows that this approach is particularly effective on long contexts (100K+), where traditional methods begin to struggle.

What This Means

Apple ML Research's findings offer a promising direction for future versions of language models. Instead of racing toward ever-larger context windows, developers can invest in smarter ways of working with long contexts — and achieve better results in the process. This could mean that in 2026–2027 we will see LLMs that not only see more information but also truly know how to use it. For users, this translates to more reliable answers on tasks requiring analysis of large volumes of information: summarizing long documents, searching archives, analyzing code, and working with scientific papers.