Apple works out when on-policy distillation actually helps model training
Apple ML Research has published a study on the limits of on-policy distillation, a method that provides dense per-token guidance when training reasoning models. The question is not the method itself but when to use it. Which teacher model should you pick? Which context should you use for self-distillation? The optimal choice differs from token to token, but computing it in practice is expensive. Apple proposes a training-free approach to answer these questions without costly experiments.
AI-processed from Apple ML Research; edited by Hamidun News
Researchers at Apple ML Research have published an analysis of the limits of on-policy distillation, a training technique in which a teacher model gives reasoning models per-token guidance. Their conclusion: the method can be a powerful tool, but only if applied correctly.
What on-policy distillation is and why it matters
On-policy distillation is a way to train a new model by giving it very detailed instruction at every step. Picture a smart teacher solving a problem out loud and explaining each step, while the student learns not just from the final answer but from how the teacher reasoned along the way.
This matters especially for reasoning models, which solve complex problems step by step, unfolding their logic. OpenAI and other labs use similar approaches when training models such as o1 and Claude 3.5 Sonnet. At first glance, the more detailed the guidance, the better the student should learn.
When it really helps, and when it can hurt
Apple found that things are more complicated. The key questions remain open:
- Which model to choose as the teacher (stronger is not always better)
- Which context to use for self-distillation, when a model learns from itself
- The optimal choice may differ from token to token
The current approach to these questions is usually this: launch an expensive training run, spend several hours (or days) computing on GPUs, and look at the final metrics. The trouble is that these aggregate figures hide the truth: at the level of individual tokens, methods often behave very differently.
How Apple proposes to solve this
The research team presented a training-free approach: a way to assess the effectiveness of on-policy distillation WITHOUT costly experiments. It lets ML engineers see whether the method works in their specific scenario before launching a full training run.
Such tools are critical for large labs: every GPU-hour costs money, and being able to predict training outcomes in advance saves resources and development time.
What this means for the ML community
On-policy distillation remains a powerful technique, but the study shows it cannot be applied mechanically. Different tasks, different models and different data call for different decisions, and Apple has provided a tool for making those decisions on solid grounds, without costly trial and error.
- Oct 29, 2026Claude Code Edits Video From a Transcript: How the video-use Skill With 30 000 Stars Works
- Oct 29, 2026AI Verified Part of Signal's Code with Math in Half a Day for $500, Versus Five Months by Hand
- Oct 29, 2026Liquid AI releases LFM3: six multimodal 2B models with a 256K context window
- Oct 29, 20268B-Parameter Agent Operates iOS Apps at Gemini 3.1 Pro Level
Key Questions Answered
What on-policy distillation is and why it matters?
On-policy distillation is a way to train a new model by giving it very detailed instruction at every step. Picture a smart teacher solving a problem out loud and explaining each step, while the student learns not just from the final answer but from how the teacher reasoned along the way. This matters especially for reasoning models, which solve complex problems step by step, unfolding their logic. OpenAI and other labs use similar approaches when training models such as o1 and Claude 3.5 Sonnet. At first glance, the more detailed the guidance, the better the student should
When it really helps, and when it can hurt?
Apple found that things are more complicated. The key questions remain open: Which model to choose as the teacher (stronger is not always better) Which context to use for self-distillation, when a model learns from itself The optimal choice may differ from token to token The current approach to these questions is usually this: launch an expensive training run, spend several hours (or days) computing on GPUs, and look at the final metrics. The trouble is that these aggregate figures hide the truth: at the level of individual tokens, methods often behave very differently.
How Apple proposes to solve this?
The research team presented a training-free approach: a way to assess the effectiveness of on-policy distillation WITHOUT costly experiments. It lets ML engineers see whether the method works in their specific scenario before launching a full training run. Such tools are critical for large labs: every GPU-hour costs money, and being able to predict training outcomes in advance saves resources and development time.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.