ProD Stops AI Agents From Looping: Tested on 8 Models (0.6B–8B)

Small language model agents get stuck in loops — repeating the wrong strategies and losing progress on complex tasks with long reasoning chains. This becomes a critical issue when distilling the capabilities of large models into smaller ones through supervised fine-tuning: the student learns fast, but can't escape local minima and ends up looping. The problem is equally present across model sizes, from 0.6B to 8B parameters.

Researchers from Pennsylvania State University and Microsoft CoreAI, in a NeurIPS 2026 paper, propose Progress-Aware Distillation (ProD) — a method that explicitly penalizes looping in the student's loss function. The idea rests on a simple principle: the progress gap between the small model and the large model is a signal of stagnation, and it can be used to adaptively penalize the student.

How ProD Works

At every step of the agent's interaction with the environment, the algorithm compares the student's progress to the teacher's progress: • If the student keeps pace with the teacher — the penalty is minimal • If the student lags but is still moving — the penalty is moderate • If the student is completely stuck (not moving, while the teacher would be) — the penalty is maximal

This gap is encoded into an adaptive margin that changes dynamically. Then comes iterative distillation: the student trains with the new loss, improves slightly, and on the next iteration the stagnation measure is recalculated based on the student's improvements.

Experiments: 8 Models, Two Types of Benchmarks

Testing covered: • By size: 0.6B parameters, 1B–3B parameters, 3B–8B parameters • By domain: in-domain (tasks from the training distribution) and out-of-domain (new domains)

In-domain testing shows how well the model learned the method. Out-of-domain testing checks whether it will work in real conditions on unfamiliar tasks.

Results

ProD stabilized agent behavior across all eight models:

1. Reduced looping: the percentage of trajectories where the agent repeats mistakes dropped significantly 2. Higher success rate: the share of correctly solved tasks (task success rate) increased 3. Efficiency: agents need fewer steps to solve tasks (lower token consumption) 4. Generalization: the effect held up on out-of-domain benchmarks, which is unusual for distilled models

The last point is particularly important: distilled models typically fail to generalize to new domains. Here, quality was preserved, suggesting that ProD teaches a fundamental skill — not looping — rather than simple memorization.

Availability and Authors

The paper has been published in full. The code is open-sourced on GitHub: https://github.com/CaoYuanpu/ProD

Authors: Yuanpu Cao, Saket Sathe, Hanyu Wang, Ziyi Yin, Fenglong Ma, Jinghui Chen (Pennsylvania State University and Microsoft CoreAI)

The author team welcomes discussion and is ready to answer questions.