Where VLMs' Physical Self-Improvement Comes From: A Scaling Law for LLM Agents

Where VLMs'

Physical Self-Improvement Comes From: A Scaling Law for LLM Agents

Why some Vision-Language models handle physical tasks better than others, and exactly how they improve with experience, has remained an open question. An international team from Galbot (authors: Zekai Li, Jiayi Yu, Jinlu Zhang, Luyang Niu, Zekun Qi, Jun Zheng, Xuchuan Chen, Dairu Liu, Yintianrun Zhang, He Wang, Li Yi) has published a study that systematically breaks down the mechanisms of physical recursive self-improvement in Vision-Language agents.

The study introduces PhysRSI-eval — a methodology for quantitatively assessing physical RSI through a five-dimensional metric. Instead of a traditional binary outcome, the authors measure five aspects of learning: • First-Attempt Performance — success rate on the first try • Efficiency — the rate of performance growth • Success — the final share of successful attempts • Budget — the number of interactions consumed • Improvement from Failure — the ability to learn from mistakes

Experimental Setup and the Physical Barrel Hypothesis

The authors tested various Vision-Language models on 18 manipulation tasks under an identical interaction budget. Analysis of the results led to the formulation of the Physical Barrel Hypothesis: different VLM capabilities bottleneck different parts of the improvement process. Just as a barrel's capacity is determined by its shortest stave, different components of an agent — visual understanding, planning logic, control precision — become bottlenecks at different stages of learning.

The five-dimensional metrics revealed hidden differences: systems with similar average scores on public benchmarks can diverge sharply in their improvement profiles. High final scores guarantee neither fast nor efficient progress — traditional average scores obscure important differences in learning trajectories.

Scaling and Predictability

The study approached scaling from two angles. Observational scaling asks: can a VLM agent's relative performance on physical tasks be predicted from the model's known public capabilities? The second angle: can late-stage progress be predicted from early physical feedback — if an agent shows a particular error pattern in its first attempts, can its subsequent trajectory be predicted?

Together, these two analyses answer the core question: how fast and how far can an agent improve?

Workspace Interventions and the Architecture of Experience

The authors ran experiments manipulating conditions to understand what remains of improvement after intervention: • Removing interaction history reduced success rates in later rounds — history is critical for learning • Freezing the workspace configuration did not destroy the usefulness of experience on new scenarios — experience is partially transferable • Three cross-task cases achieved success earlier under certain conditions — multitasking accelerates learning

The key finding: adaptation to a specific scene and reusable experience exist in parallel, not in conflict.

Context Within the Recursive Self-Improvement Landscape

Hugging Face lists related works developing similar ideas: • PhysEvo: Astra Can Act, Let It (2026) • Agent Plasticity: Measuring Self-Improvement Through Experience (2026) • SoL-Pi: Recursively Scaling Auto-Research Loops for Efficient Agent Harness (2026) • Long-Horizon Scaling: How Model Capabilities Shape the Returns to Computation (2026) • HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness? (2026) • AReaL-RSI: Adaptive Scaling and Coordination of Recursive Self-Improvement (2026) • RSIGym: A Flexible Environment for Recursive Self-Improvement (2026)

The project is available at physrsi-eval.github.io with full code, data, and documentation.