SERA teaches a VLM planner to self-evolve: robot scores 51.38% on RoboMME

A team of researchers has introduced SERA (Self-Evolving Robotic Agent), a system in which a high-level vision-language model works as a full robotics agent, while a frozen VLA executor handles low-level control. The key element is an editable "harness" that stores task context and is rewritten over time based on experience. On the RoboMME benchmark, SERA with autonomous configuration selection reaches 51.38% overall success, compared with 44.38% without self-evolution. The difference is exactly 7 percentage points, or about 16% in relative terms.

Why a single look at the scene isn't enough for a robot

VLA models handle manipulation well because they combine visual understanding, language instructions, and action generation in a single policy. Problems start on long horizons. There, decisions often depend on history: what has already been done, which step failed, where an object that the camera can't currently see was placed. These are non-Markovian dependencies, and the current frame doesn't convey them.

Partial observability and accumulating execution errors make things worse. A small slip at step three turns into failure of the whole task by step ten. The industry is already trying two approaches: adding memory to policies built on foundation models, and building hierarchies in which planning is separated from execution. SERA combines both and goes a step further.

A harness that rewrites itself

The authors set up the high-level VLM as an agent. Around it sits an editable harness that maintains task context: progress, results of past interactions, and observations. The harness updates from observation data and execution feedback, and that updated information guides the next planning step. The VLA executor is left untouched and stays fixed.

Self-evolution is built on runs in the environment. The system proposes revisions to the harness, evaluates them on rollouts, and picks the best configuration. The chosen configuration is locked in before testing on a held-out set. This is an important detail: it means the final number is not tuned to the test tasks, and the selection is made autonomously, without manual tuning for the benchmark.

Key parameters of the work:

• Architecture: a high-level VLM agent plus an unchanged low-level VLA executor.
• Editable part: a harness with task context, updated from observations and feedback.
• Improvement mechanism: proposing and evaluating revisions on environment rollouts.
• Protocol: the configuration is selected and locked in before held-out testing.
• Benchmark: RoboMME, tasks that depend on history.
• Result: 51.38% with autonomous selection versus 44.38% without self-evolution.

Where the practical value lies

The main takeaway for teams working with robots: some of the gain can be had without retraining the VLA. If the executor is frozen, improvements come through the planning and memory layer. That layer is easier to version, roll back, and test than the weights of a large policy. For companies that already have a VLA model, this is a cheaper path than a new cycle of demonstration collection and fine-tuning.

The setup also shows how ideas carry over from the world of coding agents. There, the harness around a model, meaning instructions, memory, and tools, has long been recognized as a separate source of quality. SERA applies the same principle to physical tasks and adds an automatic search for the best version of the harness.

The numbers themselves also show the limitations. A score of 51.38% means that roughly half of the episodes on RoboMME still end in failure. A 7-point gain confirms the direction but does not solve the problem of long tasks. The authors themselves word their conclusion cautiously: agentic planning looks like a promising path for manipulation that depends on history. For practitioners, the next question is how this approach will behave with other executors and in real, rather than simulated, scenes, where execution errors cost more.