VICO Trains VLMs on Rewritable Images: Up to +5.0% on New Tasks

The authors of arXiv preprint 2610.10782 propose VICO, a post-training scheme for vision-language models in which an environment "rewriter" is trained together with the model itself. On nine multimodal benchmarks for mathematical and visually grounded reasoning, VICO-8B beats the base model by up to +5.0% on out-of-domain tasks. It outperforms the strongest self-evolution baselines by 4.3% and baselines with text-based task editing by 8.4%. Against chart-specialized RLVR methods it stays on par, while using 16–160 times fewer labeled examples.

The underlying problem is familiar to anyone who has run RLVR (reinforcement learning with verifiable rewards). The task set is fixed, but the model keeps improving. Some examples become trivial, others stay unsolvable, and the useful training signal collapses. The classic remedy is to buy more annotation or collect new data by hand. The authors propose something different: change the environment to follow the actor.

How VICO works

The loop has two trainable components. The first is the actor, i.e. the VLM itself. The second is the Environment-as-Rewriter (EnvRewriter). It does not touch pixels directly. Instead, it edits the verifiable structures that an image is generated from: scene graphs, the data tables behind charts, and masks of protected regions. The image is then redrawn from scratch.

This keeps the answer valid by construction. If a value in a chart's table is changed, the correct answer is recomputed from the same table, and no human is needed. The EnvRewriter's reward is based on the actor's share of successful solutions (pass rate). The editor is rewarded for tasks that sit at the edge of the model's current abilities: neither too easy nor hopeless. Difficulty is continuously pulled toward the actor's level.

This sets VICO apart from the two neighboring approaches the authors compare themselves with. Self-evolution methods improve the model on its own data. Co-evolution via text editing changes the wording of tasks, whereas VICO rebuilds the visual side of the task.

What the numbers show

The abstract reports the following:

• 9 multimodal benchmarks: mathematical reasoning and visually grounded understanding;
• VICO-8B beats the base model by up to +5.0% on out-of-domain tasks;
• +4.3% over the strongest self-evolution baseline;
• +8.4% over the baseline with text-based co-evolution;
• parity with chart-specialized RLVR methods with 16–160 times fewer labeled samples;
• no additional human annotation is used in the loop.

The caveats are obvious. This is a preprint without independent verification, and "up to +5.0%" is the maximum across benchmarks, not the average gain. The abstract does not name the base model and says nothing about the compute cost of training two components jointly. The "16–160 times" spread also shows that the savings on annotation depend heavily on the specific competitor.

What teams can try

The core idea carries over beyond the paper: if you have data with a structural source of truth (tables behind charts, scenes, diagrams, region annotations), you can build a task generator on top of it with guaranteed correct answers. Instead of a one-off dataset, you get a controllable environment.

The practical minimum looks like this. First, measure the pass rate distribution across your training examples. If most tasks sit near 0 or near 1, static RLVR has already hit its ceiling. Next, pick a narrow class of data with verifiable structure, such as charts, and test an editor that keeps difficulty within a "working window" of successes. Finally, evaluate the result on out-of-domain sets, since that is where the authors report the main gain.

The limitation is also clear. The approach requires images that can be honestly redrawn from a structure. For arbitrary photographs without a scene graph or table, this path is closed, and it is unclear how well VICO scales to other types of visual tasks.