OpenVLHarness Gives Multimodal Models Memory and Tools: +9.7–12.6 Points

The authors of OpenVLHarness propose leaving a multimodal model's reasoning as it is and giving it external tools, plus a memory that holds the results of their work. Across 22 benchmarks and eight open and closed models, this harness raised the average score by 9.7–12.6 points compared with identical no-tool baselines. The most notable result concerns cost: GPT-6 Luna at medium reasoning effort beat the base model, Codex and Visual Sketchpad, all running at maximum effort, by 7.7–8.5 points. It used no more than 37% of their tokens.

Why reasoning alone isn't enough

The starting point of the work is a controlled experiment on the same 22 benchmarks. Simply increasing the base model's inference budget, that is, giving it more tokens to think, yields only a modest gain. The reason lies in the nature of the tasks. They require image-grounded perception, external knowledge and numerical measurements. Each of these comes from a different tool, and the model cannot derive them from its own weights no matter how long it thinks.

There is a second problem. A tool's output has to be converted into something the model can read, and then it must not be lost at the next reasoning step. This is where most agent pipelines break down: an image or a table appears once, and three steps later all that is left of it is a text summary.

How the harness works

OpenVLHarness is built around two layers and one interface.

• The capability layer combines task-level tool interfaces with evidence rendering. It provides unified access to heterogeneous backends and converts their outputs into observations the model can understand.
• Persistent multimodal memory stores images, structured data and web links as addressable artifacts for the duration of the whole request.
• The coding interface executes generated programs over the stored data, so the model can compute by combining the results of several tools.

Together, these turn answering into an iterative loop: obtain evidence, examine it, combine it with other evidence. The addressability of artifacts is the key detail here. The model refers to a specific image or dataset in memory by its address instead of dragging it around in the context in full.

The numbers

• Coverage: 22 benchmarks, eight open and closed backbones.
• Average gain over the matched no-tool baseline: 9.7–12.6 points.
• Qwen3-VL-8B: +11.2.
• Kimi K3: +9.7.
• GPT-6 Astra: +9.9.
• GPT-6 Luna at medium effort versus the base model, Codex and Visual Sketchpad at maximum effort: a lead of 7.7–8.5 points.
• The model's token consumption: no more than 37% of these competitors'.

The spread is worth noting. The compact 8-billion-parameter Qwen3-VL gains +11.2, more than flagship closed models such as GPT-6 Astra (+9.9). For smaller models, tools and memory partly make up for limited native capabilities.

What developers should do with this

The practical takeaway for anyone building visual agents: when quality falls short, first rethink the architecture around the model rather than raising reasoning effort. The comparison with Codex and Visual Sketchpad shows that medium effort plus well-organized tools can outperform maximum effort without them, at roughly a third of the token cost.

Three things are worth adopting right away. Convert tool outputs into a format the model reads without loss. Store intermediate images and data as addressable objects, not as text summaries. Let the model compute results with code over the stored data. There is one caveat: the figures come from the authors' own description, and the article includes no independent verification, so check them on your own tasks before adopting the approach.