Xiaomi-Lens-0, a 4B model, beats Qwen3.5 by 5.6 points on RefCOCO

Xiaomi-Lens-0 is a 4B-parameter multimodal language model built on Qwen3.5. It predicts object masks from its own visual representations and uses them as it reasons. Across all eight splits of RefCOCO, RefCOCO+ and RefCOCOg, it is on average 5.6 points more accurate than the base Qwen3.5 at understanding referring expressions. Among the compared models, it has the highest segmentation cIoU. The technical report was published on October 16, 2026.

The problem: text doesn't say where to look

MLLMs answer questions about an image as a whole reasonably well, but they struggle with small regions. The authors blame the way these models are trained. Text annotation records the correct answer but doesn't indicate which parts of the image support it. A model can guess right while looking at the wrong place, and the training signal won't notice.

The usual fix is to bolt on a separate segmentation model with its own encoder. Xiaomi took a different route and kept a single shared backbone.

How the architecture works

A lightweight decoder takes two inputs. The first is the patch features from the visual encoder. The second is the visual latents: the language model's hidden states on special mask tokens. The decoder combines the two to build a region mask. As a result, the gradient from the mask flows directly into the shared backbone, and no additional segmentation encoder is needed.

The key consequence is that every latent decodes into a mask. That makes it possible to insert region-bound latents into the reasoning process, in effect "thinking in visual latents." During chain-of-thought, the model refers back to areas of the image, and grounded reasoning traces provide supervision on exactly the regions it visited. Reasoning stops being text detached from pixels.

What the benchmarks show

The authors report the following results:

• Model size: 4B parameters, with Qwen3.5 as the base model.
• Referring comprehension: on average +5.6 points over Qwen3.5 across all eight splits of RefCOCO, RefCOCO+ and RefCOCOg.
• Segmentation: the highest cIoU among the compared models.
• Perception: outperforms Qwen3.5 on all perception benchmarks.
• No additional segmentation encoder is used.

The breadth of the result matters. The model improved not on one convenient split but on all eight, and it did not lose ground in general perception. Specializing in grounding often degrades general visual skills; the report shows no such effect here.

There are caveats, too. The abstract does not disclose which models make up the "compared models," how large the competitors are, or the absolute metric values. The 5.6-point gain is an average, and individual splits may differ noticeably. The "0" in the name hints at a first iteration, so this is more a statement of direction than a finished product. There are no independent reproductions yet.

Who can use it in practice

A model of this size can realistically run locally or on limited hardware. It suits tasks where showing matters, not just naming: dataset annotation, finding an object from a text description, UI agents that need to point to a specific screen element, and robotics pipelines that tie a command to an object in the frame.

Developers should check three things. First, how the model behaves on your data rather than on RefCOCO: that benchmark has been studied for years, and domain shift may eat part of the gain. Second, the time and memory cost of reasoning chains with regional latents. Third, weight availability and licensing: watch the project page at seerray-lab.github.io/Xiaomi-Lens-0 for both.

The idea of training segmentation through a shared backbone instead of attaching a second model looks portable. If the results hold up on other base models, the recipe could show up in open multimodal stacks beyond Xiaomi.