SATE-Bench: Multimodal Models Quit Scene Exploration Too Early

Multimodal language models answer questions about images well, but in the real world a frame is rarely perfect. A person in a crowd is blocked by someone else, a hand is hidden behind a back, an object sits past the edge of a table. Tianchen Guo, Tengyue Zhang, Chen Liu, Ling Chen and Xin Yu have presented SATE-Bench, an interactive benchmark of 316 real-world scenes and 2525 multi-turn QA pairs. It tests not only what a model sees, but also whether it understands what it is missing. The main finding is an unwelcome one: current MLLMs abandon exploration too early, yet claim high confidence in their answers.

A hungry model and a sated answer

The benchmark's name is a pun. The initial frame is heavily occluded, so the model is "hungry" for information. To become "sated" (sate), it has to request neighboring viewpoints from an external, exocentric position, that is, as an outside observer rather than from a first-person view. The model itself decides when to stop.

This sets SATE-Bench apart from most spatial tests. Familiar benchmarks supply static images or a pre-assembled set of views. The model has to guess instead of admitting uncertainty. Here, three skills matter: working out where to look, combining what it has seen across dialogue turns, and knowing when to stop searching.

How the evaluation works

The benchmark combines two modes:

• Passive: the model receives several views of the scene at once. This is the control scenario.
• Active: the model starts from a heavily occluded frame and iteratively requests neighboring views.
• Efficiency metric: a reward with a penalty for every step taken, so endless browsing of views does not pay off.
• Scale: 316 real-world scenes with people and 2525 multi-turn question-answer pairs.
• Separate matched-view controls, in which views are aligned across modes to separate the effect of the amount of information from the effect of behavior.

The matched-view controls let the authors conclude that the gap between Active and Passive comes mainly from gathering too little evidence, not from an inability to work with multiple views as such.

Premature stopping and GRPO overfeeding

Testing current MLLMs revealed two problems. First, models end exploration too early while reporting a high level of confidence. In other words, they lack a calibrated estimate of their own uncertainty. Second, fine-grained 3D geometry tasks are especially hard for them, and a shortage of views hurts most there.

The authors tried to fix this by fine-tuning with Group Relative Policy Optimization (GRPO), rewarding requests for new views. The effect was mixed. Activity rose, and the models did explore scenes more often. But overall accuracy fell: they began to "over-explore" and failed to integrate the growing context of many views across turns. A simple reward for curiosity teaches neither information integration nor timely stopping.

The paper's overall conclusion: scaling passive multi-view perception does not by itself produce active spatial reasoning. This is an important signal for developers of embodied agents, assistant robots and video surveillance systems, where a camera or manipulator can change its viewpoint but every step has a real cost.

What developers should do

For teams building agents with visual perception, SATE-Bench is a ready-made diagnostic tool. It shows how a model behaves with incomplete data: whether it rushes to answer or drowns in unnecessary requests. The code and data are open on the project page and in the GitHub repository, so you can run the test on your own model without complicated setup.

The practical lesson concerns reinforcement learning. A reward for the number of view requests alone degrades quality. What is needed are functions that tie exploration to final accuracy, penalize extra steps and teach models to combine data from different turns. Calibrating confidence is a separate task. As long as a model is loudly confident in its answer about an occluded scene, it is too early to trust it in robotics and security.