ImagingBench exposes Gemini, GPT and Qwen weaknesses in physics-based image processing tasks

Researchers have introduced ImagingBench, a systematic benchmark for evaluating vision-language models (VLMs) and agentic AI on computational photography and image processing tasks. The benchmark includes 20 tasks across five categories: ray optics and wave optics, image signal processing, inverse reconstruction, computational sensing and calibration.

What the benchmark showed

Tests of leading proprietary and open multimodal systems, including Gemini 3.1, GPT-5 and Qwen, showed that agentic models consistently lose to specialized methods. The gaps are especially visible in computational sensing tasks such as lensless imaging, event-based reconstruction, time-of-flight imaging and holography.

• 20 computational photography tasks in the benchmark
• Gemini, GPT-5, Qwen and other VLMs were tested
• Agentic guidance (Planner) delivered only modest improvements
• Results were visually plausible but physically inaccurate

The gap between semantics and physics

The models often generate visually acceptable results that look right to the human eye but score poorly on reference-based metrics. This reveals a huge gap between a model's ability to handle semantic visual tasks (what they see) and physically grounded imaging (how physical systems work).

Next steps

ImagingBench provides a unified testbed for measuring progress of agentic AI in computational photography. It lets researchers systematically track how well current LLMs improve at understanding and solving physical problems, as opposed to purely semantic ones.

What this means

The findings show that current LLMs handle semantic visual tasks well but remain poorly adapted to physically grounded problems. That opens new directions for improving models and points to the need for specialized approaches in computational photography, even in the era of general-purpose AI agents.