MIRAGE anchors video-based 3D reconstruction to LiDAR and beats COLMAP on three datasets

A research group has published MIRAGE on arXiv, a method that combines feed-forward 3D models, which work on a stream of images, with metric motion from LiDAR and an inertial sensor (IMU). The output is a trajectory in real-world meters and a highly detailed scene reconstruction in the form of Gaussian splats. The team tested it on three datasets: Oxford Spires, M2DGR and INS. According to the authors, MIRAGE estimates the metric trajectory more accurately than COLMAP on all three. COLMAP is a computationally expensive reference for vision-only benchmarks. The abstract gives no specific error values; readers will have to look in the paper itself.

Two halves of one problem

Each of the two approaches has its own weakness. Feed-forward 3D models produce high-quality geometry from a sequence of frames in a single pass, without long optimization. But they have no reliable metric scale, and on long routes the trajectory drifts: error accumulates step by step.

LiDAR-inertial estimation is the opposite. It measures motion in meters accurately, but it does not provide consistency in image space. That is not enough for photorealistic reconstruction, including on Gaussian splats: the geometry comes out correct but visually "unstitched" from the frames.

MIRAGE combines the strengths of both halves.

How the anchoring works

The system pins the visual geometry from the feed-forward model to the metric LiDAR-IMU motion. Online pose corrections are not applied blindly. Their weight depends on two signals: how many correspondences between frames actually support the correction, and how well the visual and LiDAR-inertial modalities agree. When the image offers weak support, for example on a featureless wall or under motion blur, the system relies on the metric motion. When the visual data is rich and consistent, it uses that data to refine the pose.

The mechanism is expected to keep a poor visual estimate from corrupting the trajectory and to stop drift from building up. Reconstruction quality is preserved at the same time.

A new indoor benchmark

The second part of the work addresses a shortage of data. There are almost no long monocular sequences of large indoor spaces with a reliable ground-truth trajectory. The authors built a multimodal data-collection platform. They then used MIRAGE to produce accurate 6-DoF reference trajectories for a new vision-only benchmark. The trajectories are built with LiDAR and IMU, while the benchmark evaluates methods that use only a camera.

On this dataset, the authors compared state-of-the-art methods for large-scale monocular reconstruction and gave a systematic evaluation of how they behave on long indoor scenes.

Key facts from the abstract:

• Input: an image stream plus LiDAR and IMU
• Output: a metric 6-DoF trajectory and a reconstruction in Gaussian splats
• Datasets: Oxford Spires, M2DGR, INS
• Comparison baseline: COLMAP
• New contribution: a multimodal platform and a vision-only benchmark for long indoor scenes

Where it could be useful

For robotics and mapping, the method is useful as a way to get ground-truth labels without manual work. For a mobile robot with LiDAR, an IMU and a camera, MIRAGE provides a metric pose and a dense scene suitable for simulation, digital twins and indoor inspection. For teams developing vision-only SLAM and reconstruction, the new benchmark offers a test on long indoor routes, where drift typically shows up.

The limitations are also clear. MIRAGE itself requires LiDAR, so the method does not replace camera-only solutions on cheap hardware. It helps evaluate and train them. Before adopting it, it is worth waiting for the code and the full per-dataset error tables, since the abstract compares against COLMAP only qualitatively.