LVSPM Beats VGGT at Camera Pose Estimation on 16–256 Frames and Doesn't Break Down on Large Scenes

Researchers have introduced LVSPM, a model that takes a set of uncalibrated photos and solves two tasks in a single pass: it estimates camera poses and synthesizes novel views of the scene. The input range is 16 to 256 frames. On three benchmarks (RealEstate10k, Co3Dv2 and DL3DV), the authors claim it beats VGGT on pose accuracy and, in view synthesis, surpasses even models that are given camera poses in advance, as measured by PSNR. The code is published.

Two Tasks Instead of a Pipeline

The classic route to novel views looks like this: structure-from-motion first computes camera poses, then NeRF or Gaussian splats reconstruct the scene from those poses. Errors from the first step carry over into the second. LVSPM merges the stages into a single feed-forward model. It takes images as they are and outputs poses and novel views, with no per-scene optimization.

The training requirements are modest. It needs only RGB frames and pose annotations. It does not use dense 3D geometry (depth maps, point clouds), which competing approaches often require. Such data is expensive to collect at scale, while videos with known camera trajectories are noticeably more common.

TTT Layers and Long Sequences

The main engineering idea is test-time training layers. The attention that models like VGGT are built on grows quadratically with the number of frames. TTT layers store context in rapidly updated weights, so, according to the authors, the model scales to hundreds of input views without a sharp rise in cost. The abstract gives no detailed memory or speed measurements; look for them in the paper itself.

The evaluation used 16, 32, 64, 128 and 256 views. On pose estimation, LVSPM leads VGGT across the whole range, with the biggest gap at strict thresholds. That detail matters. Many methods can get a pose roughly right, but reconstruction needs accuracy down to degrees and centimeters.

A Protocol Where Baseline Methods Fall Apart

The authors put particular emphasis on their view-synthesis evaluation protocol. They call it practical: the more frames, the larger the scene area. Familiar benchmarks with a fixed, small area hide a problem that shows up when you actually capture a house or a street. Under this protocol, LVSPM delivers the best quality among pose-free methods and maintains it as the scene grows, while baseline methods, in the authors' words, "collapse."

The PSNR result also stands out: the model beats pose-dependent methods, meaning those that get extra information as input. This claim will need to be rechecked through independent runs, but the direction is clear.

What the Abstract Tells Us

• Input: uncalibrated image collections, 16–256 views.
• Output: camera poses and novel views of the scene, both from one model.
• Training: RGB only and pose supervision, no dense 3D ground truth.
• Architectural technique: test-time training layers for long contexts.
• Benchmarks: RealEstate10k, Co3Dv2, DL3DV.
• Comparison: better than VGGT on poses, especially at strict thresholds; SOTA among pose-free methods on synthesis quality.
• Code: open, available on the project page.

Who Will Find This Useful

For teams working with photo and video capture for real estate, mapping, drones or digital twins, the value is that a separate calibration and SfM stage on hundreds of frames goes away. A practical test is simple: take the open code, run your own footage with camera shake, poor lighting and repetitive textures, and compare the poses against COLMAP or VGGT output. That kind of data usually shows whether generalization holds up beyond academic datasets.

Researchers will find the shift to TTT as an alternative to attention in 3D tasks interesting. If other groups confirm the gains on long sequences, the approach will quickly show up in related reconstruction models. For now, this is a preprint without peer review, and the abstract gives no key PSNR or pose-error figures. Those need to be checked in the paper's tables and in reproduction runs.