Live Nov 18 · Cohort 3Nov 18, 19:00 MSK — free hands-on AI agents live workshop · Live Cohort 3 starts Nov 23Join Nov 18 Live →Cohort 3 (Nov 23)

NVIDIA releases Déjà View: a 117M-parameter model for 3D reconstruction from photos

If you have ever built a 3D scene from a set of photos, you know the drill: you need camera poses, depth, and usually a heavy pipeline. NVIDIA, ETH Zurich and the University of Toronto have released Déjà View, a 117M-parameter model that does all of it in a single pass. The trick is that instead of a stack of different layers, one transformer block is run in a loop. The number of loops can be changed at inference time.

AI-processed from TG: @cgevent (Метаверсище и ИИще); edited by Hamidun News
NVIDIA releases Déjà View: a 117M-parameter model for 3D reconstruction from photos
Source: TG: @cgevent (Метаверсище и ИИще). Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceTG: @cgevent (Метаверсище и ИИще) ↗
Editor-in-Chief
Key metric #1117M parameters, 8–10x fewer than VGGT and Pi3
Key metric #2The number of cycles K trades speed against accuracy without retraining
Key metric #3Code and nvidia/dvlt weights are open; runs via Hugging Face Accelerate

Imagine you have filmed a room on your phone or taken a hundred photos of a building, with no annotations of any kind. To turn that into a 3D model, you need to work out where the camera stood in each frame, what its parameters are, and how far each point in the scene is from it. This is the classic Structure-from-Motion problem, and over the past couple of years it has increasingly been solved with large transformers such as VGGT, Depth Anything 3, MapAnything and Pi3.

They work well, but they have grown to billions of parameters. NVIDIA Research, together with ETH Zurich and the University of Toronto, proposes a different route: Déjà View (DVLT) does the same job with 117M parameters.

One block instead of a stack of layers

The authors noticed that the layers of deep 3D transformers essentially repeat the same operation: step by step, they refine the geometry and camera poses. So why store hundreds of millions of unique weights for nearly identical work?

In Déjà View, the deep stack is replaced by a single transformer block with shared weights, which is run in a loop. The block combines per-frame attention (what is happening within a single image) and global attention (how the frames relate to one another). The analogy is simple: instead of ten different editors who each go over a text once, one editor reads it ten times in a row.

At the output, the model simultaneously predicts a ray map, a depth map and camera poses, both extrinsic and intrinsic. The result is a full reconstruction from unordered photos or video.

A speed-versus-accuracy dial

The main practical finding here is not the size but a side effect of the loop. The number of steps K becomes a compute knob at inference time. You train the model once, then decide for yourself how many loops to give it.

A few steps suit fast, real-time streaming tracking. A deep run (K around 12–16) is for when clean geometry and accurate camera calibration matter. You need neither separate checkpoints for each mode nor fine-tuning: one set of weights is enough.

The authors say that on five popular reconstruction benchmarks, including ScanNet++ with indoor and outdoor scenes, the model holds its own against 1B+ transformers or outperforms them, while requiring comparable or noticeably less compute. The post gives no specific metric figures, so check the paper on arXiv (2605.30215) for them.

What to do with it in practice

NVIDIA has released the code, the preprocessing pipeline, Rerun visualization scripts and the nvidia/dvlt weights on Hugging Face. The model runs through Hugging Face Accelerate in BF16. For a team, this means you can experiment in an evening, without training anything yourself.

Where it could be useful:

  • prototypes of AR and spatial applications that need camera poses and depth from ordinary video;
  • photogrammetry and scene scanning, when you have neither expensive sensors nor time for manual annotation;
  • starting data for other 3D pipelines that need camera poses as input. This is our own assumption, not a claim by the authors, and it needs to be tested;
  • tasks on limited hardware: a compact model fits into memory more easily than its billion-parameter counterparts.

Limitations and caveats

First, small size does not mean instant results. Shared weights save memory and checkpoint size, but every extra loop costs compute. How many seconds that is on your GPU for your number of frames, you will have to measure yourself.

Second, the comparisons with VGGT, Depth Anything 3 and Pi3 were run by the authors themselves, on benchmarks they chose. On your data, for example dark, shiny or nearly textureless scenes, results may differ.

Third, the post calls the license "permissive", but before commercial use it is best to check the exact license type for the code and the weights in the repository and on the model page.

Bottom line: Déjà View shows that the race for size is not necessary for 3D reconstruction. A recurrent block with an adjustable number of steps gives developers control over the balance of speed and accuracy, and the open weights let you verify this at little cost. If you work with 3D from photos or video, the model deserves a spot on your shortlist for a quick benchmark.

Story

Key Questions Answered

What are the key details on One block instead of a stack of layers?

The authors noticed that the layers of deep 3D transformers essentially repeat the same operation: step by step, they refine the geometry and camera poses. So why store hundreds of millions of unique weights for nearly identical work? In Déjà View, the deep stack is replaced by a single transformer block with shared weights, which is run in a loop. The block combines per-frame attention (what is happening within a single image) and global attention (how the frames relate to one another).

What are the key details on A speed-versus-accuracy dial?

The main practical finding here is not the size but a side effect of the loop. The number of steps K becomes a compute knob at inference time. You train the model once, then decide for yourself how many loops to give it. A few steps suit fast, real-time streaming tracking. A deep run (K around 12–16) is for when clean geometry and accurate camera calibration matter. You need neither separate checkpoints for each mode nor fine-tuning: one set of weights is enough.

What to do with it in practice?

NVIDIA has released the code, the preprocessing pipeline, Rerun visualization scripts and the nvidia/dvlt weights on Hugging Face. The model runs through Hugging Face Accelerate in BF16. For a team, this means you can experiment in an evening, without training anything yourself.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…