NVIDIA Open-Sources Open-H3-Regenerate-2K-IC: Upscaling Video to 4K Without Face Drift

If you have ever run generated video through an upscaler, you know the effect. The picture got sharper, but the hero's chin drifted, the lips live separately from the speech, and the sign in the background turned into a mush of pseudo-letters. NVIDIA has open-sourced the architecture and code of Open-H3-Regenerate-2K-IC, which targets exactly this problem.

What was broken

Previous approaches to this kind of upscaling worked like this: take a low-quality draft, mix noise into it, and "regenerate" it at high resolution in three diffusion steps. The problem is that noise erases part of the information. The model no longer sees exactly what the face looked like, where the eyes were looking, or which letters were on the sign. It fills in details to its own taste, hence the drift in facial features, broken articulation, and blurry text.

Imagine a restorer who was given a coffee-stained photocopy instead of the original. They will paint something beautiful, but not necessarily what was there.

What changed in the IC architecture

IC stands for In-Context. The draft is not noised. It goes in its entirety, in clean form, into the context of the Ref2VA transformer, while the highly detailed video is generated from noise next to it. At each diffusion step, the model uses the attention mechanism to refer directly to the draft's undamaged tokens.

Back to the restorer: now the original lies on the table, and they check against it with every brushstroke. What exactly to paint, the model decides itself, but what was there originally, it sees without distortion.

According to the announcement, the scheme works with clips from 5 to 15 seconds long. The input is 384P and 768P drafts, the output is 2K and 4K. Diffusion runs in 3 steps at full resolution, and VAE decoding, according to the authors, is 5 times faster than in MiniMax's closed API. An important nuance: the number of steps (three) stayed the same as before. The gain is not in speed as such, but in the fact that the same three steps now rely on an untouched source.

What to do with this in practice

The main point: proprietary MiniMax upscaling now has an open alternative. The code is in the NVlabs/Sana repository, and the implementation is built on H3 weights and the open SoL-Refiner framework. If your product currently pays for a closed upscaling API, it is worth running your own clips through this model and comparing the results.

It is best to test on the most awkward shots:

• close-ups with speech, to check lips and facial expressions;
• scenes with signs, titles, and interfaces, to check text;
• longer clips closer to 15 seconds, to see whether drift accumulates.

The principle itself is interesting for product teams and studios. If you build a "fast low-resolution draft, separate final render" pipeline, this upscaling lets you iterate cheaply on drafts without losing character recognizability in the output.

Limitations

All figures so far come from NVIDIA's announcement and a post in a channel. I have not seen independent benchmarks or comparisons with MiniMax on third-party data. How much VRAM is needed for 4K and what license the weights carry are not stated in the materials; this must be checked in the repository before building the model into production.

In addition, an in-context scheme by definition pulls both the draft and the generated tokens into the transformer. For long clips and high resolutions, this usually means noticeable memory requirements. How noticeable it is here, only tests will show.

Bottom line: the idea is clear and right, if not revolutionary. Don't mix the model up with the source; let it look at the source. If the claims are confirmed, open video pipelines will gain a serious argument against a paid API.