NVIDIA unveils Nemotron OCR v2: a multilingual OCR model trained on 12.2 million synthetic documents

NVIDIA has shown how it built Nemotron OCR v2, a multilingual OCR system whose main leap came not from clever architectural tricks but from synthetic data at large scale. The company assembled a dataset of 12.26 million artificially generated documents and used it to train a model that reads several languages with one engine and delivers up to 34.7 pages per second on a single A100.

A focus on data

The previous version, Nemotron OCR v1, worked confidently with English but failed on other languages. The SynthDoG benchmark showed that Normalized Edit Distance values for Japanese, Korean, Russian and Chinese were too high: in places the output barely resembled the source text. The team even expanded the character set from 855 to 14,244 to cover Cyrillic and CJK scripts, but the gain was small. The model could formally output the required characters, but it hardly saw them in training.

"The bottleneck was data, not architecture."

That became the turning point of the project. Manually labeling millions of documents with word-, line- and paragraph-level boxes would have been too expensive, and web-scraping PDFs yields a noisy text layer and plenty of errors. So NVIDIA took a different route: generate documents programmatically and know in advance the exact coordinates, transcriptions and reading order of every fragment.

How the corpus was built

For text, NVIDIA used mOSCAR, a large multilingual web corpus with 163 language subsets. That let it draw on neither dictionary lists nor machine-generated text, but on more realistic phrases with a natural distribution of words and characters.

As the rendering engine, the company took SynthDoG from the Donut project and reworked it heavily. The output is not just page images but full hierarchical annotation at the word, line and paragraph level, plus a relation graph that defines reading order.

Several important features were added to the pipeline to bring the synthetic data closer to real documents:

• multi-template layouts: columns, tables, vertical text, tables of contents, slides and Word-style pages
• annotation not only by boxes but by element hierarchy and links between lines
• a switch to line-level recognition for Japanese, Korean and Chinese, where word boundaries are often ambiguous
• a large pool of open fonts, from 165 to 1,258 per language, including Google Fonts and Noto families
• aggressive augmentations: shadows, outlines, noise, blur, distortions, and changes in brightness and backgrounds

The final dataset contains 12,258,146 examples across six language categories: English, Japanese, Korean, Russian, Simplified Chinese and Traditional Chinese. The multilingual model itself works as a single stack for English, Russian, Japanese, Korean and Chinese, without a separate language-detection step. The expansion logic is also simple: if a new language has a text corpus and suitable fonts, the pipeline can be scaled further without manual labeling and without rewriting the architecture.

Speed and trade-offs

Nemotron OCR v2 was trained not only on synthetic data but also on roughly 680 thousand real images. The architecture has three parts: a text detector built on RegNetX-8GF, a Transformer-based recognizer, and a relational module that works out which lines and blocks are connected. The key idea is that a heavy convolutional backbone processes the page once, and its features are then reused by all the other components. As a result, the system does not spend extra compute on each pipeline stage separately.

On the synthetic benchmark the improvement looks very strong. For Russian, NED fell from 0.564 for Nemotron OCR v1 to 0.043 for v2; for Japanese, from 0.723 to 0.046; for Korean, from 0.923 to 0.047; for Simplified Chinese, from 0.784 to 0.035. According to NVIDIA, the single multilingual version beat even the specialized per-language PaddleOCR variants on this set.

But on the real-world OmniDocBench the picture is more complicated: Nemotron OCR v2 reaches 34.7 pages per second versus 1.2 for PaddleOCR v5, a speedup of more than 28x, yet it trails the best competitors in accuracy on some subsets. Here the product clearly strikes a balance in favor of processing speed rather than maximum quality at any cost.

What this means

Nemotron OCR v2 is a good signal for the document AI market: synthetic data is no longer just a demo effect but a working way to launch multilingual OCR models quickly and extend them to new scripts. For companies, this means a cheaper path to document recognition, especially where speed, versatility and control over annotation matter more than an absolute record on every benchmark.