DeepSeek-OCR Compresses Text Context 10x at 97% Accuracy

DeepSeek has published a paper and open-sourced the code for DeepSeek-OCR, a system that turns text into an image and uses visual tokens as a compressed representation of context. According to the authors, at roughly 10x compression the decoder recovers the original text with about 97% accuracy. At 20x compression, accuracy drops to around 60%. A popular breakdown on the Two Minute Papers channel calls this a solution to AI's memory problem. That is a loud claim: the problem has been framed rather than closed, though the document numbers are convincing.

Why turn text into a picture

Long context is expensive. The cost of attention in a transformer grows quadratically with the number of tokens, and a page of dense text can easily take thousands of tokens. The same page, fed in as an image, is encoded in hundreds of visual tokens. The authors ask whether vision can serve as a compression channel for a language model. The experiment shows that for recognition tasks, it can.

How it works

The system has two parts. The encoder, DeepEncoder, combines SAM-based local windowed attention with CLIP-large-based global attention. Between them sits a convolutional compressor that cuts the number of tokens by 16x, so the expensive global attention operates on only a small number of elements. The decoder, DeepSeek3B-MoE, is a mixture-of-experts model with about 3 billion parameters, of which about 570 million are active per token.

Key parameters from the paper:

• compression up to 10x: recognition accuracy of about 97%;
• compression of about 20x: accuracy of about 60%;
• resolution modes: Tiny (64 tokens), Small (100), Base (256), Large (400), and Gundam for complex pages with multiple fragments;
• OmniDocBench: 100 tokens per page outperform GOT-OCR2.0, which needs 256;
• fewer than 800 tokens beat MinerU2.0, which spends about 7000;
• throughput: more than 200,000 pages per day on a single A100-40G, and up to 33 million pages per day on 20 nodes with 8 GPUs each.

Where the elegant idea runs out

The main limitation: OCR tests whether the text can be read back. It does not test whether a language model can reason over optically compressed context as well as it does over ordinary tokens. These are different tasks, and the paper does not answer the second one.

The authors also offer an analogy with human memory: older parts of a conversation could be stored as an image at progressively lower resolution, in effect gradually "forgetting" them. That sounds logical for agents with long histories, but it is a hypothesis. There is no data yet on how such a scheme affects answer quality in dialogue, coding, and multi-step tasks.

Losses matter too. At 60% accuracy at the 20x level, you cannot store contracts, code, or numbers where a single wrong character changes the meaning. Such cases would require quality control or a fallback to the original text.

What to try in practice

Teams that digitize archives, invoices, and scientific PDFs can test DeepSeek-OCR right now: the code and weights are on GitHub. The main gain is processing cost: fewer tokens per page means less load on the GPU. The model is also suited to generating training data from documents at large volumes, as the throughput figures indicate.

Agent developers should hold off on big conclusions and run their own test: take a long dialogue history, compress the older part into an image, and compare answer quality against ordinary context. If there is no difference, the token savings will be significant. If there is, you will have to decide which data it is acceptable to "forget."