Local LLMs Translate Text at Up to 115 Tokens per Second

Local translation no longer looks like an experiment for people with a server rack at home. On an ordinary workstation with a 16 GB RTX 5060 Ti, you can get either fast web page translation or slower but noticeably more careful book translation. However, the author of the test did not find a single model for both tasks.

The test covered texts in English, Spanish, Chinese and Hebrew: news, technical documentation, social media discussions, contemporary prose and ancient treatises. All materials were translated into Russian. Quality was scored by Claude and OpenAI judge models, and the reference versions were prepared with Claude. So the numbers are useful primarily for comparing participants within this test, not as an absolute measure of literary skill.

Fast translation fits in 16 GB

The most practical result came from Gemma 4 26B-A4B in the QAT Q4_0 version. It fits entirely in the RTX 5060 Ti's video memory and delivers about 115 tokens per second. On modern texts and web pages, its scores fell roughly in the 80–88 out of 100 range.

In practice, this means a documentation page or a long post can be translated almost as you read. The model is not the best in every category, but the latency is not irritating, and running it does not require offloading some layers to system memory. For a browser translator, that matters more than a few extra quality points.

With ancient texts, the picture changes. On classical Hebrew the model scored 49 points, and on classical Chinese 54. In other words, the fast profile is fine for everyday language, but it is risky to use where meaning depends on the era, set expressions and cultural context.

Quality requires RAM

Qwen3.8 Flash-Next was the leader, or among the leaders, in almost all main categories: 87.6 points in English, 92 in Spanish, 88 in Chinese and 90 in Hebrew. The IQ4_XS quantization ran at about 15 tokens per second, and the more compact IQ3_XXS at about 21 tokens per second.

The price of this result is 80–95 GB of system memory in addition to the GPU. The model has a sparse architecture: only a portion of the parameters is active for each token, so it keeps running at acceptable speed even with a significant share of the weights placed in RAM. But for a computer with a typical 32 GB, this option is already impractical.

For literary prose, the dense Gemma 4 31B performed best. It scored 85 points on English and Spanish literature and wrote especially well in Russian. In addition, using it as a separate editor raised the scores of weak drafts by 5–14 points. The downside is a speed of about 6–7 tokens per second and a noticeably longer run on complex texts.

How to choose a model for your product

For built-in website translation, it makes sense to start with Gemma 4 26B-A4B: it is fast, fits on a single graphics card and is good enough for news, documentation and discussions. If translation runs in batches overnight and the machine has 96 GB of RAM, Qwen3.8 Flash-Next delivers more consistent quality across languages.

Books are best processed in two passes: first get a draft from a fast or strong general-purpose model, then hand it to Gemma 4 31B for literary editing. Still, even this setup does not remove the need for a human for poetry, historical texts, law and medicine.

The main limitation of the benchmark itself is automatic evaluation by related cloud models. They may prefer a familiar style and do not always notice a factual error successfully masked by good Russian. Before deployment, you should assemble your own set of real product documents and give at least some of the translations to native speakers. But the basic engineering conclusion is already clear: local translation on a consumer graphics card has become a working tool, provided you decide in advance what matters more — latency, quality or independence from the cloud.