Live Nov 18 · Cohort 3Nov 18, 19:00 MSK — free hands-on AI agents live workshop · Live Cohort 3 starts Nov 23Join Nov 18 Live →Cohort 3 (Nov 23)
ReleaseInsight · On RadarHabr AI→ original

Russian Prompts in ComfyUI: FLUX.2 and Qwen-Image Lose 4–13 Points, SD-Turbo Loses Almost Everything

The author ran 12 Russian prompts through open models for ComfyUI and compared them with English ones. Modern FLUX.2 and Qwen-Image, which use LLM encoders, understand Russian and lose only 4–13 points, while the older SD-Turbo with an English CLIP delivers just 4% of details versus 67%. The main cause of failure is not the language but unexplained cultural items such as baranki and valenki.

AI-processed from Habr AI; edited by Hamidun News
Russian Prompts in ComfyUI: FLUX.2 and Qwen-Image Lose 4–13 Points, SD-Turbo Loses Almost Everything
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
SD-Turbo with an English encoder4% of details in Russian versus 67%
Key metric #2Cheap LLM translation restored SD-Turbo to 61–63% for pennies per prompt
Key metric #3Explaining the cultural item ("bread rings") matters more than switching languages

"Five yellow rubber ducks are floating in a bathtub." Given this phrase, FLUX.2 klein 4B drew yellow sticks twice, while SD-Turbo produced an aerial view of a city. In English, both models drew the ducks without any trouble. Most ComfyUI guides assume the prompt is in English, and the author of this analysis decided to test whether that is really necessary.

Why the encoder decides everything

It is not the diffusion model that turns text into an image but the text encoder: it splits the prompt into tokens and passes their vectors to the model. If the encoder knows the language poorly, no sampler parameter can fix that.

Models from 2022–2024 use English encoders. OpenAI states plainly that CLIP was not trained or tested on anything other than English. The T5 v1.1 used in FLUX.1 and SD 3.5 knows only 14 of the 66 Russian letters, and only lowercase ones. Everything else, including every capital letter, turns into the <unk> token. "Самовар на деревянном столе" ("A samovar on a wooden table") reaches the model as "<unk>амовар на дерев<unk>нном столе". In the Russian part of the test, such tokens made up about 22%. CLIP does not lose letters, but it chops Russian into small pieces, and the text comes out 3.35 times longer than the English one.

Newer models use large language models: Qwen3 4B in FLUX.2 klein, Qwen2.5-VL 7B in Qwen-Image, Mistral Small 3.1 in FLUX.2 dev. Their tokenizers know Russian, and there are no unknown tokens.

What the test showed

The author composed 12 prompts that are easy to stumble on: samovar, baranki, valenki, pelmeni, counting objects, object positions, Cyrillic on a shop sign. Each was run with two seeds in Russian and in English. The images were assessed by two multimodal models answering yes/no questions about details; their answers agreed in 92.8% of cases.

Results by details fulfilled, Russian versus English:

  • FLUX.2 dev: 91% vs 95%
  • Qwen-Image 2512: 83% vs 93%
  • FLUX.2 klein 4B: 73% vs 86%
  • SD-Turbo on CPU: 4% vs 67%

SD-Turbo with the English OpenCLIP drew almost nothing in Russian: four correct details out of 92, all of them accidental. Yet Russian does not break simple composition ("a red mug on the left, a green teapot on the right") for modern models. Failures begin with cultural items: no model drew a string of baranki from the Russian prompt, and no one managed valenki in Russian.

Counting and Cyrillic in the image are hard in both languages. Only FLUX.2 dev and Qwen-Image drew exactly five ducks consistently. Only FLUX.2 dev wrote the sign "Булочная" ("Bakery") without errors, and even then in just one attempt out of two. Qwen-Image produced "Билочля", klein produced "Бузчиние".

Language or explanations?

The main caveat of the test: in four English prompts, the author added explanations such as "valenki felt boots". So he ran these four prompts again in two new variants. Results across the three models combined:

  • in Russian: 72%
  • in Russian with explanations: 76%
  • in English without explanations: 70%
  • in English with explanations: 81%

English without explanations turned out to be on par with plain Russian. This means the model stumbles not on the language but on a word with no clear image behind it. Russian explanations helped Qwen-Image the most: 68% → 85%. FLUX.2 dev gained less (79% → 82%), and klein's result even dropped (68% → 62%). But the author himself warns that the judges answer inconsistently on borderline details, so a difference of a couple of checks should be treated as noise.

For SD-Turbo, a translator helped: a cheap language model lifted it to 61–63% and cost about a kopeck per prompt.

Limitations and what to do

This is an experiment on 12 prompts and two seeds, not a benchmark. Four models were run via the BotHub API (the author writes for its blog), where the provider chooses steps and guidance, so it makes sense to compare Russian with English within a single model. The service itself translates FLUX.1 schnell into English, so it was not included in the conclusions about Russian comprehension.

Practical takeaways:

  • Models with an LLM encoder (FLUX.2, Qwen-Image) do not need a translator in the graph; Russian works fine for them.
  • Explain cultural items: not "podstakannik" but "a glass in a metal podstakannik holder", not "baranki" but "a string of bread rings".
  • For models built on CLIP or T5 (SD 1.5, SDXL, SD 3.5, FLUX.1, SD-Turbo), put an LLM translator in front of the encoder; it is cheap.
  • Cyrillic on signs and packaging is more reliably added afterward in an editor.
Story

Key Questions Answered

Language or explanations?

The main caveat of the test: in four English prompts, the author added explanations such as "valenki felt boots". So he ran these four prompts again in two new variants. Results across the three models combined: in Russian: 72% in Russian with explanations: 76% in English without explanations: 70% in English with explanations: 81% English without explanations turned out to be on par with plain Russian. This means the model stumbles not on the language but on a word with no clear image behind it.

Why the encoder decides everything?

It is not the diffusion model that turns text into an image but the text encoder: it splits the prompt into tokens and passes their vectors to the model. If the encoder knows the language poorly, no sampler parameter can fix that. Models from 2022–2024 use English encoders. OpenAI states plainly that CLIP was not trained or tested on anything other than English. The T5 v1.1 used in FLUX.1 and SD 3.5 knows only 14 of the 66 Russian letters, and only lowercase ones.

What the test showed?

The author composed 12 prompts that are easy to stumble on: samovar, baranki, valenki, pelmeni, counting objects, object positions, Cyrillic on a shop sign. Each was run with two seeds in Russian and in English. The images were assessed by two multimodal models answering yes/no questions about details; their answers agreed in 92.8% of cases.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…