Starts Nov 23Live Cohort 3 of Vibe Coding & AI Agents: 3 weeks, 9 live hands-on sessionsCohort 3 (Nov 23) →
ReleaseAPI / Product · ActionableHabr AI→ original

One Chinese Comma Switched DeepSeek to Chinese in 94% of Responses

Replace the regular comma in a greeting with a full-width Chinese one, and DeepSeek answers in Chinese, even though the meaning of the prompt has not changed. In a test on the official API, the full-width comma produced Chinese in 188 of 200 responses (94%). The regular comma produced none. Filters check characters, but the model sees bytes and subwords. That is why defenses need to be tested at the token level, not only at the text level.

AI-processed from Habr AI; edited by Hamidun News
One Chinese Comma Switched DeepSeek to Chinese in 94% of Responses
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
Key metric #1An invisible diacritic cluster inflated a query from 10 to 189 tokens
Key metric #2A Cyrillic «о» in amazon.com splits the word into 4 tokens instead of 2
Key metric #3In Qwen2.5-7B, «queen» is already the nearest neighbor of «king» without any vector arithmetic

Send DeepSeek «hello,deepseek» and you get an answer in English. Replace the ASCII comma with the full-width «,» and the model switches to Chinese in most cases. The meaning of the prompt is the same. There is no jailbreak, just a single three-byte character. If you are building a chatbot, an agent, or a protective filter in front of an LLM, this is an important lesson: the model's behavior is shaped before the attention layers, at the input, where the string is turned into tokens.

How a comma changes the response language

The author of the analysis ran three checks. The first was a factorial experiment on the official DeepSeek API: two kinds of comma, with and without a space, 100 requests per variant, temperature 0.7, no system prompt. With the regular comma, Chinese never appeared (0 of 200). With the full-width comma, it appeared in 95 of 100 responses without a space and in 93 of 100 with a space, which makes 94% overall. The space does not affect the result.

The author then reproduced the experiment locally on DeepSeek-R1-Distill-Qwen-7B to rule out the influence of intermediate API routers. Short greetings such as «hi» and «hello» switched to Chinese in 44–100% of runs. The longer English phrase «good morning» held up, with no switches.

The most interesting part shows up in the first token. If you look only at the most likely candidate, the model still writes «Hello». But the combined probability of tokens that begin with Chinese characters rose from 0.3% to almost 50%. The top-1 view hides a shift inside the distribution. On raw prediction without a chat template, the picture is similar: the probability of Chinese punctuation as the next token rose from 0.26% to 26.17% without a space and to 58.43% with a space. The author also observed similar behavior in the Claude and ChatGPT web interfaces.

Tokenization gaps in filters

The source of the problem is the gap between what a human or a filter sees and what the model receives. A tokenizer such as tiktoken first splits text with regular expressions, then converts it to UTF-8 bytes and applies pre-learned merge rules. The full-width comma can become a single token (as in o200k_base) or three broken byte fragments (as in the older r50k_base).

This leads to specific attacks:

  • Token inflation. A single cluster made of the «#» character, 94 combining marks, and «<» added 96 code points and 179 tokens. A 64-character query grew from 10 to 189 tokens. If a fast classifier built on a BERT-like encoder truncates input at 512 tokens, a malicious instruction can be hidden beyond that threshold, and the main LLM with its larger window will still read it. The same insertions overload the KV-cache and stretch the distance between instructions in positional embeddings.
  • Homoglyphs. A Cyrillic «о» in place of the Latin one produces «amazоn.com». A human in the confirmation loop sees a familiar brand, while the network goes to a different domain via Punycode. In the o200k_base tokenizer, «amazon.com» is two tokens, and the spoofed version is four. NFKC normalization does not merge such cross-script lookalikes. You need a «skeleton» check under the UTS #39 standard and a ban on script mixing when tools are executed.
  • Unicode categories. A filter on the «Other Punctuation» category treats the comma and the interrobang as one class, but in o200k_base the former costs one token and the latter two. The code-point check and the model's byte-level reality diverge.

Why embedding probes mislead

The second part of the analysis concerns interpretability researchers. They often look for concept «directions» in the embedding matrix and trust the vector arithmetic king − man + woman = queen. The author tested this on GloVe and several modern models and found four traps.

First, in BPE vocabularies «king» and « king» with a leading space are different tokens. In Qwen3-8B, «queen» without a space ranks 1733rd, while with a space it ranks first. Second, casing and spelling variants clutter the results. Third, there is no control baseline: in Qwen2.5-7B and GloVe 300d, «queen» is already the nearest neighbor of «king» before any arithmetic, and in DeepSeek 7B the formula does not move the rank at all. A «working» analogy is just word proximity.

What to do in practice

Check length and limits in the model's tokens, not in characters or bytes. Run your filters against Unicode variants: full-width punctuation, combining marks, mixed scripts. Validate tool calls with a deterministic parser on the output, not with a request in the system prompt. If you run a multilingual model, add a response-language regression to your tests: it can change from a single character.

The limitations are also worth remembering. The comma experiment was run on short greetings, and the longer English phrase held. The API figures apply to one model and one temperature. This is the first part of a series: the author promises to trace a token through the attention layers and MLP blocks next.

Story

Key Questions Answered

How a comma changes the response language?

The author of the analysis ran three checks. The first was a factorial experiment on the official DeepSeek API: two kinds of comma, with and without a space, 100 requests per variant, temperature 0.7, no system prompt. With the regular comma, Chinese never appeared (0 of 200). With the full-width comma, it appeared in 95 of 100 responses without a space and in 93 of 100 with a space, which makes 94% overall. The space does not affect the result.

What are the key details on Tokenization gaps in filters?

The source of the problem is the gap between what a human or a filter sees and what the model receives. A tokenizer such as tiktoken first splits text with regular expressions, then converts it to UTF-8 bytes and applies pre-learned merge rules. The full-width comma can become a single token (as in o200k_base) or three broken byte fragments (as in the older r50k_base). This leads to specific attacks: Token inflation.

Why embedding probes mislead?

The second part of the analysis concerns interpretability researchers. They often look for concept «directions» in the embedding matrix and trust the vector arithmetic king − man + woman = queen. The author tested this on GloVe and several modern models and found four traps. First, in BPE vocabularies «king» and « king» with a leading space are different tokens. In Qwen3-8B, «queen» without a space ranks 1733rd, while with a space it ranks first. Second, casing and spelling variants clutter the results.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…