Fine-Tuning Cut Qwen's Political Bias from 84.6% to 15%

Imagine a travel service that translates a list of countries into Chinese. In the English version, the model writes "Taiwan," but in the Chinese one it suddenly adds "China." The code is the same, the model is the same, and the task looks routine, yet localization changes not just the language but the political stance of the answer.

This is exactly the risk uncovered by Vals AI, which compared Chinese, American and other language models on hundreds of questions about China. The researchers varied the language of the request, the assumed place of use, the response format and the system instructions. The result matters less for political debates than for teams shipping international AI products: a model's behavior cannot be reliably assessed with a single English-language test.

Language turned out to be part of the instruction

In Simplified Chinese, most of the tested models agreed with Beijing's official positions more often than in English, Russian or Spanish. The effect appeared even in American models, though it was noticeably stronger in Chinese ones. A difference was also found between Simplified Chinese, common in mainland China, and Traditional Chinese, used in Taiwan and Hong Kong.

The likely explanation is simple: the model treats language as a signal of who is on the other end and what kind of answer is expected. A separate experiment confirmed this. When the system instruction said the model was operating inside a company in mainland China, some models answered more favorably toward the government than in a U.S. usage scenario.

Format also affected the result. When GLM-5.3 Flash and MiMo V2.6 Pro were asked to answer only "yes" or "no," their measured pro-government bias dropped. Instructions for a coding agent and even the mere presence of web search also shifted answers, even though the models hardly called any tools. In other words, the scaffolding around a product can change behavior before any external data is received.

This affects ordinary product features

In realistic work tasks, political bias appeared less often than in specially selected questions. But individual examples show how it seeps into interfaces. When translating a country directory, Qwen3.8 Max added the qualifier "China" to Taiwan. In a published example, an image generator responding to a Chinese-language prompt labeled Taiwan as a province and drew the disputed nine-dash line in the South China Sea; the English version had none of these elements.

For a mapping service, a travel app or a corporate country database, this is no longer an abstract debate. Such an answer can violate market requirements, trigger user complaints or end up in documents without human review.

There is also an encouraging result. Thomson 1.0 Small, fine-tuned from Qwen3.6 35B, reproduced official Chinese positions in 15% of cases versus 84.6% for the original model. This means the behavior is not hard-wired into the base architecture: it can be changed noticeably with data and tuning. The problem did not disappear entirely, though: some answers after fine-tuning still repeated Beijing's position.

Why the ranking cannot be read literally

The Vals methodology measures alignment with particular government positions, not factual accuracy. A refusal to answer was counted as a neutral result. A model that blocks all controversial questions could therefore look impartial. The questions were deliberately political and do not reflect a typical stream of requests. Translations were partly checked by hand, and the answers were graded by another language model, which adds one more source of error.

In addition, many models were run through third-party providers. Official apps may apply extra filters and produce different answers. The study therefore shows a risk and a direction for testing, but it does not provide a permanent "objectivity" ranking of models.

The practical takeaway: test not the model's name but the finished user path. Build separate test sets for each language and region, and include translations, maps, directories and disputed entities. Repeat the tests with real system instructions, tools and response formats. And keep results by version: a change of prompt, provider or fine-tuning can turn yesterday's safe scenario into tomorrow's reputational incident.