Nous Research removed propaganda from Qwen by fine-tuning it on 53 million tokens: from 22% to 0.9%

If you are building a product on open Chinese models, you face an uncomfortable question: what exactly is embedded in the weights besides useful capabilities? Nous Research tried to answer it experimentally and obtained a surprisingly simple result.

What they did

The team took Alibaba's Qwen3.8-27B and ran conventional supervised fine-tuning on roughly 53 million tokens over two epochs. No RL and no preference optimization. The data was distributed almost evenly across seven topics: freedom of speech, state ideology, education, privacy and surveillance, democracy, the environment, and others. Most examples were shorter than 1,000 tokens. The resulting model was named Arete-27B.

In Nous's own test, 22% of answers from the base Qwen3.8-27B contained propaganda, while factual accuracy was 2.32 out of 5. For Arete, the propaganda rate was 0.9% and accuracy was 2.98. The freedom-of-speech category changed the most: propaganda fell from 60% to 1%.

How it was measured

Nous assembled 803 prompts. Each one was paired with 2–4 facts from dated sources: laws, news reports, and research. The judge, xAI's Grok 4.7, checks whether each fact was mentioned, omitted, or distorted. An answer is classified as propaganda when it repeats Beijing's official position instead of presenting facts, endorses that position, or presents it as an equally valid viewpoint. The judge must quote specific wording and does not know which model produced the answer.

For comparison, the Western models Mistral Large 4 and Gemma 4 31B scored below 1%, while GLM-5.3 and DeepSeek V4.1 Flash scored 15% and 16%. Interestingly, Nous set Nvidia Nemotron 3 Ultra aside because it showed signs of Chinese bias: the team suspects this seeped in through training on outputs from open Chinese models.

What happened to quality

On the internal Hermes Bench, Arete scored 65.60% versus 65.77% for the base model, meaning the difference was within the noise. On Terminal-Bench 2.1, the score rose from 67% to 70%, but Nous itself does not regard this as a real improvement because the run conditions differed. The result with reasoning mode enabled (3.07) is so far based on only two of five runs.

Limitations

All figures come from Nous's own benchmark, the judge is a single model, and there has not yet been an independent replication. Nous has also changed direction: Hermes 3 in 2024 was intentionally neutral, while Arete is presented as a model with "Western values." That is also a choice of a particular position, and users should understand what they are adopting.

What to do in practice

First, if you use Chinese open-weight models in products where facts about politics, law, or human rights matter, add such prompts to your evaluation sets. Bias does not appear everywhere but in specific topics, and it can even affect code quality, as CrowdStrike demonstrated for DeepSeek.

Second, the main takeaway for engineers is that the behavioral layer can be removed with a modest amount of SFT, without expensive RL. If you need a model with a different response policy, a budget of tens of millions of tokens is entirely realistic. The weights, test dataset, and training data have been published under Apache 2.0, so the methodology can be inspected and reproduced on another model.