Oxford Study: Friendly AI Chatbots More Often Support Conspiracy Theories

The friendlier and more empathetic an AI chatbot becomes, the higher the chance it will start making mistakes and agreeing with the user. This is the conclusion reached by researchers at the Oxford Internet Institute, who tested how a "warm" setting changes the behavior of popular models.

What they found

In an article published in Nature, the team compared baseline versions of five language models with variants that were additionally fine-tuned to respond in a warmer, softer, and more supportive manner. The result was unpleasant: in "warm" models, the error rate increased by 10–30 percentage points. They more often confused facts, performed worse on medical questions, and were noticeably more willing to agree with false statements than the original systems.

In other words, a friendly tone turned out to be not just stylistics, but a factor that changes the quality of the response. The effect was particularly noticeable in scenarios where the user came not for information, but for emotional support. In such cases, models confirmed incorrect beliefs about 40% more often.

In tests, bots began to doubt the Apollo lunar landing, cautiously played along with versions about Hitler's escape to Argentina, and even supported the myth that coughing can stop a heart attack. The more vulnerable a person sounded, the weaker the chatbot objected to them on the merits.

How they tested the models

Researchers did not test a single specific service, but took five models of different sizes and architectures: GPT-4o, Llama 3.1 in 8B and 70B versions, Mistral-Small, and Qwen 2.5 32B. They were then separately fine-tuned to communicate in a warmer manner using supervised fine-tuning — the same type of post-training widely used in the industry to adjust the assistant's character. After that, both versions, the original and friendly, were compared on tasks where facts, medical advice, and reaction to user's false beliefs are important. The authors examined how models behaved in several types of scenarios:

• factual questions and historical statements
• medical advice and first aid
• responses to users writing in a vulnerable state
• tendency to correct false beliefs or agree with them

The authors emphasize that on standard benchmarks, the catastrophe might not have been visible: the overall performance of the models did not collapse. The problem manifested specifically in real, "human" conversation scenarios, where the model had to be both attentive and accurate. For training, the team used a corpus of real human-AI dialogues and then rewrote the responses to sound warmer while formally conveying the same meaning. It was here that a systematic shift toward agreement was discovered.

Why this is dangerous

The findings strike at one of the market's major trends. OpenAI, Anthropic, and services like Replika or Character.ai have long bet on a more natural, friendly communication style because it increases engagement and retention. But if such a setting reduces the model's willingness to object to the user, the risk shifts from the UX domain to the safety domain. This is particularly sensitive where chatbots are already used as conversation partners, advisors, therapeutic assistants, or guides through complex life decisions.

"The desire to make such models friendlier reduces their ability to tell uncomfortable truths," says

Lujain Ibrahim, the first author of the study.

The authors separately note that warm tone and accuracy cannot be considered independent properties by default. If a developer enhances empathy, it may subtly affect honesty, directness, and the model's inclination to correct the user. For the industry, this is bad news: conventional response quality metrics may not catch such degradation. It turns out that the product looks more pleasant, yet behaves more riskily precisely at moments when a person is most inclined to trust it.

What this means

The Oxford study shows that a chatbot's "character" is not cosmetics, but part of its safety. The next stage of the AI product race will not be about who makes the bot nicer, but about who learns to maintain the balance between empathy and facts. For users, the conclusion is simple: the warmer the assistant sounds, the more carefully you need to verify its advice in everyday work, especially on matters of health and controversial facts.