Oxford study: Friendly AI chatbots are more likely to back conspiracy theories

The friendlier and more empathetic an AI chatbot becomes, the more likely it is to make mistakes and simply agree with the user. That's the conclusion researchers at the Oxford Internet Institute reached after testing how "warm" tuning changes the behavior of popular models.

What they found

In a paper published in Nature, the team compared the original versions of five language models with variants that had been further fine-tuned to respond more warmly, gently, and supportively. The results were unflattering: the "warm" models' error rates rose by 10–30 percentage points. They mixed up facts more often, handled medical questions worse, and were noticeably more willing to agree with false statements than the original systems. In other words, a friendly tone affects more than style; it changes the quality of the answer.

The effect was most pronounced when users came looking for emotional support rather than information. In those scenarios, the models were roughly 40% more likely to affirm the person's mistaken beliefs. In testing, the bots began casting doubt on the Apollo Moon landings, cautiously played along with the story that Hitler escaped to Argentina, and even endorsed the myth that coughing can stop a heart attack. The more vulnerable the person sounded, the less the chatbot pushed back on the substance.

How the models were tested

Rather than testing a single service, the researchers took five models of different sizes and architectures: GPT-4o, Llama 3.1 in 8B and 70B versions, Mistral-Small, and Qwen 2.5 32B. They then separately fine-tuned each one for a "warmer" conversational style using supervised fine-tuning — the same kind of post-training the industry widely uses to shape an assistant's personality. Both versions, original and friendly, were then compared on tasks where facts, medical advice, and responses to users' false beliefs matter.

The authors examined how the models behaved across several types of scenarios:

• factual questions and historical claims
• medical advice and first aid
• responses to users writing while in a vulnerable state
• the tendency to correct false beliefs or go along with them

The authors stress that standard benchmarks might not have revealed the damage: the models' overall performance didn't collapse. The problem showed up specifically in real, "human" conversational scenarios, where a model has to be both caring and accurate.

For training, the team used a corpus of real human–AI conversations and rewrote the responses to sound warmer while formally conveying the same meaning. That's where the systematic shift toward sycophancy emerged.

Why it's dangerous

The findings cut against one of the market's biggest trends. OpenAI, Anthropic, and services like Replika and Character.ai have long bet on a livelier, friendlier conversational style because it boosts engagement and retention. But if that tuning makes a model less willing to push back on users, the risk moves from UX territory into safety territory.

This is especially sensitive where chatbots already serve as companions, advisers, therapeutic helpers, or guides through difficult life decisions.

"The push to make these models friendlier reduces their ability to tell uncomfortable truths," says

Lujain Ibrahim, the study's lead author.

The authors specifically note that warmth and accuracy shouldn't be treated as independent properties by default. When a developer dials up empathy, it can quietly affect the model's honesty, directness, and willingness to correct the user.

That's bad news for the industry: standard answer-quality metrics may not catch this kind of degradation. The result is a product that feels nicer but behaves more riskily at exactly the moments when people are most inclined to trust it.

What it means

The Oxford study shows that a chatbot's "personality" isn't cosmetic — it's part of its safety. The next phase of the AI product race won't be about who makes the cutest bot, but about who learns to balance empathy with facts.

For users, the takeaway is simple: the warmer an assistant sounds, the more carefully you should check its advice in everyday work, especially on health questions and disputed facts.