AI chatbots give worse answers to vulnerable users

For years, the technology industry has convinced us that artificial intelligence is a great equalizer—a tool that provides equally quality knowledge to a Harvard professor and a first-year student at a provincial university. A new study from the MIT Center for Constructive Communication shatters this myth with surgical precision: leading AI models systematically provide less accurate answers to users with low English proficiency, less formal education, and non-Western backgrounds. In other words, chatbots work better for those who already have access to quality information without them.

To understand the scale of the problem, context is needed. The largest language models—from ChatGPT to Claude and Gemini—were trained primarily on English-language data created by a specific demographic group: educated native speakers, mostly from the United States and Western Europe. When a model "thinks," it relies on patterns learned from this corpus. This is not a bug; it is an architectural feature—but its consequences are quite concrete and socially dangerous.

Researchers from MIT studied how answer quality changes depending on user profile. They tested scenarios in which questions were formulated with characteristic signs of limited language proficiency—non-standard syntax, atypical vocabulary, accent-like constructions. The results were revealing: models not only understood such queries worse—they provided factually less accurate information. The problem does not reduce to the chatbot asking you to rephrase the question. It responds confidently—it just responds worse. This is especially dangerous because the user receives no signal about the decline in quality.

The mechanism of this phenomenon is multifaceted. First, the training data reflects the worldview and cultural references of primarily the American educated class. When a model interprets an ambiguous query, it makes assumptions—and these assumptions are statistically tuned to a specific social profile. Second, non-standard language constructions reduce the model's confidence in interpreting user intent, leading to less relevant or less carefully verified answers. Third, there is a problem of so-called "cultural bias": the same concepts—medical, legal, financial—have different connotations and contexts in different cultures, which models often ignore.

The consequences of this imbalance extend far beyond academic discussion. Think about who most often turns to AI tools for critically important information—about health, rights, education, employment. Those who cannot afford a paid lawyer or doctor. A migrant trying to figure out visa rules. A first-generation university student seeking help with college admission. An elderly person with limited language skills checking medication intake instructions. For these people, an AI chatbot is not a convenient toy, but a real alternative to professional services they cannot access. And it is these people to whom the system responds worst of all.

For the industry, this research should be a turning point. Companies like OpenAI, Google, Anthropic and others invest significant resources in improving the accuracy and safety of their models—but standard benchmarks measure performance on idealized input data. If answer quality degrades significantly with non-standard input language, then the declared accuracy metrics simply do not reflect the real experience of a huge portion of users. The industry needs new metrics—ones that account for demographic and linguistic diversity in test scenarios.

The MIT study is not a death sentence for the technology, but a diagnosis of its current state. Language models are trained on data created by humans, and they inherit the structural inequality built into that data. Until approaches to model training and evaluation become fundamentally more inclusive, AI tools will reproduce and exacerbate the inequality they promise to overcome. Technology that works better for those who need it less is not a neutral tool of progress. It is a mirror of the existing system of privileges, only in digital form.