LLMs Quietly Rewrite African American English as Standard English, New Study Finds

A new study on arXiv has uncovered a large-scale problem: six modern LLMs (ranging from 14B to 70B parameters) systematically rewrite African American English (AAE) as Standard American English (SAE), even when the context requires keeping the dialect.

What bias did the researchers find?

African American English is not an error but a full-fledged dialect, spoken by more than 30 million people in the US. Yet modern LLMs treat it as "incorrect" text that needs to be fixed into the standard variety.

The authors tested six instruction-tuned LLMs. In every case, the models preferred the SAE continuation, even when the context clearly called for AAE. In effect, LLMs rewrite users' speech, replacing dialect with standard English automatically and without notice.

• 6 instruction-tuned models (14B–70B parameters)
• 30+ million AAE speakers
• Negative concord ("ain't nobody") is a universal bias trigger across all models
• Syntactic constructions trigger rewriting consistently

How the bias was detected and measured

The authors developed a systematic audit method: the conditional Dialect Group Invariance (cDGI) metric, which separates genuine model bias from external artifacts. A feature-level analysis identified which specific AAE markers trigger the bias most.

For validation, the researchers built REAL-AAE, a dataset of 17,479 AAE/SAE/AAE_back example triplets from natural tweets. That is 2–6 times larger than all previous AAE resources. The dataset was validated automatically (BERTScore F1 = 0.95) and manually by three AAE speakers (83% semantic agreement). This two-step validation ensures the dataset reflects the real subtleties of the dialect.

Activation steering: a fix without retraining

The researchers proposed activation steering, the first application of this method to dialect bias. It is a training-free method that works only at test time, with no changes to the model's weights.

Here is how it works: causal tracing extracts dialect directions from the model's activations, and those directions are then injected into the layers responsible for generation. The result: the model preserves AAE in contexts where it is appropriate.

The results are strong: activation steering reduces dialect bias 5–20 times better than prompting approaches, while keeping fluency in standard English when that is actually required. This shows that a fix is possible without fully retraining the model.

Why this matters

Dialect bias is a form of hidden cultural assimilation. LLMs built into education systems, legal platforms, HR and voice assistants can silently correct users' speech. A user writes in their own dialect, and the system returns it in the standard one, erasing linguistic identity. The study shows that this can be detected and corrected with a method that does not require retraining.

Frequently asked questions

Why do LLMs rewrite AAE?

Because they are trained mostly on text in Standard English. The models interpret AAE as a grammatical error to be fixed in line with the dominant pattern in the training data.

Is this a problem only for English?

This study focuses on AAE/SAE, but the bias is typical of any dialect–standard language pair. A similar bias likely exists for other languages in which dialects are a minority in the training data.

When will commercial models get activation steering?

For now, it is a researchers' method. Integration into LLMs depends on providers. But the study shows that a fix is technically possible without deep retraining.