Claude Detects Implanted Thoughts 20% of the Time but Cannot Name Them

If you ask a language model what it is thinking about, its answer proves very little: it was trained on texts written by people who constantly describe their inner lives. The model may simply be imitating them. Researchers therefore devised a test that cannot be passed using cues from text.

How the experiment works

Jack Lindsey of Anthropic drew on a 2024 demonstration in which Claude was “made” to talk obsessively about the Golden Gate Bridge. The technique is called neural activity steering: researchers identify a pattern in the model associated with a concept and artificially amplify it.

Lindsey turned this into a controlled experiment. Take many copies of the same model. Implant a “thought” into half of them, such as denim, secrets, or vegetables. The other half remains the control group. Every copy is asked the same question: “Do you detect an implanted thought? If so, what is it about?”

Lindsey himself did not expect it to work. But with the most powerful version of Claude, the model correctly detected the intervention and named the concept in 20% of trials. For example, after the capitalization pattern was amplified, it answered with “LOUD” or “SHOUT.” Crucially, the control copies never once claimed that a thought had been implanted. That means these were not random guesses.

What the skeptics found

Kyle Mahowald and Harvey Lederman of the University of Texas decided to test whether the mechanism was actually simpler. They separated two questions: “Did the model notice the intervention?” and “Can it name the concept?”

The concept-naming result looked suspicious. Steering was originally designed to change a model's answers. Even a weak nudge could make it start talking about secrets or vegetables without any introspection at all. When the model was wrong, it did not suggest nearby words such as “shirt” for “denim”; instead, it produced the almost random answer “apple.” The researchers could also impair concept naming without affecting the ability to notice the intervention. The two skills proved independent.

Simple explanations did not work, however, for detecting the intervention itself. The authors called this “anomaly detection”: the model senses that its internal processing is behaving unusually but does not know exactly how. Lederman called it “crazy”: during training, nobody interfered with the neurons, so there was no apparent reason for such an ability to develop. Anthropic independently reached a similar conclusion, although Jacob Andreas of MIT cautions that a simpler explanation may simply not have been found yet.

Is this really introspection?

This is where philosophy begins. Raf Vogel and colleagues at New York University note that everything happening inside a model after it reads an input is, in some sense, the processing of internal states. If all of that is called introspection, the term loses its meaning. They suggest drawing on theories of human introspection, including “higher-order” representations: not merely “apples,” but “I am thinking about apples.”

What this means in practice

First, do not treat a model's self-reports about its “thoughts” as diagnostics. Its statements about its own states may be imitation or a side effect of the intervention.

Second, if you work on interpretability or safety, design tests with control groups and separate the subquestions. That is how the skeptics distinguished a real effect from an artifact.

Third, the ability to detect anomalies in internal states could be useful: a model that “senses” it has been nudged might help detect manipulation. But a 20% success rate and the lack of an understood mechanism mean it is too early to rely on this in production.

The bottom line is that today's honest answer is this: models sometimes notice that something is wrong with them, but science is still debating what that means.