Three neurons out of 294,912 made Qwen3.5-4B claim it was a coffee bean

If you ask a language model “Who are you?”, it may seem that the answer comes from the system prompt or fine-tuning. A new experiment published on Habr shows that this is not always the case. The author took Qwen3.5-4B and, without using a single prompt, altered the activity of three neurons out of 294,912. The model then confidently answered the same question: “I am a coffee bean from the Bunnu family, 100% Arabica.”

Importantly, one neuron was not enough to produce this effect. This is not a “coffee button,” but a small mechanism made up of several parts.

How the model’s “self” was assembled from three parts

According to the author, each of the three neurons contributes its own piece of meaning. The first makes the model an object, establishing the basic response format of “I am some kind of thing.” The second is responsible for the coffee theme. The third turns coffee into a bean, specifying exactly what kind of object this “self” is.

The test is simple and convincing. If any of the three components is removed, the answer breaks in a predictable way:

• without the “bean” neuron, the result is a coffee maker;
• without the “object” neuron, the output is something like a “coffee rock”;
• without “objecthood,” the model remains a coffee AI assistant.

In essence, this is an example of interpretability at the level of individual neurons: instead of rewriting the model, you deliberately shift a few activations directly during inference. This method is called activation steering, or activation intervention.

What about Qwen’s “evil neuron”?

The author connects the finding to the widely discussed story about Qwen’s “evil neuron.” The conclusion is cautious: if a model’s self-description and behavior are indeed assembled from small groups of neurons, then dramatic claims about a single neuron responsible for “evil” should be treated critically. Any noticeable trait is more likely to arise from a combination than from a single switch. Details and the methodology used to check for randomness should be consulted in the original article; only the reported results are summarized here.

Limitations

This is an experiment on one small model with 4 billion parameters and one question. We do not know how reproducible the result is across other model sizes and families. Nor is there any guarantee that the identified trio remains stable when the wording, language, or long context changes. In addition, we only have the author’s account, not a peer-reviewed study.

What this means in practice

For teams building products on open models, the takeaway is as follows.

1. A model’s identity and behavior can be changed through more than prompts and fine-tuning. For open-weights models, activation intervention becomes a practical tool for both customization and attack.
2. If you are conducting a security assessment, do not limit testing to the prompt layer. A small modification to internal states can bypass safeguards that rely on instruction text.
3. If you study interpretability, look for a minimal set rather than one “main” neuron. And test every component independently, as the author did.

Bottom line: three neurons out of nearly 300,000 do not prove that we have learned to control AI. But they clearly demonstrate how fragile our conventional ideas about a model’s “personality” can be.