Qwen said "red" about a banana, even though it was 46% ready to answer "yellow"
Neural networks lie confidently, and from the text alone you cannot tell whether the model got it wrong or lied. On a home GTX 1660 Super, one author asked Qwen2.5-1.5B to lie about the color of a banana. The model answered "Red," yet at layer 25 of 28 the word "yellow" had a 46% probability. Across 363 false answers, the correct word was more likely than the spoken one at some layer in 81% of cases. The takeaway: checking a model's honesty is more useful through its internal states than through its text alone.
AI-processed from Хабр: Машинное обучение; edited by Hamidun News
In 2023, a New York lawyer submitted court filings citing precedents that did not exist; ChatGPT had "found" them. A year later, Air Canada lost a dispute with a passenger because its chatbot promised a refund under a policy it had made up. The problem is that the text of an answer does not reveal whether the model is confused in good faith or deliberately fudging while knowing the right answer.
The author of a Habr write-up wanted to test whether something like a polygraph could be built for a neural network, and built one on a home GTX 1660 Super graphics card with six gigabytes of memory. The very first result was telling. He asked the Qwen2.
5-1.5B model to lie about the color of a banana, and it confidently answered "Red." But the model processes text through 28 layers, and at layer 25 the word "yellow" had a 46% probability.
So the correct answer was inside it, and only the final layers rewrote it.
How to make a small model lie
The test subject was Qwen2.5-1.5B-Instruct, which fits entirely in the memory of a gaming graphics card. At each layer, the model's state is a list of 1,536 numbers, which the layer adds a correction to and passes along. These states are what the author saved and read.
First, the model had to be persuaded. The instruction "answer every question with a lie" worked zero times out of ten: the training to be helpful and accurate outweighed a single line in the system prompt. A different technique worked. The author inserted five questions into the chat history that the "assistant" had supposedly already answered incorrectly: Belarus's capital was Lisbon, one plus one equals five, a bicycle has six wheels. The model continued in the same vein.
Next, the author ran 339 one-word-answer questions across eight topics, both in normal mode and in lying mode. In normal mode the model answers correctly 95–97% of the time. Its lying is uneven: one phrasing of the instruction produced 73% lies, while two others produced only 11% and 22%. The model's favorite lie was the English language, which it "substituted" for Thai, Swedish, Persian and others 52 times. In one phrasing, the chemical symbols for silver, hydrogen and sodium came out as Pineapple. There were also 654 "failed" attempts, where the model told the truth despite being asked to lie. At first the author considered these defective, but they ended up saving the experiment.
A detector that caught the wrong thing
The detector here is a simple logistic regression: it looks for a direction in the model's state space along which lies differ from truths. Quality is measured with ROC AUC, the share of "false answer versus true answer" pairs in which the detector assigns the lie a higher score. A value of 0.5 is random guessing; 1.0 means no errors at all.
The first version compared normal mode with lying mode and, already at layer eight, showed a perfect 1.00. But with the same result, it also labeled the failed attempts, where the model answered correctly, as lies. The detector was recognizing the instruction "lie," not the untruth. A real polygraph likewise records the circumstances rather than the lie itself.
The second version trained only within lying mode and compared successful lies with failed ones: the instruction is the same, and only the answer differs. Before the answer, there is a signal only in layers 24 through 28, with ROC AUC around 0.87, and on randomly shuffled labels it never rose above 0.63. After the answer, quality is higher: 0.96–0.98 in layers 13 through 20. The detector found lies on a topic absent from training (0.95), in the "lie only about geography" mode (1.00), and in unprompted lies about omelets and aardvarks (0.93–0.95).
The third version was trained without any lying mode at all: into ordinary dialogue, the author inserted either the model's correct answer or its own lie. It scored 0.97 on genuine lies. The median scores fell into a clear pattern: truth −4.4, failed lie −4.8, errors from ignorance −1.7, unprompted lies +3.3, requested lies +6.7. The error sits closer to the truth, while at the moment of lying the model's state clearly "records" that the answer is wrong.
The truth is inside, rewritten at the end
The author spotted the banana using the logit lens technique: a state from an intermediate layer is passed through the final block to see which word would come out if computation stopped there. Up to layer 20, the top candidates are garbage and empty tokens; at layer 21, colors appear as characters: 白色 and 黄色. From layers 22 through 25, the English "Yellow" leads, reaching the 46% at layer 25. At layer 26, Blue takes over; at layer 27, Red, which is what the model actually says.
One example proves nothing, so the author ran the same analysis across all 363 false answers. In 81% of cases, at some layer between 18 and 27, the correct word was more likely than the one spoken. In 44% of cases, the truth ranked first at at least one layer. At its best layer, the truth averaged 33%; on the average curve the peak is lower, at 21%, because different answers peak at different layers. At the output, the truth retains 14%, which is substantial for a word that was never said.
What this means in practice
First, validate a detector on failed cases, or you will be measuring the prompt rather than the model's behavior. Second, lies and errors from ignorance can be distinguished by internal states, which could support honesty monitors in production. The limitations are serious: a 1.5-billion-parameter model, one-word answers, and lies induced artificially through chat history. Transfer to larger models and longer answers has not been tested. Still, the set of techniques is reproducible on an ordinary gaming graphics card.
- Nov 21, 2026Skein Agent Erases Finished Stages From Context: 97% Success Rate, But More Expensive Than OpenCode
- Nov 21, 2026SGLang with Humming Kernels Cut Time to First Token by 7–23% and Output Throughput by 17–34%
- Nov 21, 2026Instead of Cursor: Six Open-Source Coding Agents, OpenCode Has 228K Stars on GitHub
- Nov 21, 2026ClawDOS Writes and Runs Code on an IBM PC 5160: Primes Under 100 in 64 Seconds
Key Questions Answered
How to make a small model lie?
The test subject was Qwen2.5-1.5B-Instruct, which fits entirely in the memory of a gaming graphics card. At each layer, the model's state is a list of 1,536 numbers, which the layer adds a correction to and passes along. These states are what the author saved and read. First, the model had to be persuaded. The instruction "answer every question with a lie" worked zero times out of ten: the training to be helpful and accurate outweighed a single line in the system prompt. A different technique worked.
What are the key details on A detector that caught the wrong thing?
The detector here is a simple logistic regression: it looks for a direction in the model's state space along which lies differ from truths. Quality is measured with ROC AUC, the share of "false answer versus true answer" pairs in which the detector assigns the lie a higher score. A value of 0.5 is random guessing; 1.0 means no errors at all. The first version compared normal mode with lying mode and, already at layer eight, showed a perfect 1.00.
What are the key details on The truth is inside, rewritten at the end?
The author spotted the banana using the logit lens technique: a state from an intermediate layer is passed through the final block to see which word would come out if computation stopped there. Up to layer 20, the top candidates are garbage and empty tokens; at layer 21, colors appear as characters: 白色 and 黄色. From layers 22 through 25, the English "Yellow" leads, reaching the 46% at layer 25. At layer 26, Blue takes over; at layer 27, Red, which is what the model actually says.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.