Галлюцинации от «умных» промптов: разработчик разобрался, почему Qwen Code сбоит

A developer published a detailed breakdown on Habr: a CLI agent based on Qwen Code spent approximately two hours on a task that a "find and replace" tool in an IDE would solve in 20 minutes — due to continuous hallucinations and addition of unnecessary artifacts.

When an agent is worse than simple find and replace

The task looked trivial: make uniform changes to a codebase using a CLI agent. Qwen Code — a code-specialized model, available to the developer in a work environment instead of the preferred Claude — should have handled it quickly. Instead, the agent worked for about two hours: it added unnecessary constructs, lost the context of the task, and required constant manual correction.

The author continued attempting to guide the model "already out of sporting interest" — and this very experience forced him to finally understand the nature of hallucinations instead of simply switching tools.

Key parameters of the situation:

• Task: uniform code changes via CLI agent
• Expected time: 20 minutes with a manual tool
• Actual time: approximately 2 hours with an agent
• Model: Qwen Code
• Symptoms: hallucinations and addition of extra artifacts

Why do "smart" prompts amplify hallucinations?

The central paradox of the article — attempts to write more detailed and "smart" prompts do not reduce the number of hallucinations, but often increase them.

Context overload. A long prompt with many conditions and exceptions increases the probability that the model will lose the thread and start reconstructing logic from its own weights instead of following the instruction.

Conflicting constraints. Detailed conditions may implicitly contradict each other. The model does not return an error — it chooses between them and chooses incorrectly, generating a plausible but incorrect result.

Exceeding distribution boundaries. Code-specialized models were trained on a specific type of requests. Non-standard or composite prompts push the model into a zone where there were few examples during training — and hallucinations sharply increase there.

Illusion of understanding. A detailed instruction creates an impression in the developer that the model understood the task. The model is simply generating the next token based on similar patterns from training data.

What should a developer do with an AI agent?

The experience described in the article points to several practical conclusions:

• Atomize tasks: one agent call — one specific operation, not a set of interconnected steps
• Verify artifacts: the result of each step should be verified before passing it further down the pipeline
• Simplify, do not complicate: if the model is hallucinating, try a shorter and more specific prompt
• Choose the tool for the task: for uniform mechanical changes, grep or a script is often more reliable than an agent

"Only after this remarkable experience did I finally decide to read

the instructions," the author writes.

What this means

LLM hallucinations are not a bug that a smarter prompt will fix. This is a systemic property of autoregressive models, which is amplified when exceeding the training distribution. Understanding this once means building AI pipelines with the real limitations of the tool in mind, not against them.