Habr explains how to make LLMs do math without errors by generating Python code

Habr has published a hands-on breakdown of why LLMs regularly get arithmetic wrong and how to work around the problem in a real product. Instead of asking the model to do the math itself, the author gives it a different job: write a Python script and hand the calculations off to an ordinary program.

Why LLMs make mistakes

The problem isn't that a particular chatbot has "gotten worse." A transformer predicts the next token by probability; it doesn't call a calculator. So when it multiplies numbers, scales a recipe, or works out a utility bill, the model can produce an answer that looks convincing but is off by a few percent, or even by tens of percent. To users this looks like degradation, but it's actually a fundamental limitation of the architecture: an LLM is good at reproducing the pattern of a calculation, but it doesn't perform the operation itself.

"The model doesn't calculate.

The model programs. And the program calculates."

That makes tasks where errors aren't immediately obvious especially risky. If someone can already check the result with pencil and paper, they don't need an LLM. But when people use a model precisely so they don't have to calculate by hand, a plausible-looking number easily slips through unchecked. The article gives the example of utility payments: the model may recall an outdated rate, multiply it "in its head," and present a neatly formatted answer even though the calculation underneath is wrong.

How the setup works

The setup is built around a division of roles. The user sends a task through a messenger, and the LLM receives a system prompt with context and the necessary data, then generates Python code. That code runs in an isolated Docker sandbox, and the service returns not just a line-by-line text breakdown but also a ready-made Excel file. In this scenario, the model is responsible for understanding the request and structuring the program, while arithmetic accuracy is handed entirely to the Python interpreter.

• Input can be meter readings, a spreadsheet, or a cost estimate
• Rates and reference data are fed into the prompt from a config file
• The model must return Python code, not a finished answer
• The script runs in an isolated container with a timeout
• The user gets a text breakdown and an Excel file as output

The author says he uses Qwen and DeepSeek for these tasks rather than expensive top-tier models. The reasoning is pragmatic: when you need a script of 20–200 lines, the gap in code quality between premium and more affordable models is small, while the price gap is significant. He puts particular emphasis on one point: rates and reference data must come into the prompt from a config file, not from the model's "memory." If a rate changes, updating a single line of data is enough, without touching the model itself.

Where things went wrong

The most common early mistake was asking the model to look up the rates itself. In that mode, it confidently plugs in outdated or irrelevant data, and the error looks plausible. So the author moved all sensitive numbers into the config and updates them separately from official sources. The second problem: some models, even after being instructed, still try to "calculate in their head" and return a finished answer. The fix is simple: validate that the response contains Python code and, if needed, resend the request with stricter wording.

More technical issues surfaced in practice, too: Cyrillic text in Excel broke without explicit UTF-8, the model pulled in unnecessary libraries like pandas, and without the full stderr it couldn't fix its own errors after a script crashed. Once the service started feeding the traceback back to the model, though, the number of wasted iterations dropped fivefold, according to the author. The same approach has already been applied to a more complex task, analyzing renovation cost estimates, where one test revealed an overcharge of 54 168 rubles and eight line items priced more than 50% above market.

What this means

The "LLM writes code, not the answer" approach looks like one of the most practical ways to use models where accuracy matters. For accounting, cost estimates, taxes, and any calculation-heavy scenario, it removes the main risk: the model handles the text, structure, and automation, while a regular program handles the numbers that need to be verifiable.