ChatCourt's AI Court Author Cut GigaChat Token Usage per Case Run from 1.7M to 0.55M
ChatCourt is an arbitration court simulator in which neural networks play the judge and the opponent, while a student acts as plaintiff or defendant. After the switch from Claude Sonnet to GigaChat Ultra, the model got stuck in loops, and a single case run consumed up to 1.7M tokens. A narrow menu of 1–3 actions plus the provider's cache brought usage down to 0.55M tokens, and the number of calls fell to 62 from 100. A model that is weak at function calling is better kept within tight constraints.
AI-processed from Habr AI; edited by Hamidun News
ChatCourt is a training simulator for arbitration court proceedings. A student plays the plaintiff or the defendant, while neural networks portray the judge, the opponent, and the advisor. Tokens are units of text that you pay for when you call a model.
The project's author built the trainer on Claude Sonnet, but production needs the Russian model GigaChat Ultra, and everything broke during the migration. One full run of a case on Sonnet consumed 1.2–1.
7M input tokens, while on GigaChat, after the reworks, it took about 0.55M, and the number of model calls dropped to 62 from a peak of 100. For anyone building agents on a model that is not especially agent-oriented, this is a set of ready-made lessons.
What the trainer is and why it runs on a Russian model
The idea came from an MIT experiment: seven student teams spent a month building a small jet engine with the help of AI. Two teams made it to testing, and one engine fired. The neural networks hallucinated and sycophantically agreed with users, while the winners treated the hints more cautiously and relied on their own knowledge. The author decided to put the student in the same position.
The student chooses a side and goes through six stages: preparation, preliminary hearing, trial, evidence and motions, closing arguments, and the verdict. The AI judge issues the decision. There are no correct answers: each case develops in its own way, and the instructor grades the course of the proceedings, not the outcome.
The model has to be Russian: universities face restrictions on foreign AI services, and Chinese models fall under them too. Technically, it is a synchronous monolith on FastAPI with SQLite, which is enough for a pilot with a few students. The guiding principle: the code decides what is permissible, and the model decides what to do next. The author rejected the LangGraph framework. Role isolation would have had to be written by hand, since the judge must not see the parties' strategies. A shared checkpointer would have stored students' texts alongside everything else.
Compatibility and looping
Tests ran on Sonnet, and the author assumed the function-calling format was the same for both models. It turned out that Sonnet calls several functions in a single step, while GigaChat calls only one. The engine turned out to be nearly incompatible with the new model, and it was rewritten.
Next, the model began repeating the same phrase. When the engine rejects a model response, it sends the same prompt with an error note attached. Sometimes the prompt goes out unchanged, and at near-zero temperature the model picks the most probable word every time. GigaChat does not accept zero. The temperatures were tuned by hand: 0.2 for the judge, 0.3 for the AI side and the hints, 0.01 for service checks. They cannot be raised: the model stops following instructions and starts hallucinating.
A "pause" mode guarded against endless cycles: the engine detects repetitions and halts the process. The root cause was fixed by editing the prompt. It received an explanation of what went wrong and what to do instead, along with 45 rejection codes, each with a correction example. In addition, a log of the proceedings was added, listing the proposals that had already been rejected. If a response was cut off by length, the retry carries the note "write shorter" and a limit of 4,000 tokens.
Even that was not enough: in one run, the judge asked the defendant questions seven times in a row, even though the floor belonged to the student. So a "guided" mode was introduced. The engine itself selects the step type and offers the model one to three actions. An action rejected for repetition is removed from the menu until the end of the turn. If the floor belongs to the student, the judge may only question the student.
Memory, caching, and the final economics
The model has no memory of its own. Before each call, the engine rebuilds a "table" for the role from seven sections: the role's position, the case inventory, transcripts of past hearings, up to 15 role notes, a working section, the last 10–15 remarks, and the proceedings log. Nothing persists after the call.
The provider-side cache saved a noticeable share of the budget. An X-Session-ID header was added to every call, a hash of the "attempt and role" pair. Without it, the cache barely hit. The unchanged sections were placed at the beginning of the request, and the changing ones at the end. On a sample of 25,981 input tokens, 25,920 were served from the cache. The cache share in some cases rose from about 8% to 44%, and cached tokens are billed at a lower rate.
The total for a full run: about 0.55M tokens, of which 0.39M are billed at the full rate and 0.17M come from the cache (30% on average). A run takes 7.5 minutes without pauses. The share of empty responses across four versions fell from 30% to 21%, then to 12.5%, and finally to 0%.
What the trainer cannot do yet
The judge and the opponent rely on the case materials, the script, and the model's own knowledge. They have no RAG layer with court-practice or current-legislation databases, whereas foreign analogues do have such a layer. The risk of substantive errors by the AI judge has not gone away. A beta test with students is underway now, and there are no results yet.
The practical takeaway: do not give a model that is weak at function calling freedom of choice; constrain the action menu and return errors with a concrete correction example. Put the unchanged parts of the prompt at the start so the cache works, and catch repetitions on the code side.
- Nov 21, 2026Opus rewrote llama.cpp code: Qwen runs 40% faster on a CPU alone
- Nov 21, 2026Jensen Huang Steps Down as NVIDIA CEO After 33 Years, CFO Colette Kress Becomes Interim CEO
- Nov 21, 2026OpenAI Disbands Robotics Team Again: GPT-6 Astra Completed 74.8% of Tasks in Simulation Versus 51%
- Nov 21, 2026AWS gives agents a shared LLM cache skill: up to 40% cheaper, first token 80% faster
Key Questions Answered
What the trainer is and why it runs on a Russian model?
The idea came from an MIT experiment: seven student teams spent a month building a small jet engine with the help of AI. Two teams made it to testing, and one engine fired. The neural networks hallucinated and sycophantically agreed with users, while the winners treated the hints more cautiously and relied on their own knowledge. The author decided to put the student in the same position.
What are the key details on Compatibility and looping?
Tests ran on Sonnet, and the author assumed the function-calling format was the same for both models. It turned out that Sonnet calls several functions in a single step, while GigaChat calls only one. The engine turned out to be nearly incompatible with the new model, and it was rewritten. Next, the model began repeating the same phrase. When the engine rejects a model response, it sends the same prompt with an error note attached.
What are the key details on Memory, caching, and the final economics?
The model has no memory of its own. Before each call, the engine rebuilds a "table" for the role from seven sections: the role's position, the case inventory, transcripts of past hearings, up to 15 role notes, a working section, the last 10–15 remarks, and the proceedings log. Nothing persists after the call. The provider-side cache saved a noticeable share of the budget. An X-Session-ID header was added to every call, a hash of the "attempt and role" pair. Without it, the cache barely hit.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.