Narrow AI Agent vs. Universal Swiss Army Knife: How We Cut Response Latency from 17 to 7.7 Seconds

• Reduced response latency: median dropped from ~17 s (13.3–20.6 s) to 7.7 s (7.25–7.93 s)
• Compressed input prefix: from 15,266 to 1,424 tokens per request (a 10.7x savings with no change to business logic)
• Size of the deterministic core: 766 lines of a Python gamification plugin on top of SQLite, plus 342 lines of a notation verifier
• Practical ceiling for the personality file: around 20,000 bytes for `SOUL.md` before the model loses attention to the middle of the instructions

The first temptation when assembling an AI agent is to give it access to everything at once: terminal, browser, web search, files, and MCP servers. The sobering moment came with a children's chess trainer running on a local `/v1/chat/completions` endpoint. A child would move a piece on the board, and the coach's comment would arrive 17 seconds later — by which point the young player had already made two more moves.

Analysis showed that of the 15,266 tokens in the input prefix, nearly 14,000 were taken up by tool JSON schemas that were never called during the chess match, plus an automatic OS environment scan on startup. What's more, the agent, given access to `execute_code`, would start writing and running scripts in the sandbox at the slightest difficulty, turning a single move into twenty API calls.

How do three lines in a config file shrink the prompt by 10.7x and cut latency?

Turning a thoughtful generalist into a fast service didn't require rewriting the plugin's code. We locked down three runtime configuration parameters:

1. We restricted the `api_server` platform to a single domain tool set (`[chess]`), removing the file and shell toolsets. 2. We disabled `environment_probe`, which polled the host on every startup and packed an environment snapshot into the system prompt. 3. We set `reasoning_effort: none` — since the tactical facts are already computed by the Stockfish engine, the model doesn't need to build a chain of reasoning before responding; its job is simply to voice the ready-made verdict in a lively way.

In control measurements with a warmed-up cache, response time shifted from `13.3 / 17.3 / 20.6 s` to `7.25 / 7.74 / 7.93 s` (median 7.7 s), and input token consumption dropped 10.7x — from 15,266 to 1,424. Trimming the toolset saves tokens and protects the system from unauthorized terminal activity, while disabling the extra reasoning step delivers the main gain in seconds. One important detail: tool profiles for the messenger and for the scheduler (`cron`) are set separately — if you forget to register the plugin in the `cron` section, the morning digest will silently try to fetch data by reading files instead.

How do you split a personality into two layers and offload streak

calculation to SQLite?

A narrow agent's personality is designed in two tiers: a short trigger block in `config.yaml` (10–15 lines) handles dispatching (`tutor_state` at the start of a conversation, `tutor_record_lesson` after an exercise), while the tone and pedagogy rules live in `SOUL.md`. The practical ceiling for this file is around 20,000 bytes (my bots run 6.7–19.9 KB): push it further and the model starts losing rules from the middle of the file.

A language model is great at playing out emotions but bad at calculating dates, experience points, and rubric-based scores. In the language tutor, all the gamification math (10 levels from 0 to 8,000 XP, streak milestones from 3 to 365 days, 15 badges, and crystals) is offloaded to a 766-line Python plugin on top of SQLite:

```python if last_date == today: streak = max(1, prev_streak) elif last_date == yesterday or not last_date: streak = (prev_streak + 1) if last_date == yesterday else 1 else: gap_days = (parse_date(today) - parse_date(last_date)).days if freezes > 0 and gap_days == 2: freezes -= 1 streak = prev_streak + 1 events.append({"type": "freeze_used", "msg": "A freeze saved your streak!"}) else: if prev_streak > 1: events.append({"type": "streak_reset", "old": prev_streak}) streak = 1 ```

The tool returns an `events` array, and the `description` field of the function's own JSON schema has a directive baked in: congratulate the student emotionally for every event in `events` and don't invent anything about the balance on your own. Similarly, in the procurement negotiation trainer, a rubric of 4 modules on a 40-point scale checks stem-markers (risk flags, fact-checking, bargaining) via code, so a participant can't "talk" the neural network into raising the score.

Why did progress silently reset after a restart, and how do you fix

the framework?

In the first builds, the engine passed a `task_id` parameter to the tool handler equal to the current session's ID, and I used that as the student's primary key. Pressing `/new` or restarting the container generated a new `task_id`, and the person would see a blank slate instead of 30 days of learning. The fix was a two-tier key resolver that uses a regex to extract a persistent account ID from `telegram:dm:<user_id>` or cross-checks `sessions.json`, guaranteeing a stable key of `tg:<id>`.

To avoid rebuilding a gigabyte-sized image for every library, the deployment installs packages via `pip install --target` into the `data/pylibs` volume, runs `chown -R` for the non-root user, and cleans up `__pycache__`. And in places where the base transport lacked a bottom keyboard or was swallowing `/start`, self-healing patches from `bootstrap.sh` check for anchors in the adapter's code at startup, inject the missing block, and verify the file via `py_compile`.

The pattern of "a deterministic plugin does the calculations, a narrow model plays the role" is built into the foundation of [Nomad AI](https://nomadnet.ai) and the [agent.nomadnet.ai](https://agent.nomadnet.ai) cloud environment. Labs on building such agents are available at [Hamidun Academy](https://academy.hamidun.com) (72-hour free trial), and we break down ready-made plugins and 370+ skills in [Vibe Club](https://hamidun.com/club).