Live Nov 18 · Cohort 3Nov 18, 19:00 MSK — free hands-on AI agents live workshop · Live Cohort 3 starts Nov 23Join Nov 18 Live →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableHabr AI→ original

Developer cuts AI agent context by 97–98%: 100 KB goes to the model instead of 4 MB

Feed an AI agent a 500-file project whole and the prompt balloons to 4 MB. The model invents methods and runs into the context limit. A developer writing on Habr built a filter in PHP and SQL that picks 10–20 files in advance based on the words in the task, ranking them by word frequency (TF-IDF). About 100 KB goes to the model instead of 4 MB, a reduction of 97–98%, and a search on a ready index takes 1.5–10 ms. For code, exact names are more reliable than embeddings.

AI-processed from Habr AI; edited by Hamidun News
Developer cuts AI agent context by 97–98%: 100 KB goes to the model instead of 4 MB
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
Key metric #1First indexing of 500 files takes 8–30 seconds, with a median of 28
Zero calls to a paid embedding APIeverything runs on the CPU
Key metric #3Path and role bonuses surface files that don't yet contain any of the task's words

Feed an AI agent a 500-file project whole and the prompt swells to several megabytes. The model starts inventing classes and methods that don't exist, answers slowly, and sometimes hits the context window limit, wasting the API budget. The developer who wrote the Habr post solved this without vector databases or paid embedding services. She built a small module in PHP and SerenaDB that picks the 10–20 relevant files in advance based on the text of the task. As a result, the context shrank from about 4 MB to 100 KB, a reduction of 97–98%.

How file selection works

The module takes the project root and the task text, and returns a list of paths worth showing to the model. It does no code generation and no "semantic" search, only selection. Everything runs on an ordinary CPU.

First, the code is split into words. Names like ItemService and signing_date are broken into parts: item, service, signing, date. This has to happen before lowercasing, otherwise the word boundaries in camelCase disappear. Language keywords (public, return, null) are discarded. Russian words from the task go through a lightweight stemmer, which strips endings and keeps a stem of at least four characters.

Next, a table records the link "word → file → number of occurrences". The file's size and modification time are saved too, so on a repeat run only new and changed files are reindexed. The first 30 KB of content is kept for future snippets.

Ranking runs in two layers. The first is computed in SQL using TF-IDF. Put simply, a word that appears in three of 500 files carries a lot of weight, while a word found almost everywhere means almost nothing.

Large files are dampened by dividing by the logarithm of their size, so they don't win simply on volume. The second layer is computed in PHP. It adds a bonus if a word from the task appears in the file's path, and applies a multiplier by file type: .

php weighs 1.0, tests 0.6, .

md documentation 0.3. If SerenaDB doesn't accept the LN and GREATEST functions, a fallback mode kicks in that uses a simple sum of frequencies.

The logs always show which mode ran.

Why not vectors or BM25

In code, what matters are specific methods, classes and variables, not closeness in meaning, so exact identifier matches behave more predictably. Besides, SerenaDB supports neither pgvector nor proper full-text search. There is another advantage: the result is explainable. For every file you can see which words, at what frequency, put it in the top.

The author also considered BM25, a popular ranking standard, and rejected it. The main reason isn't the formula but the task itself. BM25 improves the ordering among files that already contain the query words.

But the file you need often doesn't contain them at all. For example, the request is to "fix DTO keys in contracts", and ItemDto.php contains neither "keys" nor "fix": they are yet to be added there.

No ranking based on matches will lift such a file. That work is done by role terms (the word "contract" adds related names to the query) and the path bonus, and they delivered the main gain. In addition, the author has no labeled sample for tuning BM25 parameters, and word frequencies in source code are usually 3–20, so term-frequency saturation barely works there.

Numbers and limitations

On a backend of ~500 files, the context for the model shrank from ≈4 MB to 84–111 KB, against a target of no more than 150 KB. A search on a warm index takes 1.5–10 ms, and first indexing takes 8–30 seconds (median 28). There are no embedding API calls. On a frontend of ~1200 files over a Docker bind mount, the first pass takes up to 80 seconds. The author doesn't count the LLM's own response time, since it depends on the external API more than on this code.

There are limitations too. The validation is small: the target file landing in the top 7 is confirmed on a single case, not on a labeled set of 10–20 tasks. The role and synonym rules are written for a specific domain, and another project will need its own. The path bonus works only if files have meaningful names.

The practical takeaway is simple. Before connecting a vector database to a coding agent, try the cheap option: tokenization with camelCase splitting, TF-IDF in an ordinary SQL database, and a synonym dictionary for your domain. It is free, explains every decision, and saves tokens on every request. Embeddings are best kept for tasks that truly need search by meaning.

Story

Key Questions Answered

How file selection works?

The module takes the project root and the task text, and returns a list of paths worth showing to the model. It does no code generation and no "semantic" search, only selection. Everything runs on an ordinary CPU. First, the code is split into words. Names like ItemService and signing_date are broken into parts: item, service, signing, date. This has to happen before lowercasing, otherwise the word boundaries in camelCase disappear. Language keywords (public, return, null) are discarded.

Why not vectors or BM25?

In code, what matters are specific methods, classes and variables, not closeness in meaning, so exact identifier matches behave more predictably. Besides, SerenaDB supports neither pgvector nor proper full-text search. There is another advantage: the result is explainable. For every file you can see which words, at what frequency, put it in the top. The author also considered BM25, a popular ranking standard, and rejected it. The main reason isn't the formula but the task itself.

What are the key details on Numbers and limitations?

On a backend of ~500 files, the context for the model shrank from ≈4 MB to 84–111 KB, against a target of no more than 150 KB. A search on a warm index takes 1.5–10 ms, and first indexing takes 8–30 seconds (median 28). There are no embedding API calls. On a frontend of ~1200 files over a Docker bind mount, the first pass takes up to 80 seconds. The author doesn't count the LLM's own response time, since it depends on the external API more than on this code. There are limitations too.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…