Codex agent built a personal knowledge base in Obsidian from 4 Habr articles instead of RAG
RAG, where a neural network searches documents for chunks and answers, "answers and forgets": tomorrow you have to rebuild the synthesis from the articles again. Ramil tested Karpathy's idea: the Codex agent maintains a knowledge base of markdown files in the Obsidian note-taking app. From 4 Habr articles he got cards, concept pages, and links. For personal research it works, but at 50–100 sources, loading articles into the base costs a noticeable number of tokens.
AI-processed from Raft; edited by Hamidun News
You read a dozen articles, asked a neural network to find what they have in common, got a great answer, and by the next day it was left in a closed chat tab. The knowledge didn't accumulate. This is how RAG usually works: the model finds suitable fragments in the uploaded documents and answers based on them.
Quality engineer Ramil tested a different idea, popularized by Andrej Karpathy: LLM Wiki. In it, the agent doesn't just answer; it maintains a personal knowledge base itself. The agent role went to Codex, the OpenAI assistant that can read and edit files in a project folder.
The base is Obsidian, a note-taking app that stores everything in plain markdown files and shows the links between them as a graph. The material was four articles from Habr.
How this differs from RAG
In the usual setup, you upload PDFs or articles, ask a question, the system pulls out fragments, and it generates an answer. For production applications with users, access rights, and SLAs, this is the right approach. But for personal work it has a drawback: it "answers and forgets." Suppose you read about guardrails, quality evaluation, OCR, and AI agents in BI. The model may produce a good synthesis, but if it stays in the chat, you will have to assemble it again.
LLM Wiki proposes compiling knowledge into a persistent layer of markdown files. The scheme has three parts:
- raw: source articles, PDFs, notes, and transcripts. This is the source of truth, and the agent does not edit them;
- wiki: the processed layer, including source cards, concept pages, links, syntheses, and saved answers;
- a rules file: instructions for the agent on how to add sources, how to name files, how to separate fact from interpretation, and what to update after each operation.
The author compares this to software development: Obsidian is the IDE for knowledge, markdown is the codebase, and Codex is the maintainer agent.
Three operations and a real experiment
The work rests on three commands. Ingest adds a new source: Codex reads the article, creates a card, extracts the idea, theses, limitations, and techniques, and updates or creates concept pages. The author stresses the difference: an ordinary retelling answers the question "what is the article about?", while ingest answers "how does this article change my knowledge base?" Query answers questions based on the wiki already accumulated, and a valuable answer is saved back into it, so the question also becomes part of the base. Lint checks the health of the base: it looks for duplicates, contradictions, outdated conclusions, concepts without pages, and "islands" in the graph. The author calls such problems knowledge smell, by analogy with code smell.
For the test, they took four Habr articles: about GLiNER Guard and guardrails for LLMs, about evaluating a classifier with DeepEval using GPT-4o and YandexGPT, about HunyuanOCR and document recognition, and about AI-powered BI with Apache Superset and MCP. Individually, these are four unrelated texts. After ingest, Codex saw a common theme: all of them are about LLM systems in production. One touches on security, another on metrics, a third on working with documents, and a fourth on agents with tools, permissions, and auditing. Concept pages appeared in the wiki, including information security, and Obsidian's graph view showed how the articles converge on shared nodes.
Technically it is simple: Codex runs in the project folder, reads the rules file, looks at the structure, and edits the markdown. The difference lies in how the task is framed. A request to "make a nice retelling" gives a random result, while a clear order ("don't touch raw, make the card using the template, update concepts, index, relations, and log, record the limitations") gives a predictable one. The author compares this to the difference between asking an intern a question and handing an engineer a repository with rules.
What it costs and where the limits are
The author's main warning: ingest is not a free retelling but an expensive operation of reading, analysis, and writing. Five sources can be processed almost painlessly. At 50–100 sources, especially multi-page PDFs, any bad tag, extra entity, or poor section in the scheme costs more: the model rereads large chunks of text, rewrites dozens of pages, and can spread the erroneous structure across the whole wiki. In the cloud, this is a direct token bill. A local model doesn't send a bill, but it requires a powerful machine, otherwise the quality of links on large datasets drops or you wait a long time.
To estimate the budget, the author calculated the cost on Yandex Foundation Models for his demo vault as of June 29, 2026, counting only what actually takes part in the process: the four source articles in raw and the generated wiki layer (sources, concepts, synthesis, index, relations, log).
Codex was chosen for convenience. The same process can be built with scripts: a watcher on the raw folder sends new files to a local or cloud model, receives a structured response, and regenerates the cards and indexes. After that, it is worth adding a human review step: the agent proposes a diff, and the person accepts or edits it.
What to take on
You can start small: a folder, markdown, Obsidian, an agent, and three commands. First debug the rules file on 3–5 sources and only then scale up. Count tokens for ingest in advance and leave the review of changes to a human, otherwise the agent will lock its own mistakes into the structure. The limitation is obvious: this is a personal tool for research and preparing materials, not a replacement for a production RAG with access rights and SLAs. The experiment is small, just four articles, so for behavior on hundreds of documents, one has to rely on the author's own warnings.
- Nov 21, 2026VibeCBS collects project nicknames in advance: the agent finds 2.8 times more documents
- Nov 21, 2026FDE job postings grew 4.6x in a year: what a forward deployed engineer actually does at the client's site
- Nov 21, 2026Sakana AI Opens Applied Research Lab for Market and Economic Modeling
- Nov 21, 2026Meta Is Reportedly Preparing the $300 Muse Charm AI Tamagotchi With a Year of Verizon 5G
Key Questions Answered
How this differs from RAG?
In the usual setup, you upload PDFs or articles, ask a question, the system pulls out fragments, and it generates an answer. For production applications with users, access rights, and SLAs, this is the right approach. But for personal work it has a drawback: it "answers and forgets." Suppose you read about guardrails, quality evaluation, OCR, and AI agents in BI. The model may produce a good synthesis, but if it stays in the chat, you will have to assemble it again.
What are the key details on Three operations and a real experiment?
The work rests on three commands. Ingest adds a new source: Codex reads the article, creates a card, extracts the idea, theses, limitations, and techniques, and updates or creates concept pages. The author stresses the difference: an ordinary retelling answers the question "what is the article about?", while ingest answers "how does this article change my knowledge base?" Query answers questions based on the wiki already accumulated, and a valuable answer is saved back into it, so the question…
What it costs and where the limits are?
The author's main warning: ingest is not a free retelling but an expensive operation of reading, analysis, and writing. Five sources can be processed almost painlessly. At 50–100 sources, especially multi-page PDFs, any bad tag, extra entity, or poor section in the scheme costs more: the model rereads large chunks of text, rewrites dozens of pages, and can spread the erroneous structure across the whole wiki. In the cloud, this is a direct token bill.
Build & Ship Production AI Agents Without Boilerplate Code
Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.