Enthusiast builds a RAG bot on a T4 for 9th- and 10th-grade social studies textbooks

From the new school year, schools switch to unified social studies textbooks, and a student's answer is checked against the textbook text rather than against general common sense. A classic RAG setup fits this mode: the bot does not reason "in general" but pulls fragments from the book and answers from them. The author of a Habr post built such a prototype for 9th- and 10th-grade textbooks. It names the page each answer comes from and runs on a free Nvidia T4 GPU with 16 GB of memory.

What the prototype is made of

The whole stack is open and fits on a single accelerator. pymupdf4llm parses the PDFs, turning pages into structured Markdown and preserving the hierarchy of headings and paragraphs. This matters for a textbook: paragraphs, definitions and sidebars carry meaning, and if the text is chopped blindly, search starts returning fragments.

The rest is a standard pipeline. multilingual-e5-large computes the embeddings, and FAISS handles the index and nearest-neighbor search. Qwen2.5-3B, a small 3-billion-parameter model from Alibaba, writes the answer.

• PDF parsing: pymupdf4llm
• Vector search: FAISS
• Embeddings: multilingual-e5-large (about 560 million parameters, 1024-dimensional vectors)
• Generation: Qwen2.5-3B
• Hardware: Nvidia T4, 16 GB of VRAM, free tier of cloud notebooks
• Corpus: social studies textbooks for 9th and 10th grades

Why a page reference

The bot's main function is not paraphrasing but attribution. If a grade depends on the wording of a specific textbook, students and teachers need an answer that is verifiable, not just "plausible." The page number turns the model's output into a hint that can be opened and checked against the book in a few seconds. For a 3-billion-parameter model, it is also a safety net. Such models noticeably more often mix up details and make things up when context is thin. Tying the answer tightly to the retrieved fragments reduces that risk but does not remove it entirely.

The model choice is also easy to explain. Qwen2.5-3B in half precision takes about 6 GB, and the embedder another 1 GB or so, so both fit on a T4 with room left for context. Multilingual E5 remains a common choice among open embedders for Russian. It needs the query and passage prefixes, and without them retrieval quality drops.

The announcement includes no measurements: no share of correct answers, no retrieval accuracy, no generation speed. So for now the prototype should be judged as a working design, not as a proven product.

How to replicate it and what to watch

If you want to build a similar system for your own study materials, the order is as follows:

• Convert the PDFs to Markdown and manually check several pages with tables, diagrams and footnotes. This is where parsers break most often.
• Split the text by semantic blocks (section, paragraph) rather than by a fixed number of characters, and store the page number in the metadata. Without it, citing the source is impossible.
• Put together a small set of test questions, say 50–100, with reference pages. It lets you measure whether the right fragment lands in the top 3 or top 5 results, even before you plug in generation.
• In the prompt, require the model to answer only from the retrieved context and to say honestly, "the textbook doesn't cover this." For a school scenario, a refusal is better than an invented formulation.
• If 3B quality is not enough, the next step is obvious: a 7–8-billion-parameter model with 4-bit quantization also fits on a T4.

The full walkthrough with code is in the original Habr article. The idea itself is instructive. What drives RAG here is not fashion but a change in grading rules: when an answer must match a specific text, source retrieval is needed before "smart" generation.