Локальный RAG на FastAPI и Ollama: от простого вызова LLM к production backend

A developer on Habr breaks down a specific moment: when a single call to a local model no longer suffices — and what needs to be added to get a real backend system.

When a simple call is not enough

The first version looks straightforward: frontend sends a question, FastAPI receives `POST /ask`, backend calls Ollama and returns a string with the answer. For a demo, that's enough. For a documentation assistant — no. Questions arise immediately: which documents does the answer rely on? Which fragments made it into the prompt? Why does one request take two seconds to process, while another takes fifteen? What happens if the index hasn't been updated in three weeks? You can't answer these questions with a simple LLM call.

What the backend layer adds

The author introduces components one by one, explaining the specific problem each one solves: request_id — a unique request identifier for correlating logs and debugging specific failures sources — a list of document fragments that made it into the model's context, with source attribution timings — time breakdown by stage: index search, prompt formation, answer generation rebuild index — a separate endpoint for rebuilding the index without restarting the service * negative tests — checking behavior with an empty document corpus, questions outside the scope, and an outdated index The stack is entirely local: FastAPI for the HTTP layer, Ollama for generation, custom embeddings. No external API keys.

API contract as a system boundary

One of the article's key points — the importance of an explicit contract at the response level. When `/ask` returns only a string, the service remains a black box. Once the response includes `request_id`, `sources`, `timings`, and index status — it becomes possible to monitor, debug, and improve.

"I'm showing not how RAG works in general, but the path from a simple local LLM call to a small backend project with an API contract, logging, sources, timings, and honest limitations".

This approach changes how you view the service: instead of "the model somehow answers," you get "the system behaves predictably."

Honesty about limitations

The author explicitly lists what's missing from the project: authorization, caching, multi-user load support, CI/CD. This makes the article more useful than most tutorials, where the limitations section either doesn't exist or amounts to a standard disclaimer. Negative tests are a separate topic. The behavior is checked with an empty document corpus, with questions lacking relevant content in the index, and when accessing the index after adding new files without rebuilding. A minimal set, but enough to understand exactly where the system fails.

What this means

The article fills a specific gap: most RAG materials either explain theory or show "hello world" with LangChain. Here — a working local service with observability tools that can serve as a foundation for a real project.