Microsoft GraphRAG and Ollama: How Graph RAG Performed on Local Models

A detailed review of Microsoft GraphRAG paired with Ollama and local LLMs has appeared. The author checked whether a graph RAG can be set up without expensive infrastructure, and ran the system on William Gibson's short story "Johnny Mnemonic" to judge answer quality on familiar cyberpunk material.

How the test was set up

The experiment centered on a practical question: can a graph approach really replace the familiar vector search in enterprise RAG systems? The author chose Microsoft GraphRAG, a local run through Ollama and a text of about 38 thousand words. The result was not just a search index but a full knowledge graph with entities, relationships and communities.

Visualization in Gephi showed that the system can build a fairly rich structure from a single work of fiction. Notably, GraphRAG forms not only literal links between objects but also thematic clusters. In the community reports, groups formed around the Yakuza, Johnny, Molly Millions and other key elements of the plot.

Typical limitations also turned up: entities are not always merged when names differ in form, so some duplicates have to be accounted for separately. For answers in Russian, the author adjusted the system prompts, though he advises asking the questions in English, otherwise accuracy drops.

"In short, it kind of works even on 4b models."

How the system answers

The test compared several modes. Global search works over community descriptions using MapReduce logic and suits questions about the whole corpus. Local search blends graph relationships with fragments of the source text and is more useful when you need to dig into a specific character, object or episode. There is also BASIC, a plain chunk search, and DRIFT, a heavier mode that resembles query expansion and tries to widen the context.

• Global search pulled out the story's main cyberpunk themes: the merging of technology and biology, a dystopian city, corporate conflict and technological inequality.
• Local search gave a more concrete answer about the character Jones and his ties to Johnny, Molly and Yakuza.
• DRIFT search took about forty minutes on the same question and gave no noticeable jump in quality over local mode.
• BASIC remains a useful control point, because vector search does not disappear inside GraphRAG.

From this the author draws an important practical conclusion: a real product will need a separate agent or router that picks the search type based on the wording of the question and the query history. Otherwise all modes have to be switched manually.

Another detail: GraphRAG answers cite human_readable_id values from parquet files, so for a user interface these references must be additionally resolved and processed. This turns GraphRAG from plain search into a layer that has to be adapted to a live user scenario.

Where the problems arose

With local models the picture was uneven. Of the examples found, Mistral 7B failed at global search because of problems with structured JSON output: map queries simply crash. Gemma 3 in 4B and 12B versions kept the main entities but simplified the graph and in places distorted facts, even to the point that Jones turned into a human instead of a dolphin. The author named Qwen3 14B the most workable option. For embeddings he used the user-bge-m3 model, which behaves well in both Russian and English.

There are many infrastructure nuances too. GraphRAG relies on LiteLLM, and the author specifically warns against going above version 1.82.6, because 1.82.7 and 1.82.8 were compromised. With Ollama, a false 404 error appears when requesting model parameters, and long calls can hit queue timeouts. Embeddings behave even worse: bge-m3 via Ollama sometimes crashes because of Inf and NaN serialization, so the embedding had to be moved to a separate proxy on HuggingFace. On top of that, you must edit settings.yaml by hand, set api_base and a vector size of 1024, and enable graphml for visualization. Even on a machine with a 16 GB GPU, indexing a text of this size takes more than an hour.

What this means

The article's main conclusion is that Microsoft GraphRAG does not look like a direct replacement for classic vector RAG. It is more useful where depth of semantic links matters more than response speed: analytics, expert systems and complex document corpora. The approach already has an API, a test application and a clear path to an MVP. But more relevant answers come at the price of indexing time, a brittle pipeline and noticeably more complex setup than a regular vector database.