Fudan and Ant Group researchers made an LLM follow a graph-based legal procedure

Almost all lawyers use AI, but very few trust it. The reason is not that a model may confuse facts. Far more dangerous is when it silently skips a procedural step and produces a polished report with something missing. We examine TG-RAG, an approach designed to solve precisely this problem, and what anyone building agents for regulated industries can learn from it.

The gap between what is possible and what happens in practice

According to a March 2026 Anthropic study, models are theoretically capable of performing almost 90% of tasks across legal professions, while in practice AI covers about 20%. An ILTA survey shows that 94% of law firms use or test generative AI, up from 80% a year earlier. Yet most tools remain pilots. Russia shows a similar picture: about 88% of lawyers already work with neural networks, while 84% fear unreliable information and 80% have to verify every result.

The author of the analysis, applied linguist Sergei Slepukhin, sees the root of the problem in the separation of knowledge. We already know how to store declarative knowledge—what rules and practice say—in vector and graph databases. Procedural knowledge—how to act—usually resides in prompts and files such as skills.md, meaning in the same kind of text as the data itself. An LLM has no boundary between data and instructions, so during a long dialogue it gradually departs from the original instruction. This is instruction drift, which the authors of TG-RAG call cognitive drift.

How TG-RAG works

The idea is simple. A reasoning model's reasoning is ordinary text between the <think> and </think> tags. If you run an open model on your own server—for example, Qwen or DeepSeek R1-Distill on vLLM—you control the chat template and can append this text on the model's behalf. It will continue it as its own thought and will not distinguish it from text it generated itself.

The system is built around this property. An expert procedure, or SOP, is formalized as a graph: vertices are analytical steps, while edges define their order and transition conditions. The software executes the graph itself. At the boundary of each step, it interrupts generation, inserts the directive for the next step into the reasoning, and supplies the knowledge needed at that exact moment. The procedure sets the channel, while database retrieval works selectively instead of dumping everything into a single context.

Consider due diligence for the acquisition of an ownership stake in an LLC. After documents are requested, the work branches into tracks covering corporate matters, assets, contracts, taxes, disputes, debt and bankruptcy, regulation, and sanctions. They all converge in a risk matrix. This requires both multi-hop work—collecting facts from different sources—and multi-step work—following every branch in sequence. The failure may not be an incorrect fact, but a skipped step that is difficult to spot during review.

Limitations and what to do about them

The first limitation is technical: the system needs access to the raw prompt and chat template. Closed APIs do not allow text to be inserted into the reasoning, at least not in this form. The approach is therefore tied to open models running on the organization's own infrastructure, which is not always a disadvantage for lawyers handling confidential data.

The second limitation is methodological. Someone must construct the procedural graph, and that is an expert's job. The expert must describe the SOP and transition conditions, and the graph must be maintained as regulations change. The approach fits tasks with a standard or describable procedure, but not open-ended creative analysis. Moreover, there are no recognized benchmarks for Russian-language law, making it difficult to measure the gain honestly. The author explicitly notes this and promises an experiment on proprietary data, along with an analysis of the limitations, in the second part. Results from the paper's authors are not yet available in this part.

The practical takeaway for developers and product teams is this: if your agent operates in a domain with a strict procedure—law, compliance, medicine, or finance—stop keeping that procedure inside the prompt. Move it into an explicit structure, such as a graph, and execute it programmatically. Supply knowledge one step at a time. The output then becomes easier to audit: instead of reviewing the entire chain of reasoning, you check whether the mandatory graph nodes were traversed.