BatchDAG: LLM Planner Builds an Operation Graph and Cuts Model Calls by Up to 47x

In July 2026, researchers published BatchDAG on arXiv, a system in which a language model builds an execution graph for a query over enterprise data and a separate engine runs it. The approach cuts model calls by up to 47x, and in production at Brevian.ai it handles queries over 50,000+ meetings in under 60 seconds.

How BatchDAG works

BatchDAG works in two steps. First, a large language model (LLM) generates a typed directed acyclic graph (DAG) of operations. Then a deterministic engine executes it, running nodes in parallel by topological wave. The graph's nodes are SQL queries, semantic search, in-memory transformations, parallel branches (fan-out) and single analyses, and data passes between them as structured JSON.

The BatchDAG planner produces a valid graph in 98.8% of cases, measured across 300 planning calls, according to the arXiv paper.

• LLM call reduction: up to 47x through entity-aware batching
• Valid DAG: 98.8% of 300 planner calls
• Scale: 50,000+ meetings processed in under 60 seconds
• Cost per query: $0.02–$0.24 at published GPT-5.1 prices
• Answer quality: 3.74/5 versus 3.25/5 for an expert pipeline

Why it hallucinates less

Structured JSON intermediate results reduce hallucinations by 27% compared with text summaries, according to the authors' controlled ablation experiment (paired t-test, p=0.107, n=12). Intermediate data with an explicit schema leaves the model less room to fill in facts between steps.

The key optimization is entity-aware batching: before a parallel fan-out, the system groups rows by logical entity, which is what yields the reduction of up to 47x in LLM calls. It also makes answers easier to verify: the share of answers backed by a quote from the transcript was 77% for BatchDAG, versus 46–60% for baseline approaches.

How it performed in tests

In controlled experiments on 12 queries over meeting transcripts, BatchDAG scored 3.74/5, ahead of the expert pipeline (3.25/5) and a ReAct agent (3.09/5, p<0.01). The authors stress that the goal of the system is not accuracy as such but generality.

"BatchDAG is less about an accuracy gain over hand-optimized pipelines

than a general-purpose orchestration layer that replaces many handwritten scripts with a single system," the arXiv paper says.

In production at Brevian.ai, the system runs on top of GPT-5.1 and keeps the cost of a single query between $0.02 and $0.24.

What this means

BatchDAG points to a practical approach for analytics over large enterprise datasets: instead of dozens of handwritten pipelines, one system translates a natural-language query into an execution plan on its own. For teams, that means faster, more verifiable answers at the scale of 50,000+ documents with response times under a minute, without manual tuning for each type of question.

Frequently asked questions

What is BatchDAG?

BatchDAG is an orchestration layer in which an LLM builds a typed graph of operations (SQL, semantic search, transformations) and a deterministic engine executes it in parallel. The system was presented in a paper on arXiv in July 2026.

How much does a query cost?

According to the authors, a single query costs between $0.02 and $0.24 at published GPT-5.1 prices, while processing a corpus of 50,000+ meetings in under 60 seconds.

How does BatchDAG differ from a ReAct agent?

BatchDAG builds the full execution graph up front and runs independent branches in parallel, whereas a ReAct agent acts step by step. In tests, this yielded 3.74/5 versus 3.09/5 for ReAct, and it removes the linear latency of sequential tool calls.