IBM Research Cuts Inference Costs 11.5x With Open-Source Agents

IBM Research says that open models combined with the llm-d framework let it explore hardware solutions for quantum error correction at an inference cost 11.5 times lower than in a fully proprietary configuration. The setup is an "agent factory": a system in which many AI agents work in parallel on chip design. It is one of the few cases where the economics of an agentic workload are expressed as a concrete multiplier.

What IBM claimed and what it did not disclose

The wording in the announcement is brief. It mentions open models, llm-d, a quantum error correction task, and a comparison against a configuration in which every component is proprietary. It does not give a list of models, the number of GPUs, the volume of tokens processed, or the quality of the results compared with the closed setup. Without that, 11.5× reads as a cost under specific conditions, not as a universal constant. It is unknown which proprietary stack served as the baseline or how the costs were calculated.

A price cut of more than tenfold is never free. The question is how comparable the agents' results are on the two configurations. IBM has not yet shown what was actually produced as output.

The role of llm-d

llm-d is an open, Kubernetes-native framework for distributed inference. It is built on top of vLLM and was launched in 2025 with the participation of Red Hat, IBM Research, Google, NVIDIA and CoreWeave. Its key ideas are splitting the prefill and decode stages across separate resource pools and routing requests with KV-cache awareness.

This matters for agentic systems for a simple reason. Chip-design agents run long shared prefixes: specifications, simulation logs, architecture descriptions. There are many requests, and their contexts often overlap. If the router sends them to replicas that already have a warm cache, redundant computation disappears. IBM's announcement does not break down how much llm-d contributed to the overall 11.5×, so this is engineering logic, not confirmed attribution.

Why quantum error correction

Hardware solutions for error correction face an enormous space of options: codes, decoders, connectivity topologies, latency constraints. Searching through them by hand takes a long time. An agentic approach turns the search into a stream of iterations: propose a configuration, run a simulation, analyze the result, adjust. Each cycle involves several LLM calls, and there are thousands of such cycles.

Under this kind of load, inference becomes the biggest line item in the research budget. A multiplier of 11.5 means that the same money can fund an order of magnitude more experiments, or keep agents running continuously.

Key parameters

• Organization: IBM Research
• Field: chip design by agents, quantum error correction
• Stack: open models and llm-d
• Claimed savings: 11.5× versus a fully proprietary configuration
• Not disclosed: models, hardware, token volume, quality metrics

What teams can take from this

First: measure cost not per token but per successfully solved task. A cheap model with many failed attempts can lose out to an expensive one. Second: if the workload is agentic and the context repeats, evaluate the inference layer with caching in mind, not just the model itself. Third: a hybrid setup, where open models handle the bulk steps and closed ones step in for the hard ones, is often more cost-effective than either extreme. Finally, whenever you read a claim of "N times cheaper," first ask what the baseline was and how quality was verified. Complete data is not available here yet, and it is worth waiting for IBM's technical report.