Coding Agents Built an LLM Engine for AWS FPGAs: 213 Tokens/s and 38% Cheaper Than CPU
Coding agents wrote an engine for running a language model on an FPGA, a programmable chip rented from AWS, and pushed its speed from 6.43 to 213.13 tokens/s. All 24 responses matched the CPU reference bit for bit. At nearly the same hourly price ($1.98 vs. $1.95), a million tokens costs $2.58 on the FPGA versus $4.18 on a CPU, which is 38% cheaper. But this is a calculation for a single model, not an estimate for production.
AI-processed from X: @AIQuanting; edited by Hamidun News
Imagine you need to run a language model cheaply and at large scale. The usual choice is simple: a graphics card or a regular CPU. This project took a third path: an FPGA, a chip that can be reprogrammed for one specific task and rented in the cloud from AWS (F2 instances). Coding agents wrote the engine for it, and speed grew from 6.43 to 213.13 tokens per second. A token is roughly a word or part of a word, so this is about text generation speed. The most interesting part here is not "33 times," but that the result was checked for accuracy and recalculated in money.
What was built and what "bit-exact" means
The team ran the DeepGrove model with ternary weights. In such a network, each weight takes only three values: −1, 0, or +1. The arithmetic becomes very simple, and it is convenient to build highly specialized hardware for it. According to the project title, agents wrote about 3 million lines of code for the implementation and optimization.
The growth from 6.43 to 213.13 tokens per second gives those same 33 times. This is a speedup of the engine itself, not a 33-fold lead over the CPU: relative to the CPU, the difference is much more modest (more on that below).
Accuracy was checked separately. "Bit-exact" means the answer matches the reference not "approximately" but down to the last bit. In the final test there were eight dialogues of three turns each, and all 24 responses matched the fixed CPU reference. This matters in practice: when you optimize for non-standard hardware, answers easily begin to drift slightly, and in a language model even a small divergence quickly turns into a completely different text.
Comparison with the CPU in money
An hour on the FPGA instance costs $1.98. The exact 32-thread CPU version costs $1.95, so almost the same. But the CPU delivers 129.5 tokens per second, while the FPGA delivers 213.13, about 1.65 times more.
The author recalculated price and speed per million generated tokens: about $2.58 on the FPGA versus $4.18 on the CPU. The savings are roughly 38%. This is not a separate benchmark but arithmetic: the hourly price is divided by the number of tokens that can be generated in an hour.
On long context the picture is more modest. When the context is about 8,000 tokens, FPGA speed falls to roughly 76 tokens per second, and the run took 77.8 seconds. If you work with long documents or a large dialogue history, the advantage may shrink noticeably. The post contains no data on how the CPU behaves in this mode, so it is too early to draw a conclusion.
Limitations and what to do with them
The author himself calls the result useful for this model and this implementation, but not a calculation of total cost of ownership. Development time, maintenance, and the risk of rewriting the code for a different model are not included in the price. During setup there were hardware failures and freezes. There is no independent replication yet, so the figures should be treated as claimed, not confirmed.
What you can take into practice:
- If you run small quantized models at large volumes, rented FPGAs stop being exotic. But the 38% savings should be verified on your own workload and with your own context length.
- Checking against a frozen reference is useful on any hardware. If an agent optimizes inference, have every step verified for bit-exact matching, not for "looks about right."
- Count money per million tokens, not per hour of rental. The nearly identical hourly price ($1.98 vs. $1.95) hides the speed difference that produces the final savings.
Judging by this project, agents can already take on tasks that previously required a team of hardware acceleration engineers. But millions of lines of code and hardware freezes are a reminder that verifying the result remains the human's job. Going forward, watch two things: whether independent replication appears, and whether the result holds on other models.
- Nov 21, 2026AG-UI 1.0 gives AI agents a common language for talking to apps: 31 event types
- Nov 21, 2026OpenAI launches ChatGPT Wallet: agents buy without ever seeing your card number
- Nov 21, 2026US Advertisers Spent $70.7 Million on ChatGPT Ads in October, Up 127% From April
- Nov 21, 2026ONNX Runtime vanishes from GitHub: Microsoft hides repositories after package hack
Key Questions Answered
What was built and what "bit-exact" means?
The team ran the DeepGrove model with ternary weights. In such a network, each weight takes only three values: −1, 0, or +1. The arithmetic becomes very simple, and it is convenient to build highly specialized hardware for it. According to the project title, agents wrote about 3 million lines of code for the implementation and optimization. The growth from 6.43 to 213.13 tokens per second gives those same 33 times.
What are the key details on Comparison with the CPU in money?
An hour on the FPGA instance costs $1.98. The exact 32-thread CPU version costs $1.95, so almost the same. But the CPU delivers 129.5 tokens per second, while the FPGA delivers 213.13, about 1.65 times more. The author recalculated price and speed per million generated tokens: about $2.58 on the FPGA versus $4.18 on the CPU. The savings are roughly 38%. This is not a separate benchmark but arithmetic: the hourly price is divided by the number of tokens that can be generated in an hour.
What are the key details on Limitations and what to do with them?
The author himself calls the result useful for this model and this implementation, but not a calculation of total cost of ownership. Development time, maintenance, and the risk of rewriting the code for a different model are not included in the price. During setup there were hardware failures and freezes. There is no independent replication yet, so the figures should be treated as claimed, not confirmed.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.