I Ran Qwen on a $50 FPGA: How a Homemade AI Accelerator Works

I wanted to build my own AI processor and found openTPU — an open accelerator from Felipe Sens Bonetto, developed with help from AI agents. The question was simple: can you actually take a project from GitHub, flash it onto a physical FPGA, and run real large language model inference?

I bought a Xilinx Kintex-7 XC7K480T with 4GB of RAM on AliExpress. I connected it via PCIe to a MINISFORUM mini PC, built the firmware in Vivado, configured the drivers and PCIe DMA, prepped the software — and loaded real neural network weights. Inference started running.

What I got working

All systems passed testing. PCIe DMA transfers data successfully. The memory subsystem works fully. Processor instruction tests pass 100%. The FPGA is correctly detected by the system. The language model runs inference on the FPGA.

Based on measurements on the same board, these models run at the following speeds:

• Qwen3-0.6B — 31 tokens per second • Qwen3.5-2B — 12 tokens per second • LFM2.5-230M — 86 tokens per second • Qwen3.5-35B — 4 tokens per second (35 billion parameters on 4GB of memory)

That last one is especially interesting: a large model running on a card with minimal memory. The FPGA itself isn't exactly new technology.

Why this is more than just another neural network demo

We usually think of GPUs as a ready-made platform: buy a graphics card, install CUDA, run the model. But openTPU's idea is the opposite — don't pick hardware to match the neural network, build your own specialized hardware for specific computations instead.

That's exactly what TPUs are for — processors optimized for machine learning operations. That used to be a job only Google and NVIDIA could do, requiring years of development. Now one engineer can put together and test a prototype over a weekend. The current board isn't an H100 competitor. A working prototype isn't a production ASIC. But now you can experiment with your own AI processor architecture on real hardware, not just in theory.

What's next

I'm thinking about faster memory and a more powerful FPGA to evaluate the architecture of my own inference chip. There's an interesting trend emerging here: we already use AI to write software, and now AI is designing the processors it will itself run on.

Thanks to Felipe Sens Bonetto for openTPU. Stay tuned for part 2, where I'll show the results from my test bench — 8 neural networks, 165 runs, real tokens-per-second numbers, and an analysis of where DDR3 becomes the bottleneck.