Colibri, written in pure C, runs 700B+ parameter MoE models without a GPU

Talk about running huge open-weight models usually ends the same way: "you need a GPU rack." The Colibri repository offers a different path. According to a post on the AI Radar aggregator, the project has gathered about 47.4 thousand stars, 802 of them in a single day. That is a strong signal of interest, even though the post itself was viewed only 29 times and contains almost no details. Let's go through what is known and what is worth checking yourself.

What is claimed

Colibri is described as a pure C engine for running large MoE models on ordinary computers. The key feature: experts are loaded from disk as needed. The listed supported families are GLM, Kimi, DeepSeek and Qwen. The main claim of the post's author: models of 700B and larger can be run without a graphics card.

Pure C is also an argument in itself. It means minimal dependencies, a simple build and clear control over memory. For infrastructure code, where it matters how many bytes are read from disk and when, this is a reasonable choice.

How it works, with a simple example

MoE (Mixture of Experts) works like this: each layer of the model has dozens or hundreds of "experts," that is, separate blocks of weights. For each token, a special router picks only a few of them. The rest sit idle at that moment.

Imagine a library of thousands of books. To answer a specific question, you need pages from eight books. The classic approach drags the entire library into the work room, that is, into video memory or RAM. For a 700B model that means hundreds of gigabytes. Colibri's approach, as described, is different: only what is needed lies on the desk, while the rest stays on the shelf, that is, on the SSD. When the router asks for new experts, the engine reads them from disk.

Simple arithmetic shows the scale. 700 billion parameters in 4-bit format take roughly 350 GB. That will not fit in the RAM of an ordinary PC, but it will fit on a good NVMe drive. Only a fraction of these weights is actually read per token, and the whole idea rests on that.

Limitations the post does not mention

First, speed. An NVMe drive reads orders of magnitude slower than GPU memory. If every token requires fresh experts, generation will be bottlenecked by the disk. The post gives no tokens-per-second figure at all, and that is the key metric. It may be tolerable for batch processing or experiments, but an interactive chat or an agent loop with dozens of calls could prove painful.

Second, storage. You need hundreds of gigabytes of free space and a fast drive. On an old SATA drive or HDD the picture will be noticeably worse.

Third, quality and compatibility. It is unknown which quantization formats are supported and how closely the output matches reference implementations. A claim of "works with DeepSeek, Kimi, GLM, Qwen" does not yet mean that every version of every model works.

Finally, the source itself. The post looks like an automated digest of a trending repository. It is best to verify the star count and the list of models directly on GitHub.

What to do about it in practice

If you are a developer or founder evaluating self-hosted LLMs, Colibri deserves a place on your shortlist for a quick test. The steps are simple:

• Read the README and the list of supported models and formats.
• Take one mid-sized MoE model and measure tokens per second on your own SSD.
• Compare the result with renting a GPU or using a regular API. For rare tasks, private data and offline scenarios, a disk instead of a graphics card may turn out cheaper.
• Do not put the project into production until you see stable benchmarks and an active commit history.

The big picture is this: MoE architecture makes giant models less demanding in compute per token, and tools like Colibri try to make them less demanding in hardware. If the claims are confirmed, the barrier to entry for experimenting with 700B-class models will drop noticeably. But for now this is a promise, not a measured result.