Strata developer builds a live map of 24,576 MoE experts on a GTX 1070

Running a huge model on an old graphics card no longer surprises anyone. It is much rarer for someone to show what happens inside during such a run. In the third installment of his series on Qwen3.8-Flash-Next (125 billion parameters) and the GTX 1070, the author looks inside his own engine, Strata. He built a live visualization of how the experts work and tried to answer whether they really specialize by topic.

An important caveat: the available excerpt contains no final numbers and no conclusion. We know how the problem is framed, what the architecture looks like, and that the author found a methodological error in his own work. The excerpt contains no experimental results, and we will not invent any.

How 125 billion parameters fit into 8 GB

Strata is an inference engine for MoE (Mixture-of-Experts) models. In an ordinary dense network, all weights work on every token. In an MoE, each layer has a set of "experts," and only a small subset, the so-called top-K, is activated for each token.

Qwen3.8-Flash-Next has 48 layers of 512 experts each, which makes 48 × 512 = 24 576 separate nodes. At each step, only a small fraction of them actually compute. But all of them have to be stored, and that takes hundreds of gigabytes. A set that size will not fit into the 8 GB of VRAM on a GTX 1070.

Strata solves this with an adaptive cache:

• the hottest experts stay in VRAM permanently;
• the rest are read from RAM or from SSD as needed;
• the cache decides which experts to keep and which to evict using a decayed access counter, meaning a counter in which older accesses gradually lose weight.

Put simply, it is an LRU/LFU cache for chunks of a neural network. How well this scheme works depends entirely on how predictably the router picks experts. If accesses concentrate on a small core, the hit rate is high and the model runs tolerably. If they spread out evenly, everything is limited by SSD speed.

Why a "live map" was needed

Until now, the author saw the mechanics only through aggregates in monitoring: how many experts are currently in VRAM and what the hit rate is. That is a typical situation in inference engineering: the dashboard tells you whether things are fine or not, but does not explain why.

A real-time visualization of 24 576 nodes closes exactly that gap. It lets you watch which experts light up on which tokens, how the hot core forms, and how the cache evicts and loads data. For an inference stack developer, such a map works like a profiler: it shows access patterns and helps decide what to change in the eviction policy.

Expert specialization and a methodological trap

The author did not stop there and asked a popular question: do experts specialize by topic, say with separate ones for code, math, and conversational speech? The idea is appealing. If specialization exists, the cache could be warmed up in advance for the type of task.

Partway through, the author caught himself making a methodological error that "almost led to a beautiful but wrong conclusion." What exactly the error was is not visible in the excerpt. But the typical traps in such experiments are well known (this is our own general commentary, not a retelling of the article):

• experts may respond to formal features of a token, such as language, case, frequency, or position in the text, rather than to topic;
• without a control comparison against a baseline, any activation map looks "meaningful";
• a small sample of prompts produces vivid but random clusters.

What to take away

First: if you run an MoE on limited hardware, look beyond the average hit rate and examine the distribution of accesses across experts and layers. That distribution shows whether the cache works on your workload.

Second: the hypothesis that "experts specialize by topic" has to be tested with control groups and different types of text. A beautiful visualization is not proof.

Third: it is worth building observability tools for inference early. A black box with aggregates hides both problems and optimization opportunities.

The limitations are obvious. This is the project of one hobbyist on one old graphics card, and, judging by how the question is framed, the conclusions about specialization call for caution. But as an example of honest engineering work, in which the author publicly describes his own mistake, the piece is worth reading in full.