Cerebras splits a 3-trillion-parameter model across 59 chips: 1,000–2,000+ tokens per second
Even with 4-bit compression, a 3-trillion-parameter model weighs about 1.5 TB, while a single Cerebras WSE-3T wafer-chip has just 44 GB of fast memory. So Cerebras cut the open-weight Kimi K3 into 93 layers of roughly 16 GB each and spread them across 59 wafers, like machines on an assembly line: the weights stay put, and only intermediate results travel between chips. Cerebras claims 1,000–2,000+ tokens per second for models of this class. When choosing a provider, compare tokens per second per megawatt.
AI-processed from Hacker News AI; edited by Hamidun News
A 3-trillion-parameter model does not fit in the memory of a single chip: even with 4-bit compression, its weights take up about 1.5 TB, while one Cerebras WSE-3T wafer has 44 GB of built-in memory. Cerebras showed, using the open-weight Kimi K3 as an example, how such a model is cut into 93 layers and spread across 59 wafers (35 + 24) to deliver 1,000–2,000+ tokens per second. For developers, this answers why cost and serving speed depend on the model's architecture, not just on its parameter count.
Why one wafer is not enough
The WSE-3T wafer is a chip the size of an entire silicon wafer. Its SRAM is very fast but small: 44 GB. Frontier-class model weights run to terabytes. In Cerebras's example, the model uses 4-bit quantization, as in the Hugging Face version: 3 trillion parameters at 4 bits each come to about 1.5 TB for the weights alone.
Kimi K3 has 93 layers: 69 with linear attention and 24 dense, meaning full attention. Dividing 1.5 TB by 93 layers gives about 16 GB per layer. That is a convenient number: a whole layer fits in a 44 GB wafer, so it does not have to be split across chips.
According to Cerebras itself, Kimi K3 scores 88.3% on Terminal-Bench 2.1 (GPT-5.6 Sol scores 88.8%) and 91.2% on BrowseComp (GPT-5.6 Sol scores 90.4%). These are figures from the vendor's post; the material includes no independent verification.
How layers are laid out across chips
The space a layer needs on a wafer depends on its type. A dense layer stores keys and values for every token of every active request. This is the KV cache, essentially a notebook of everything said so far in the conversation, and it grows with context length. That is why each wafer hosting a dense layer holds only that one layer, with the rest of its memory reserved for the cache.
A linear layer needs a small state of fixed size per request that does not grow with context. Two such layers fit on one wafer (2 × 16 GB = 32 GB).
This works out to 35 wafers for the 69 linear layers and 24 for the dense layers, 59 in total. On the dense-layer wafers, 28 GB remains for the cache: 44 GB minus 16 GB for the layer.
Next comes the pipeline. The Cerebras compiler assigns layers to regions of wafers, and the runtime routes tokens through them. The weights stay where they are; only activations, the intermediate results of the computation, move between wafers. They travel over RDMA on top of Ethernet, at roughly 2 microseconds per hop. In factory terms, the machines stay in place while the workpieces move down the shop floor.
How the pipeline serves many requests
The obvious objection: while a token passes through all the wafers, the others sit idle, so the system is effectively running a batch of one request. Cerebras says that is not the case. The pipeline is filled with different requests: while the third stage processes a token from request A, the second is busy with request B, and the first with request C. Each stage can take not a single request but a small group, a micro-batch. The number of requests in flight is roughly the number of stages multiplied by the micro-batch size.
There is a trade-off here. Small micro-batches speed up each individual request, while large ones raise overall throughput. In production, according to the company, the stages are much smaller than a whole layer, and there are more of them.
The ceiling on parallel requests is set by KV-cache memory: each request in flight holds its own cache on every wafer with a dense layer. To estimate the maximum, the free 28 GB is divided by the cache size of one request; the example uses an average request of 256 thousand tokens. Unlike batching on GPUs, requests here do not wait for one another but flow continuously through the pipeline.
What this means in practice
First, a model's architecture affects the cost of serving it. Linear attention needs almost no cache memory, so those layers pack twice as densely. If you are choosing an open model for long context, the share of dense layers matters at least as much as the parameter count.
Second, Cerebras proposes comparing inference systems not by batch size but by total tokens per second per megawatt, at the same generation speed for a single response. This metric is worth requesting from any provider, including GPU clouds.
Third, the approach scales: you calculate the memory needed for weights and request state, lay the model out across wafers, and connect them into a pipeline. Each new wafer adds compute and memory.
There are limitations too. This is an illustrative calculation by the vendor, not an independent benchmark. The stated 1,000–2,000+ tokens per second refers to models with several trillion parameters in general, not to a separate measurement of Kimi K3. The material includes no prices, only links to a free API and the sales department. To verify the claims, use the free API and run your own long-context requests.
- Nov 21, 2026Eight Open-Source Tools for AI Agents: Browser Use Leads with 121,000 Stars
- Nov 21, 2026Thinking Machines Releases Open-Weight Inkling 2 Mini with 26.4B Parameters for 32 GB Macs
- Nov 21, 2026AURA 2.0 Video Model Starts Talking on Its Own: Response Audio Arrives in 344 ms
- Nov 21, 2026Anthropic raised a record $81.3B in its IPO, but the founders kept control
Key Questions Answered
Why one wafer is not enough?
The WSE-3T wafer is a chip the size of an entire silicon wafer. Its SRAM is very fast but small: 44 GB. Frontier-class model weights run to terabytes. In Cerebras's example, the model uses 4-bit quantization, as in the Hugging Face version: 3 trillion parameters at 4 bits each come to about 1.5 TB for the weights alone. Kimi K3 has 93 layers: 69 with linear attention and 24 dense, meaning full attention. Dividing 1.5 TB by 93 layers gives about 16 GB per layer.
How layers are laid out across chips?
The space a layer needs on a wafer depends on its type. A dense layer stores keys and values for every token of every active request. This is the KV cache, essentially a notebook of everything said so far in the conversation, and it grows with context length. That is why each wafer hosting a dense layer holds only that one layer, with the rest of its memory reserved for the cache. A linear layer needs a small state of fixed size per request that does not grow with context.
How the pipeline serves many requests?
The obvious objection: while a token passes through all the wafers, the others sit idle, so the system is effectively running a batch of one request. Cerebras says that is not the case. The pipeline is filled with different requests: while the third stage processes a token from request A, the second is busy with request B, and the first with request C. Each stage can take not a single request but a small group, a micro-batch.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.