Inkling 2 Small: 68 Hours of Pretraining Is Only the First Stage, Followed by 9 Days of RL on 2,048 GPUs

When a new model's announcement says "8.4 trillion tokens in 68 hours," the urge is to immediately recalculate the budget for training your own model. Don't rush. The figure is honest, but it refers to one piece of a large pipeline. Let's break down what exactly Thinking Machines Lab did with Inkling 2 Small and how to read numbers like these.

What the model is and why 68 hours is both a lot and a little

Inkling 2 Small is a multimodal language model that the company introduced on November 5, 2026. It has 594 billion parameters in total, but it uses a Mixture-of-Experts (MoE) architecture: for each token, only a portion of the "experts" is activated, around 17 billion parameters, or roughly 3% of the model. Per-token compute is comparable to a mid-sized model, while knowledge capacity is that of a giant.

Pretraining on 8.4 trillion tokens took 68 hours, under three days. That works out to about 34 million tokens per second on average. This pace was made possible by two things: NVIDIA GB300 NVL72 racks and the four-bit NVFP4 format. Fewer bits per number means less memory and less traffic between chips, and faster matrix multiplications. The flip side is precision, which is why such formats require careful tuning of the training process.

What lies behind the headline

The main caveat from the announcement itself: 68 hours is only the first stage. It was followed by three more training stages on additional data. What exactly was in that data and how long those stages took is not disclosed in the available portion of the material, so you shouldn't conclude that the whole model was "assembled in three days."

Next came Reinforcement Learning: nine days on 2,048 GPUs and more than 17 million trajectories. A trajectory is one complete run of the model: it receives a task, acts, and then its result is evaluated. Dividing 17 million by nine days gives at least 22 trajectories per second, continuously. At this stage the model learns not just to continue text but to carry tasks through to a verifiable result: code, reasoning, tool use.

This is a typical picture of modern training. The cheap, fast pretraining phase provides the foundation, while the real cost and quality are built up in later stages, where data is more expensive and the signal is more precise. The beginning of the Habr article ends on the phrase "The resulting model scored...", so final benchmark results are not given in the available excerpt. Check them against the primary source.

What to take away from this

First: compare models by the full cycle, not by an eye-catching number. Pretraining time without the number of GPUs, the stages, and the cost of RL says little about actual expenses.

Second: MoE with a small share of active parameters is a working way to get a large model without a proportional rise in compute cost. But at inference, all 594 billion parameters have to be held in memory somewhere, so the savings concern FLOPs more than hardware footprint.

Third: four-bit training, judging by this release, is moving from labs into production pipelines. If you are planning your own pre- or post-training stack on Blackwell-class hardware, NVFP4 is worth testing at the very least.

And finally: RL on millions of trajectories requires infrastructure for generating and verifying them, not just GPUs. That is where teams trying to replicate the result most often get stuck.