IBM details how it built Granite 4.1: 15 trillion tokens, 512K context and a bet on quality

On April 29, 2026, IBM published a detailed breakdown on the Hugging Face blog of how the Granite 4.1 family was built. The company disclosed not only the model architecture but the whole pipeline: from dataset composition and long context to SFT data filtering and multi-stage reinforcement learning.

How the series is built

Granite 4.1 is a family of decoder-only dense models with 3B, 8B and 30B parameters. All three variants share the same basic design: Grouped Query Attention, Rotary Position Embeddings, SwiGLU, RMSNorm and shared input and output embeddings. The differences are mostly in scale: number of layers, hidden-state size and MLP parameters. This lets you compare models within one lineup without adjusting for different architectural philosophies.

IBM's central thesis is that a small model's quality depends not only on compute budget but also on discipline in handling data. That is why Granite 4.1 was built as dense rather than MoE models, with the bet placed on carefully shifting data mixtures during training. All models are released under the Apache 2.0 license, and the instruct versions support 12 languages, including English, German, Spanish, Japanese, Arabic, Chinese and Portuguese.

Five training phases

Granite 4.1 was pretrained from scratch on roughly 15 trillion tokens. IBM split the process into five phases: first the model builds a broad language base on web data, then strengthens math and code, and then moves gradually to higher-quality, more specialized samples. In the later phases the mix adds long reasoning traces, synthetic data and instruction datasets, and the final phase separately trains the model to handle very long context.

• Phase 1: 10 trillion tokens of general pretraining, with about 59% of the mix coming from CommonCrawl.
• Phase 2: another 2 trillion tokens with a sharp increase in the share of math and code, to 35% and 30% respectively.
• Phase 3: 2 trillion tokens of high-quality annealing, introducing chain-of-thought, synthetic and instruction data.
• Phase 4: another 0.5 trillion tokens focused on the highest-quality mix, with the learning rate decaying to zero.
• Phase 5: long-context extension, which widens the window from 4K to 32K, 128K and then 512K.

So that long context does not degrade quality on short prompts, IBM merges the model after each LCE stage. For the final extension to 512K in the 8B and 30B versions, it used a mix of books and code repositories. On the base models this produced notable RULER results: the 8B variant keeps the metric high even at 128K, and the 30B goes higher still. This is an important signal for teams that need more than chat answers, such as work with long documents, logs and large chunks of code.

Post-training and quality

After pretraining, IBM ran the SFT dataset through a strict quality-control pipeline. The final selection contains about 4.1 million examples, but before that every answer was checked with an LLM-as-Judge scheme and a set of deterministic rules. The judge model assessed instruction following, correctness, completeness, brevity, naturalness and calibration, while hard grounds for rejection included hallucinations, false premises and calculation errors. Normalization, schema validation, length filters and global deduplication were also applied.

In the RL stage IBM did not stop at a single pass. It used on-policy GRPO with DAPO loss and assembled four sequential stages: multi-domain RL, RLHF for general helpfulness and dialogue, identity and knowledge-calibration RL, and then a separate math RL that restores and improves math skills after RLHF. According to IBM, RLHF alone added an average of about 18.9 points on AlpacaEval relative to SFT checkpoints.

The most striking result: the Granite 4.1 8B instruct model consistently matches Granite 4.0-H-Small 32B-A9B and beats it on a number of tests. IBM also released FP8 variants, which roughly halve memory and disk requirements.

What this means

IBM has shown that open-source LLMs can compete not only on model size but on the quality of the training recipe. For companies, this makes Granite 4.1 a practical candidate: predictable latency without long reasoning traces, long context, an open license and a lower cost of running compared with heavier systems.