Thinking Machines Lab Releases Inkling-Small: Open 276B MoE Model Outperforms Its Teacher

On August 2, 2026, Thinking Machines Lab open-sourced the weights of Inkling-Small, a multimodal Mixture-of-Experts model with 276 billion total parameters and 12 billion active. The model is four times smaller than the original Inkling (975B total, 41B active), yet it outperforms it on most key coding and reasoning benchmarks.

How does Inkling-Small differ from Inkling?

Inkling-Small is a 42-layer decoder-only transformer that uses a sparse MoE layer in place of standard FFNs. Each token is routed to 6 of 256 experts, plus 2 shared experts that are always active. Attention is a hybrid of local and global layers.

The model is natively multimodal and does not need a separate encoder:

• 276B total parameters, 12B active at inference
• A 1M-token context window with an adjustable reasoning "effort" level
• Input: text, images (40×40-pixel patches through a four-layer hMLP) and audio (dMel spectrograms, 16 kHz WAV, up to 2 minutes)
• Numeric formats: BF16, MXFP8 and NVFP4
• Apache 2.0 license, with weights published on Hugging Face

How to run Inkling-Small

The BF16 checkpoint requires at least 600 GB of total VRAM, which means 4× NVIDIA B300 or 8× NVIDIA H200. The quantized NVFP4 version lowers the threshold to 180 GB: the W4A4 mode runs on a single B300 card (SM100+ architecture required), while W4A16 runs on two H200s.

Supported runtimes are SGLang, vLLM, TokenSpeed, Unsloth and Hugging Face Transformers. The single-GPU path takes a 276B model out of the "large labs only" category: startups can rent a single B300 instance, and companies with existing H200 clusters can avoid new hardware purchases. Regulated industries such as finance, healthcare, insurance and telecom get an option with private weights and no dependence on an external API.

Which benchmarks did Inkling-Small beat Inkling on?

Inkling-Small beat its "teacher" on most key tests. On Humanity's Last Exam (text-only version) it scores 31.6% versus 29.7% for Inkling. On SWE-bench Verified it scores 80.2% versus 77.6% (bash-only harness). Toolathlon Verified comes in at 54.4% versus 45.5%. GPQA Diamond is 89.5%, AIME 2026 is 95.5%, and ARC-AGI-2 is 40.1% versus 36.5%. Terminal-Bench 2.1 is 64.7%.

There are two clear regressions: SimpleQA Verified drops to 20.6% from 43.9% for Inkling, and Tau 3 Banking falls to 15.5% from 23.7%. Multimodal results stay close to the teacher's: MMMU Pro is 74.0%, VoiceBench is 90.1% and MMAU is 77.0%, according to Artificial Analysis, Scale AI and ARC Prize.

"The model does not create significant uplift compared with the existing open-weights ecosystem,"

Thinking Machines Lab says in the Inkling-Small model card, recommending downstream moderation, such as Llama Guard, in consumer applications.

What this means

Inkling-Small shows that targeted distillation from the Inkling teacher, plus two weeks of RL training on agentic coding, can let a compact model outperform its larger predecessor. Open weights under Apache 2.0 make a 276B agent available for self-hosted deployment, and widen the circle of companies that can build products without depending on closed APIs.

Frequently asked questions

How much VRAM does Inkling-Small need?

At least 180 GB of VRAM in the quantized NVFP4 mode on a single B300 card (SM100+). Full-precision BF16 requires 600 GB in total, for example 4× NVIDIA B300 or 8× NVIDIA H200.

How is Inkling-Small better than the original Inkling?

On six key benchmarks: SWE-bench Verified (80.2% vs 77.6%), GPQA Diamond (89.5%), AIME 2026 (95.5%), ARC-AGI-2 (40.1% vs 36.5%), Toolathlon Verified (54.4% vs 45.5%) and HLE (31.6% vs 29.7%). It falls behind on SimpleQA Verified (20.6% vs 43.9%) and Tau 3 Banking (15.5% vs 23.7%).