SmolLM2-1.7B обгоняет модели на 34B в генерации MLIR-кода без дообучения

In July 2026, researchers presented a method on arXiv that enabled the SmolLM2-1.7B language model to generate correct MLIR code on par with models 15-34 times larger — without fine-tuning, through decoding constraints mechanically derived from dialect specifications.

Why MLIR is hard for models

MLIR (Multi-Level Intermediate Representation) is the intermediate representation underpinning the infrastructure of modern ML compilers: TensorFlow, JAX/StableHLO, PyTorch Inductor, and IREE. However, MLIR appears only in trace amounts in language models' training corpora, so models generate it poorly.

The main difficulty is extensibility. MLIR is designed to be extended with new "dialects" for each application domain, and fine-tuning a separate model for every dialect doesn't scale. The authors posed the question: can inference-time priors, mechanically derived from a dialect's operation specification (Operation Definition Specification, ODS), replace adaptation via gradient-based fine-tuning.

How the constraints from the specification work

The method builds a three-layer constraint stack directly from the dialect schema, without training. Each layer filters out invalid variants already at the generation stage.

• C1 — a context-free grammar (CFG) over operation signatures
• C2 — type-based partitioning from a type lattice extracted from the ODS
• C3 — an SSA scope validator with rejection sampling and five retries

According to the annotation on arXiv, porting the stack from the arith+func+memref+linalg dialect set to StableHLO required not a single line of new constraint code — the schema is derived from the specification automatically.

How much a small model beat large ones

On dialects where the verifier's semantics are defined by structural constraints, schema priors let SmolLM2-1.7B match or outperform models with 15-34B parameters. On the linalg dialect, the model reached 80.0% valid programs (average across three seeds, n=125).

• Advantage over CodeLlama-34B, Granite-Code-34B, and StarCoder2-15B — 21 to 44 percentage points, with non-overlapping confidence intervals
• Generation speed — 8-25 times higher than the large base models
• 4 NL-to-MLIR benchmarks released: MLIR-Spec-150, Linalg-Spec-30, StableHLO-Spec-30, and StableHLO-Held-Out-200 — 410 "text → MLIR" pairs in total
• Datasets licensed under Apache-2.0, with datasheets following the Gebru methodology and Croissant 1.0 metadata
• The benchmarks, decoder, all generations, and a Docker image for reproduction have been released publicly

But the method doesn't win everywhere. On arith+func and on the parametric StableHLO-Held-Out-200 set, where correctness depends on attribute values rather than code structure, large models matched or beat the small one — the authors honestly flag these cases as "non-win cells."

"Schema priors allow

SmolLM2-1.7B to match or outperform 15-34B code models at 8-25x generation speed," the study's annotation on arXiv states.

What this means

For narrow, structurally strict languages like MLIR, constraints derived from the specification can replace expensive fine-tuning: a small model with the right "guardrails" outperforms a large one and runs many times faster. This lowers the cost of generating compiler code and removes the scaling problem for every new dialect.

Frequently asked questions

What is MLIR?

MLIR (Multi-Level Intermediate Representation) is an intermediate code representation that powers the ML compilers TensorFlow, JAX/StableHLO, PyTorch Inductor, and IREE. It is extended with "dialects" for specific tasks.

Why did a 1.7B model beat a 34B model?

It wasn't fine-tuned — instead it received decoding constraints derived from the dialect specification. On linalg this produced 80.0% valid programs — 21-44 points more than CodeLlama-34B, Granite-Code-34B, and StarCoder2-15B.

Where did the method fail?

On the arith+func dialects and the StableHLO-Held-Out-200 set, where correctness depends on attribute values rather than code structure. There, large models matched or beat SmolLM2-1.7B.