Helion from PyTorch Picks GPU Kernel Settings at Compile Time Instead of 25 Hours of Autotuning
A GPU kernel only runs fast if you pick the right block sizes, thread counts, and pipeline depth. In Helion, PyTorch's language for these kernels, an autotuner handles this by trying variant after variant. One paged-attention kernel tuned across 72 data shapes took 10 hours in Helion's fast mode and 25 hours in full mode. Now the compiler identifies the kernel type and sets the parameters right away, with no search. For prototypes and new shapes, the long search can therefore be skipped.
AI-processed from PyTorch Blog; edited by Hamidun News
Writing a GPU kernel is not enough: to make it run fast, you have to choose dozens of settings, including block sizes, thread counts (warps), and load pipeline depth. In Helion, PyTorch's language for portable machine learning kernels, an autotuner handles this. It compiles and benchmarks many variants, then picks the best one. It works reliably but slowly. One team tuning a single paged-attention kernel across 72 data shapes spent 10 hours in Helion's fast mode and 25 hours in full mode. As the number of kernels, shapes, and GPUs grows, the time climbs fast. So Helion added compile-time heuristics that pick the settings immediately, without a search.
How the compiler guesses the settings
One Helion kernel can turn into thousands of different Triton kernels, and the autotuner has to pick one of them. The heuristic works differently. It reads the kernel's intermediate representation (IR) and records structural facts: which axes are split into blocks, which sizes can change, and which tiles are live in each section. It then produces a ready-made configuration from those facts.
First, the kernel is assigned to one of three families. Any kernel with matrix multiplication, for example one using hl.dot or torch.baddbmm, follows the matmul branch, even if it contains other operations. A kernel with a reduction (collapsing a row into fewer values) and no matmul follows the reduction branch. Everything else counts as pointwise. If not all sizes are known at compile time, the heuristic does not run, and Helion falls back to its previous default settings.
All branches rely on one analysis: how many registers, shared memory, and, on Blackwell, tensor memory a single thread block (CTA) needs at the chosen sizes. From that, it determines how many blocks fit on a multiprocessor. In the article's example, a block needs one tenth of the registers and shared memory but six tenths of the tensor memory. Two such blocks would not fit, because 0.6 × 2 = 1.2, meaning the budget is exceeded by 20%. So only one block fits per multiprocessor.
These estimates then drive the settings. More pipeline stages hide load latency better, but six stages can consume all the shared memory and leave one block per multiprocessor. Three stages with two blocks are often the better trade-off. If there are fewer blocks than multiprocessors, the block size is reduced. If a reduction kernel reads a row twice, as RMSNorm does (first computing the scale, then applying it), it is better to keep the row in memory and avoid re-reading it.
What the measurements showed
The authors stress that the goal is not to beat an autotuner that tries thousands of variants. They want to add an intermediate point between runtime speed and waiting time.
Tests ran on H100. Nine linear attention variants across 96 kernel-and-shape combinations were compared with the handwritten Triton kernels from the FLA library. The computed configuration beat them and was faster than Helion's previous settings without autotuning. On six production kernels for vLLM (two FP8 quantizers, RMSNorm, SiLU-and-multiply with quantization, and QK normalization with RoPE), the test covered 153 kernel-and-shape combinations. Here, the computed configuration outperformed vLLM's handwritten CUDA/C++ operators. Full autotuning on these kernels adds only a little on top of the free configuration.
Separately, the team compared everyday kernels with torch.compile in max-autotune mode: ten reductions across 80 combinations and thirteen families of matmul kernels across 70 shapes. The cost of the analysis itself is negligible: the difference in compile time with and without the heuristics is tens of milliseconds, lost in the noise of Triton compilation.
Hints for the autotuner and limitations
The same heuristics can also output 5–20 related configurations as starting points for the autotuner. The main configuration has to be good on its own, while the fallback configurations let the search begin with promising candidates.
There are limitations too. The heuristic predicts Triton's decisions: what will end up in registers, what will go through shared memory, and what will not be materialized at all. The estimate is approximate. When one kernel contains several matmuls, their settings can conflict; for example, they may all share one num_warps value. In that case, the matmul with an accumulator in a loop wins, followed by the one with the larger amount of work. The measurements were taken on H100 and apply to the specific kernel sets tested.
The practical takeaway for anyone writing kernels in Helion: for prototypes, new shapes, and rare kernels, long autotuning can be postponed, because out-of-the-box settings are now closer to tuned ones. If you ship a fixed set of shapes, it is still worth running full autotuning once before release.
- Nov 21, 2026Nvidia Looks for a New CEO as Board Considers Arm Chief Rene Haas
- Nov 21, 2026Уход Хуанга: Nvidia подтвердила Корее 260 000 GPU, в Китае под вопросом проекты на $3 млрд
- Nov 21, 2026Applied Digital Asks Oracle and Meta for Billions in Power Guarantees
- Nov 21, 2026Soros Fund Sold 19.3% of Its Nvidia Stake Before Huang Stepped Down as CEO
Key Questions Answered
How the compiler guesses the settings?
One Helion kernel can turn into thousands of different Triton kernels, and the autotuner has to pick one of them. The heuristic works differently. It reads the kernel's intermediate representation (IR) and records structural facts: which axes are split into blocks, which sizes can change, and which tiles are live in each section. It then produces a ready-made configuration from those facts. First, the kernel is assigned to one of three families.
What the measurements showed?
The authors stress that the goal is not to beat an autotuner that tries thousands of variants. They want to add an intermediate point between runtime speed and waiting time. Tests ran on H100. Nine linear attention variants across 96 kernel-and-shape combinations were compared with the handwritten Triton kernels from the FLA library. The computed configuration beat them and was faster than Helion's previous settings without autotuning.
What are the key details on Hints for the autotuner and limitations?
The same heuristics can also output 5–20 related configurations as starting points for the autotuner. The main configuration has to be good on its own, while the fallback configurations let the search begin with promising candidates. There are limitations too. The heuristic predicts Triton's decisions: what will end up in registers, what will go through shared memory, and what will not be materialized at all. The estimate is approximate.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.