IBM Spyre Moves torch.compile From the Homegrown sendnn to Open-Source Inductor

IBM Spyre has 32 accelerator cores and up to 128 GB of LPDDR5 memory on a single PCIe card, and it now has a different compiler in PyTorch. The Spyre software stack has moved from sendnn, a separate compilation system written from scratch, to backend=inductor. This is the open-source torch.compile backend that PyTorch uses by default. The PyTorch Blog reported the change. The publicly available part of the announcement states only that the backend has changed, with no benchmarks, so what can be assessed is the architectural logic of the move.

What exactly changed

Until now, a model's path to Spyre looked like this. The user called torch.compile, the graph went to sendnn, and from there IBM's compiler, unconnected to the rest of the PyTorch ecosystem, made all the decisions. Now the graph passes through Inductor, which handles graph capture, operator decomposition, scheduling and kernel fusion. Only an extension plugged into Inductor deals with Spyre specifics.

This is a standard pattern for Inductor. It generates Triton code for GPUs and C++ for CPUs, and its extension points let third-party devices plug in without forking the entire compiler. Spyre becomes one more such device.

Why not a custom compiler

A proprietary stack gives full control, but it is expensive. Every PyTorch innovation, from new operators to dynamic shapes and changes in AOTAutograd, has to be caught up with by hand. New models break operator coverage, and the team updates the compiler for each architecture separately.

By using Inductor, IBM gets part of this work for free:

• operator decompositions maintained by the PyTorch community;
• shared graph optimizations and operation fusion;
• a single debugging and profiling path, familiar to developers who have already worked with GPUs;
• less code for IBM to maintain on its own.

There is a cost, too. Inductor was designed around GPUs and CPUs, with their memory and parallelism models. Spyre is built differently: it is an accelerator with its own organization of compute and data. Some of the optimizations sendnn applied directly for the hardware now have to be expressed through Inductor extensions, and that may prove harder than it sounds.

What is known about Spyre

Spyre grew out of IBM's AIU research chip and is aimed at inference in enterprise systems, primarily IBM Z mainframes and Power servers. Key specifications according to IBM's public data:

• 32 accelerator cores;
• about 25.6 billion transistors, 5 nm process;
• up to 128 GB of LPDDR5 memory per card;
• PCIe form factor, around 75 W per card;
• target use case: inference of generative and classical models inside the corporate perimeter.

There is clear logic behind the accelerator. Banks and insurers want to run models next to their data without moving it to the cloud, and that requires a stack developers already know. The move to Inductor pushes Spyre in exactly that direction.

In practice

With no speed or operator-coverage tables publicly available yet, the transition is worth judging by several criteria.

1. Whether your model works with backend=inductor on Spyre without modifications, and how many graph breaks remain after compilation. 2. How compilation time has changed compared with sendnn, especially for large LLMs with dynamic shapes. 3. Whether performance on your models is comparable to the previous backend. If not, look at where the Inductor extension fails to use the hardware's capabilities. 4. How quickly new PyTorch versions arrive in the stack. This is the main practical benefit of the architectural change: the less proprietary code, the shorter the lag.

Teams planning inference on IBM Z or Power would do well to assemble a test set of their own models right away and run it on both backends while sendnn is still available for comparison. If Inductor holds up without a loss of speed, Spyre will have one less reason to differ from other accelerators in PyTorch.