HyenaND: A Subquadratic Operator That Replaces Attention on Multidimensional Data

Researchers have introduced HyenaND, a subquadratic neural network operator that replaces the attention mechanism on multidimensional data. It processes images, volumes and partial differential equations (PDEs) in their native geometry, without flattening them into a one-dimensional sequence. The preprint appeared on arXiv in July 2026, and its key promise is O(L log L) complexity instead of attention's quadratic O(L²).

Where attention falls short

Attention scales quadratically: for an input of length L, its cost grows as O(L²), which makes long context expensive. According to the authors, every existing subquadratic alternative has to make compromises on multidimensional data.

Standard convolutions lack a global receptive field and input dependence. Recurrent models require unrolling an image or a three-dimensional volume into an artificial one-dimensional traversal order, and that order destroys the original spatial structure of the data.

How HyenaND works

HyenaND is a global, input-dependent operator that acts directly on the native geometry of multidimensional data through convolutions with implicitly parameterized multidimensional kernels. Unlike a regular convolution, the kernel here is global and adapts to the specific input.

Practical speed comes from nSubQ, a CUDA implementation that fuses the convolution path through the fast Fourier transform (FFT) into a single kernel, turning the theoretical O(L log L) complexity into real wall-clock speedups.

• HyenaND is a subquadratic, global, input-dependent operator
• Works with images, 3D volumes and PDEs without flattening them into 1D
• The nSubQ CUDA kernel fuses the FFT convolution path into a single operation
• Complexity of O(L log L) versus attention's O(L²)
• Tested across 4 domains: genomics, vision, medical imaging, PDEs

What the experiments showed

Pure HyenaND stacks matched strong attention baselines in accuracy across four types of tasks: long-context genomics, computer vision, medical imaging and modeling of partial differential equations. In other words, dropping quadratic attention did not cost quality.

Hybrid configurations did even better: interleaving HyenaND and attention layers outperformed both pure attention and recurrence-based hybrids.

"Hybrid configurations that interleave

HyenaND and attention layers outperform both pure attention and strong recurrence-based hybrids," the preprint's abstract on arXiv states.

What this means

The race to replace attention is shifting from one-dimensional text to multidimensional data. HyenaND shows that it is possible to keep transformer-level accuracy and eliminate the quadratic cost where the data is inherently two- or three-dimensional, as in medicine, biology and physical simulation.

FAQ

What is HyenaND?

HyenaND is a subquadratic neural network operator and an alternative to attention. It processes multidimensional data (images, volumes, PDEs) in its native geometry through global, input-dependent convolutions and scales as O(L log L).

How does HyenaND differ from attention?

Attention scales quadratically, as O(L²) in input length. HyenaND offers the same global receptive field and input dependence, but at O(L log L) complexity, and does not require flattening multidimensional data into a one-dimensional sequence.

Where is HyenaND used?

According to the preprint, HyenaND has been tested on long-context genomics, computer vision, medical imaging and modeling of partial differential equations.