Nestor: nested sparsity in attention beats MoE at equal compute

Mixture-of-experts has long been the standard backbone of large language models: only a small fraction of the weights fire on any given forward pass. A team from the University of Hong Kong and Berkeley proposes adding a second layer of this economy. Tianzhe Chu, Druv Pai, and Yi Ma describe the Nestor architecture, in which conditional computation is built into the attention operator and nests inside MoE sparsity. According to their findings, at equal compute and equal sparsity, Nestor achieves better language model pretraining quality than a classic MoE transformer. The paper appeared on arXiv on October 22, 2026 (2610.31818), with code and checkpoints published.

Where the second sparsity hides in the transformer

In a standard MoE transformer, sparsity lives in one place. The MoE operator generalizes the multilayer perceptron from the classic transformer: a router selects which experts process a given token, while the rest of the weights sit idle on that pass. The attention block, meanwhile, remains dense.

The Nestor authors go symmetric. They introduce a conditional computation mechanism that generalizes the attention operator itself, rather than the MLP. This yields two independent axes of savings. They combine hierarchically — that is, they nest inside one another, hence the name. A single transformer gets sparsity in both its attention part and its MoE part, rather than just one of the two.

The comparison is set up carefully: models are compared at equal compute and equal sparsity. This cuts off the simplest objection — that the gain was bought with extra FLOPs or a larger number of active parameters.

Fine-tuning on long context and new domains

The most practical part of the work concerns continual pretraining, i.e., continuing to pretrain an already-trained model. For labs and companies, this is routine: a base model is regularly stretched to a longer context or further trained on a new domain.

According to the authors' results, Nestor handles this better than standard MoE:

• it adapts more efficiently to longer context;
• it switches more efficiently to new domains;
• it forgets less on old domains.

That last point matters on its own. Catastrophic forgetting remains the main price paid for fine-tuning, and an architecture that lowers that price without separate tricks like replay buffers is of interest to anyone running models through multiple stages of training.

The authors also checked scaling. The advantages hold as model size, dataset size, and context length grow. For architectural ideas, this is the main test: many improvements look good on small models and vanish on large ones.

What's known about the work

• Architecture name: Nestor (nested sparsity).
• Authors: Tianzhe Chu, Druv Pai, Yi Ma; affiliation listed on the paper's page — The University of Hong Kong.
• Idea: a conditional computation mechanism that generalizes the attention operator, by analogy with how MoE generalizes the MLP.
• Relation to MoE: the sparsities nest hierarchically inside one another.
• Pretraining: gains over a standard MoE transformer at equal compute and equal sparsity.
• Continual pretraining: better adaptation to long context, more efficient transition to new domains, less forgetting of old ones.
• Scaling: the effect holds as model size, data, and context grow.
• Artifacts: code and checkpoints in the Ma-Lab-Berkeley/Nestor GitHub repository, with a project page at ma-lab-berkeley.github.io/Nestor.

What practitioners should do with this

For now this is an academic paper without deployment in production models, and it has a single vote on the Hugging Face Daily Papers page. But the material is open, meaning the idea can be checked without negotiating with the authors. Teams training their own MoE models would do well to take the checkpoints and code and compare Nestor against their baseline architecture on context-extension and domain-adaptation tasks — that's exactly where the biggest effect is claimed.

For infrastructure teams, there's a separate question: how the second sparsity axis will map onto real kernels and distributed training. A gain at equal compute in theory doesn't guarantee a wall-clock gain on a cluster as long as there are no optimized implementations for attention routing. If such implementations appear, MoE will stop being synonymous with sparse MLP, and the field for architectural experimentation will expand noticeably.