CircuitTree assembles 39 HuBERT phoneme circuits into a syllable transcriber with 64.8% accuracy

Researchers have proposed CircuitTree, a data structure that assembles an arbitrary number of small subnetworks, or "circuits," from one large neural network into a single working model. They tested the method on the HuBERT speech model. They merged 39 circuits, each of which recognizes a single phoneme, into a syllable classifier, and then into a pipeline that transcribes open syllables with 64.8% accuracy. A model that does not share components between circuits reaches 66.0%. The gap is 1.2 points, even though 99.5% of the circuit components in the new system are shared.

Why circuits matter and what makes assembling them hard

Circuit discovery tries to break a monolithic network down into tiny modular subnetworks. Each one is supposed to handle a clear subtask, which makes it interpretable. This is one of the central directions in mechanistic interpretability. But taking a model apart is only half the job. If the pieces cannot be put back together into something meaningful, the value of the teardown is limited.

Earlier attempts at composition relied on naive circuit union. The authors argue that this approach has limitations, and they propose a greedy algorithm with density-scaled composition. The abstract does not describe the specific mechanisms, but the goal is clear: merge circuits so that the resulting subnetwork does not bloat and keeps the capabilities of the original parts.

How CircuitTree works

Pairwise merging solves the problem for two circuits. The authors generalized it to N circuits and built CircuitTree, a data structure based on a tree-compose schedule with Θ(N log N) complexity. Merging hundreds of circuits one by one is already impractical, and the tree schedule keeps the growth in cost manageable.

CircuitTrees are then chained together to form a hierarchy-of-circuits pipeline. The bottom level handles phonemes, and the next level assembles syllables from them.

What the team got on HuBERT

The researchers extracted circuits for classifying individual phonemes from HuBERT, 39 in total. None of them can distinguish syllables on its own, but their composition can. In effect, a new capability emerges from combining narrow ones.

Key figures from the abstract:

• number of phoneme circuits: 39;
• open-syllable transcription accuracy: 64.8%;
• ceiling for a model with no shared components: 66.0%;
• gap to the ceiling: 1.2 points;
• share of shared circuit components: 99.5%;
• composition schedule complexity: Θ(N log N).

The 99.5% figure matters most. The 66.0% ceiling comes from a model whose circuits do not overlap at all. The new system nearly matches it, even though almost all of its hardware is shared. Modularity here costs almost nothing in quality.

Where this could be useful and what remains open

The practical value of the work is that interpretable modules can be not only found but also assembled into a working system. For model auditing, this makes it possible to check which parts a decision is built from. For speech-system developers, it points toward compact, specialized subnetworks that inherit the knowledge of a large model. The abstract does not measure gains in size or speed, so any conclusion about savings would be premature.

The limitations are substantial. The task is narrowed to open syllables rather than arbitrary speech. The absolute accuracy of 64.8% is far from the level of production speech recognition systems. The abstract includes no comparison with conventional ASR models. This is a study of a composition tool, not a ready-made transcription solution. But as proof that dozens of narrow circuits can be combined into a new capability with almost no loss in quality, it looks convincing. The next thing to watch is whether the approach scales to other modalities and to language models, where there are orders of magnitude more circuits.