Inclusion AI speeds up LLM generation by up to 6.9x with a draft tree

Inclusion AI has introduced UBTree, a way to speed up text generation in large language models. A small model sketches a draft in advance as a tree of 64 candidate continuations, and the large model checks all branches in a single pass. The "6.88×" figure from the benchmarks means that on Qwen3-4B, generation ran almost seven times faster than the standard mode, where the model emits one token per pass. On Qwen3-8B the speedup was 6.94×. These are averages across seven benchmarks, ranging from the grade-school math of GSM8K and the olympiad math of AIME to the coding tasks of LiveCodeBench and the dialogues of MT-Bench.

Where fast parallel drafters break down

Speculative decoding rests on a simple idea. A large model is slow because it runs all of its weights for every token. So a small draft model (the drafter) proposes several tokens ahead. The target model checks them in one pass and accepts everything up to the first error. The longer the accepted chain, the bigger the gain.

Drafters come in sequential and parallel varieties. A sequential drafter writes tokens one by one, while a parallel one emits a whole block at once. DFlash and DSpark are parallel drafters, and UBTree builds on that family. They are fast, but they have a weak spot: they score each position in a block almost independently of its neighbors. When the target model's distribution is flat and several continuations are nearly equally likely, such a drafter guesses poorly which words will sit next to each other. The chain breaks off early.

Two passes: gathering candidates and stitching them into a tree

UBTree fixes this without giving up parallelism. The draft is assembled in two parallel steps.

• Step one. The DFlash model picks 64 token candidates for each of the 16 positions in the block. It still scores the positions independently.
• Step two. A bigram selector scores how well one candidate works as a continuation of another. It passes each pair of adjacent tokens through two small perceptrons on top of the target model's frozen embeddings, one for the predecessor and one for the successor. It adds the resulting score to the DFlash score. The weight of the score depends on the depth of the position.

All pairs are computed simultaneously. The transition scores are used to assemble a tree of 64 nodes, and the target model verifies the whole tree in a single pass. One branch may not match the large model's choice, but another will.

The selector was trained on responses regenerated at high temperature, with a renormalized KL loss. This labeling puts not only the most likely continuations into the training data but also plausible alternatives. As a result, the tree more often contains the branch the target model will pick.

What the measurements showed

• Qwen3-4B and Qwen3-8B: average speedup of 6.88× and 6.94× across seven benchmarks.
• Versus DARTree: UBTree beat the closest tree-based method in all 28 comparisons, by 7.4–13.4% on average.
• Ling3-Flash-124B in SGLang: 3.27× versus 2.73× for DSpark and 2.80× for DFlash.
• Qwen3.8-27B: 4.78× versus 4.32× for DSpark.
• Long context (LongBench-v2): the number of tokens accepted per verification round is 31.0–39.7% higher than for the strongest competitor.

Note the spread. The nearly sevenfold gain comes from small Qwen3 models. On the 124-billion-parameter Ling3-Flash in production serving, the speedup is more modest at 3.27×. The difference is still noticeable: the same measurements give 2.80× for DFlash and 2.73× for DSpark.

What you can use right now

Alongside the method, Inclusion AI has released two draft models under the MIT license:

• Ling3-Tiny-UBTree, 183 million parameters. It works in tandem with Ling-3.0-tiny (8 billion parameters, 1 billion of them active).
• Ling3-Flash-UBTree, 482 million parameters. It works in tandem with Ling-3.0-flash (124 billion parameters, 5 billion active).

These models generate nothing on their own. They work only as drafters for their own target model. UBTree support for Ling3 in SGLang has not been published yet, and the team promises additional draft models within a month.

For engineers who host models themselves, the takeaway is this. The speedup comes without loss of answer quality, through a small add-on, and the large model's weights do not need to change. But ready-made drafters exist only for two models in the Ling family so far. For any other target model, you will have to train a drafter yourself, and the Qwen3 numbers are worth rechecking on your own workload.