Tencent открыл AngelSpec: фреймворк для ускорения инференса LLM через speculative decoding

Tencent open-sourced AngelSpec on July 30, 2026 — a torch-native framework for training draft models for speculative decoding that, on Hy3 models, raised the average acceptance rate of proposed tokens from 52,8% to 66,4% without any loss in generation quality.

What is speculative decoding

Speculative decoding is a way to speed up the generation of a large language model without losing response quality. A lightweight draft model proposes several future tokens at once, and the main (target) model verifies all of them in a single pass via rejection sampling. The method is lossless: the resulting text matches what the main model would have produced on its own. The speedup depends on two things — how many draft tokens are accepted and how long the full "draft — verify" cycle takes.

• Release date — July 30, 2026, open source license
• Torch-native framework, works on models from the Hy3 family
• Covers two approaches: autoregressive MTP and the block-parallel DFlash family (DFly, D-cut)
• Average token acceptance rate: 52,8% → 66,4% at T=0
• Average accepted draft length: 2,58 → 2,99 tokens

Why a single universal model loses out

A single universal draft model loses out because real-world traffic is heterogeneous, and two speed metrics pull in opposite directions. In open-ended dialogue with high entropy, dozens of continuations are plausible, so acceptance drops quickly with proposal depth — generating a long block isn't worthwhile. Code and math behave differently: syntax, repeated identifiers, and step-by-step derivations constrain future tokens much more strongly and create long predictable stretches. That's why AngelSpec ships two draft models instead of one compromise: MTP is trained on diverse dialogue data, while block-diffusion is reinforced with code and math examples.

"AngelSpec treats workload heterogeneity as a first-class design constraint," states the

MarkTechPost breakdown.

How the MTP path works

The MTP path resolves the train-inference mismatch through a scheme with shared parameters and multiple depths. The original Hy3 model was trained with a single MTP layer without recurrent unrolling, so tokens at the second and third draft positions were accepted noticeably worse. AngelSpec keeps D logical depths but reuses a single physical MTP block, unrolling it over D steps during training. Following the Training-Time Test principle from EAGLE-3, depth k+1 receives not the ground-truth token but the argmax prediction from depth k. Additionally, the main backbone and the language head are frozen, and training runs on rollouts of the target model itself.

The effect is concentrated exactly where it's needed. According to Tencent, on the GSM8K benchmark, third-position acceptance rose from 0,290 to 0,706, and on HumanEval — from 0,387 to 0,757, while the first position barely changed (0,799 → 0,814).

What this means

AngelSpec shows that it's more effective to build inference acceleration not around an averaged benchmark, but around the real heterogeneity of workloads — with different draft models for dialogue and for code. For teams running LLMs in production, this is an open tool for squeezing more speed out of the same hardware without a quality drop.

Frequently asked questions

What is speculative decoding in simple terms?

It's a method for speeding up LLMs where a small model guesses several upcoming tokens in advance, and a large model verifies them in a single pass. If the guess is correct, generation goes faster; the quality of the text doesn't change.

How does AngelSpec differ from a single draft model?

AngelSpec trains two specialized models: MTP for high-entropy dialogue and block-parallel for code and math with long predictable stretches. According to Tencent's measurements, this raised the average token acceptance rate from 52,8% to 66,4%.