Baidu Open-Sources ERNIE 5.5 Flash Weights: 14 Billion of 280 Billion Parameters Are Active per Token (5%)

Baidu has published the weights of ERNIE 5.5 Flash Base, an open language model with an MoE (mixture of experts) architecture. The headline contains two numbers, and it's worth understanding what they mean.

280 billion is the total parameter count, i.e. the "knowledge" the model stores.

14 billion is how many of those parameters actually take part in processing each token (a token is a fragment of a word, the pieces from which the model assembles text). 14 out of 280 is exactly 5%. Picture an office of 20 specialists where each task goes to the one suitable person rather than to everyone at once: the staff is large, but you pay for the work as if it were a small team.

The takeaway for developers is simple: if you are counting inference costs, a model like this can be cheaper than a dense model of the same size, but you first need to test it on your own data.

What exactly was released

Baidu Research has open-sourced the base version, ERNIE 5.5 Flash Base. "Base" means a raw model without the final tuning for dialogue: it is not a ready-made chat assistant but a starting point, for example for your own fine-tuning on industry-specific data. The company promises to release the full ERNIE 5.5 Flash later.

Inside the model are specialized blocks, the "experts." For each token, a dedicated router chooses which of them to activate. As a result, the compute required is roughly that of a 14-billion-parameter model, while the stored knowledge is that of a 280-billion-parameter one. The developers claim the model matches the results of leading models of comparable size. The announcement gives no specific figures, and there were no independent measurements at the time of publication either, so for now this is a hypothesis.

Three engineering decisions

Baidu highlights several technical innovations.

Latent-Gated Sparse Attention handles memory when working with long text. While generating a response, the model keeps a KV-cache, a cache of already processed tokens, which grows sharply on long documents. The new mechanism allocates this memory so that the model can still find the information it needs in a long context without wasting space. In practice, this is about long contracts, logs, or a large codebase in a single request.

Multi-Router Latent MoE uses several routers, not just one, to select experts. According to Baidu, this identifies more precisely which parts of the model a given token needs, without increasing compute costs.

ECHO combines multi-token prediction with the model's main layer. The same mechanism can be used for speculative decoding. The idea is that the model first proposes a draft of several upcoming tokens, then verifies it. If the guesses are right, the response is generated faster. For services where response time matters, this may be more useful than yet another benchmark gain.

For training, the team developed a distributed Muon optimizer and an optimized attention kernel, FlashMask-4.

What this means in practice

First, the savings with MoE apply to compute, not memory. All 280 billion parameters still have to be stored somewhere: at 16-bit precision, that is on the order of 560 GB for the weights alone. You will not be able to run such a model on a single GPU; you will need a server with several accelerators, or quantization (compressing the weights). The gain is more noticeable where request volume is high: each token takes less compute than in a dense model of the same size.

Second, since this is a base model, don't expect a ready-made agent or assistant out of the box. If you specifically need dialogue or agentic scenarios, it makes more sense to wait for the full ERNIE 5.5 Flash.

Third, for the open-model market this is another large candidate to serve as the foundation for in-house stacks. The weights are on Hugging Face, and a technical write-up is published on the ERNIE website.

It's best to start small. Take your own set of test tasks, run ERNIE 5.5 Flash Base on it, and compare it with what you use now. Look at both quality and serving cost, because the latter is exactly why such models are built.