Baidu Open-Sources ERNIE 5.5 Flash Base: A 280B-Parameter MoE with 14B Active

Chinese labs keep releasing large open models, and here is another notable release. On November 5, Baidu open-sourced ERNIE 5.5 Flash Base, a foundation model with a Mixture-of-Experts architecture. For anyone choosing what to build their own stack on, it is worth a closer look at the numbers.

What's inside and why MoE matters

The key specs are: 280 billion parameters in total, 14 billion active per token, 33.5 trillion tokens of pretraining, and a context of up to 128K during pretraining. According to Baidu, an even longer context is supported during fine-tuning and inference, but the figure is truncated in the text available to us, so check the model card for the exact value.

The easiest way to explain MoE is with an analogy. Imagine a large consulting firm with hundreds of specialists. For each question, you don't bring in everyone: a dispatcher, the router, picks a few suitable experts. The knowledge is held by the whole firm, but only the chosen ones work on a given token. Here, that means 14 billion out of 280 billion.

The practical consequence is twofold. Generation speed and cost are closer to a dense 14B-class model, while knowledge capacity is potentially higher. But there is a caveat that is often forgotten: all the weights still have to be loaded into memory. By my rough arithmetic, that is about 560 GB at 16-bit precision and about 280 GB at 8-bit. These are not Baidu's figures but an estimate based on parameter count, excluding cache and overhead.

Why the word Base matters

A Base model is not a chat assistant. It is trained to continue text, not to follow instructions. If you write "Write a SQL query to select the past week's orders," it may continue the sentence as if it were the start of a document instead of producing the query. To get product-ready behavior you need SFT, preference training and, possibly, RL.

On the other hand, this is good news for teams with their own post-training. A base model carries no one else's alignment or response style, and it can be tuned to your domain: legal texts, internal documentation, code in a niche language.

As for benchmarks: Baidu claims the model set a new SOTA among models of similar size on several tests. This is a vendor claim, and we have not yet seen independent measurements. Moreover, base models are usually compared on knowledge, math and code tests in few-shot mode. Such tests say almost nothing about how a model will behave in agentic scenarios with tool calling.

What to do with it

• If you have a GPU cluster and your own dataset, especially with Chinese or a narrow subject area, the model is worth a pilot.

• If you need a ready-made chat or API for a product, Base will not work. Look for an instruct version or prepare your own post-training.

• Before starting, check the license and whether your inference stack supports the model. For MoE this is critical: expert routing is not implemented equally well everywhere.

• Collect 30–50 examples from your real tasks and run them before investing anything in fine-tuning. Public benchmarks are no substitute.

The limitations are obvious. Memory requirements are high, independent comparisons are scarce, and quality in Russian you will have to check yourself: nothing is known about it from the available data.

Bottom line: ERNIE 5.5 Flash Base is interesting as one more open MoE foundation with a large pretraining volume and a reasonable per-token compute cost. But it is raw material for engineers, not a finished product. The decision to adopt it should be made after your own measurements, not based on a headline about SOTA.