The download is roughly the size of a small film archive, and when it finishes there is no chat window waiting at the end of it.
Baidu's ERNIE team published ERNIE 5.5 Flash Base on 5 November: the raw end-of-pretraining weights of its mid-tier model, before any instruction tuning or preference training. Ask it a question and it will not answer politely. It will continue your text, because that is all it has been taught to do.
That is the point.
Key takeaways
- ERNIE-5.5-Flash-Base is a sparse mixture-of-experts language model with 280 billion total parameters and 14 billion active per token, pretrained from scratch on 33.5 trillion tokens.
- It supports a 128K training context and up to a 1M-token context for fine-tuning and inference, and ships in FP8 and BF16 on Hugging Face under Baidu's own licence, not Apache or MIT.
- The weights are the pre-training checkpoint only — no mid-training, no post-training — so the release is a starting point for domain adaptation, instruction tuning and reinforcement learning, not a product.
- The architecture is built around long-context cost: most layers compress every 128 tokens into a single cache entry, and only one layer in seven keeps token-level access to the full context.
- Baidu reports strong base-model scores against three comparable open checkpoints, but those numbers are vendor-run and the comparisons do not sweep every benchmark.
What a base checkpoint is for
Most open-weight releases are finished products: someone else has already done the supervised fine-tuning, the preference optimisation and the safety work, and you inherit their choices.
A base checkpoint transfers the most expensive and least visible part of the pipeline — the pretraining run — and nothing else. For a lab with a domain corpus and a strong opinion about how a model should behave, that is often better than inheriting a competitor's personality. For a company that wants a chatbot by Friday, it is useless.
Baidu is explicit about the trade. The release notes frame the checkpoint as a "common starting point" for domain adaptation, instruction tuning and reinforcement learning research, and say a post-trained ERNIE 5.5 Flash will follow.
The economic logic is straightforward. Pretraining a model of this class costs a great deal; post-training is comparatively cheap. By giving away the first and keeping the second, Baidu can seed an ecosystem and still sell the finished model through its API.
The architecture is an argument about cost
The interesting part of this release is not the parameter count. It is how aggressively the design trades detail for memory.
ERNIE 5.5 Flash has 43 Transformer layers, a hidden size of 4,096 and 64 attention heads. In 37 of those 43 layers the model uses what Baidu calls heavily compressed attention: every 128 tokens are folded into one cached entry, and the layer attends over those compressed entries plus a short window of recent, uncompressed tokens. The layer still sees the whole sequence, but stores a small fraction of what a full-context layer would need.
Only one layer in seven keeps token-level access to the full history, using multi-head latent attention with sparse selection. Baidu says that at a comparable KV-cache budget, this split outperformed the alternative of using more compressed layers.
On the expert side, each MoE layer selects 10 of 512 routed experts plus one shared expert, working in a compressed latent space that Baidu says roughly halves communication and expert compute. A multi-router design replaces the usual single router; a low-rank gate cuts gate projection parameters by 75% without changing the number of active experts.
Everything here points the same way: the model is designed to be cheap to keep running, not to win a single-turn benchmark.
The benchmark table, read carefully
Baidu evaluated the checkpoint against three other recently released open base models of similar size — DeepSeek-V4-Flash-Base, Hy3-preview-Base and MiMo-V2.5-Base — using what it says is a single harness, prompt set, few-shot configuration and judge.
In that setup the company reports MMLU-Pro at 78.4 against 69.2 for the DeepSeek checkpoint, and HumanEval+ pass@1 at 82.3 against 69.5.
The same table shows the comparisons winning elsewhere: the DeepSeek checkpoint leads on ChineseSimpleQA, on BBH with chain-of-thought prompting, and on InfiniteBench. That is a more useful disclosure than a clean sweep would have been, and it is the sort of result you expect from a model tuned for efficiency rather than for aggregate score.
None of it is independent. Vendor-run base-model comparisons are directional evidence, not a leaderboard.
Where the weights actually land
The release is on Hugging Face in FP8 and BF16 safetensors, with the BF16 repository at roughly 573 GB. Serving is documented through SGLang and vLLM, and there is a fine-tuning example in NVIDIA's Megatron-Bridge repository.
The licence is Baidu's own agreement, version 1.0 — not Apache 2.0, not MIT. For most research use that distinction will not matter; for a company planning to build a commercial product on the weights, it is the first document to read, not the last.
What this says about the open-weight race
Until recently, releasing a Chinese model's raw pretraining checkpoint at this scale would have been unusual. Now it is a pattern: DeepSeek, Alibaba's Qwen, Tencent's Hunyuan and Baidu are all publishing base weights and letting the community do the alignment work.
The strategic bet is that the binding constraint has moved. When everyone can pretrain a competent base, differentiation shifts to post-training data, agent harnesses and, above all, the cost per useful token. A "Flash" model that is cheap to run for millions of long agent sessions is a more valuable asset than a marginally smarter one that is expensive to serve.
Honest limitations
The architecture description and all benchmark scores are Baidu's account of its own model; I did not re-run any evaluation, and I found no independent leaderboard entry for this checkpoint. The 280B/14B, 33.5T-token and context-length figures are company statements. The compared models were evaluated by Baidu under its own harness, which is not the same as a neutral third-party run. The post-trained model had not been released when this was written, so no claim about chat quality or tool use can be tested. The licence's commercial terms were not analysed here.
What readers can do now
- If you fine-tune, benchmark the base checkpoint on your own task before comparing it with an instruct model; the gap is usually larger than the vendor table suggests.
- Read the licence before the architecture blog; a non-standard agreement can decide a product plan faster than a benchmark can.
- Watch whether the post-trained ERNIE 5.5 Flash keeps the efficiency profile — that, not the base scores, is what determines serving cost.