--- title: "Baidu open-sources ERNIE 5.5 Flash Base: a 280B-parameter model that bets on efficiency over size" date: 2026-11-07 category: Companies & Stack site: NeuroAI canonical: https://neuroai.site/a/na-corp-baidu-ernie-55 language: en --- # Baidu open-sources ERNIE 5.5 Flash Base: a 280B-parameter model that bets on efficiency over size > Baidu released and open-sourced ERNIE 5.5 Flash Base on 5 November, a 280-billion-parameter MoE model trained on 3.35 trillion tokens, with up to 1-million-token context and a string of efficiency-focused architecture changes. A product manager opens a 400-page contract, asks a model to summarize the risky clauses, and then watches the model stall — the document is longer than the model can hold in memory. Multiply that by every enterprise that now wants AI to read filings, codebases, and chat histories, and you get the real pressure cooker in large model (大模型) development this year: not raw intelligence, but how much context a model can chew through before the bill and the latency become unbearable. On 5 November, Baidu released and open-sourced ERNIE 5.5 Flash Base (文心 5.5 Flash Base), a base model built around exactly that problem. It is not the biggest model Baidu has shipped, but it is the clearest signal yet that the company is competing on efficiency rather than parameter bragging rights. ## Key takeaways - ERNIE 5.5 Flash Base uses a Mixture-of-Experts (MoE) architecture with **280 billion total parameters** but only **14 billion active per token**. - It was pretrained on **3.35 trillion tokens**, with a 128K-token pretraining context that extends to **1 million tokens** during fine-tuning and inference. - Baidu reports state-of-the-art results among same-sized models, claiming wins over DeepSeek-V4-Flash-Base, Hy3-preview-Base, and MiMo-V2.5-Base. - The model is open-sourced on Hugging Face, giving developers a base checkpoint to fine-tune rather than a closed endpoint. - The design theme is "efficiency": new attention, routing, and training-target mechanisms aim to cut long-context cost. ## What the model actually is ERNIE 5.5 Flash Base is a *base* model, not a chat-ready product. That distinction matters. A base model is the raw pretrained checkpoint that developers and enterprises take and adapt — for a legal assistant, a coding agent (智能体), or an internal search tool. By open-sourcing the base rather than only the tuned endpoint, Baidu is targeting the downstream developer ecosystem, not just end users. The headline numbers: 280 billion parameters in total, of which 14 billion are activated for each token processed — a classic sparse-MoE trade, where most of the model stays dormant and only a slice fires per query. Pretraining used 3.35 trillion tokens, and the context window reaches 128K tokens in pretraining, expandable to 1 million tokens when fine-tuned and served. For perspective, a million-token window is roughly the length of several full novels in a single prompt. ## The efficiency bet Baidu's engineering story centers on three mechanisms. For attention, it proposes **Latent-Gated Sparse Attention**, combining hybrid sparse attention, low-rank gated attention, and virtual-head attention; the gated projection's parameter count drops by 75 percent versus a conventional design. For expert routing, it uses **Multi-Router Latent MoE**, where several independent routers jointly decide which experts to activate, meant to reduce the bias of any single router's score. For the training target, it introduces **ECHO**, which reuses the backbone's final layer to predict upcoming tokens instead of adding a separate multi-token-prediction module, improving inference and code ability while keeping the draft model usable for speculative decoding. Baidu also improved its distributed Muon optimizer and released **FlashMask-4**, an attention operator tuned for large head dimensions, to lift communication and memory efficiency in large-scale training. None of these are consumer features; they are the plumbing that decides whether a model is cheap enough to deploy at scale. ## Why open the base model The base-model release fits a broader Chinese pattern this year: the competitive frontier has moved from "who has the smartest closed model" to "who builds the largest usable ecosystem." Open-weight bases let startups, universities, and enterprise teams skip the cost of pretraining and instead specialize. Baidu is explicitly pitching ERNIE 5.5 Flash Base as a foundation others build on — the company notes that, unlike an end-user application model, a Base model reserves room for fine-tuning and customization against specific business needs. That is also a hedge. As agents (智能体) and long-document reasoning become the actual workload, the value shifts to the training recipe and the ecosystem around a model, not the flagship benchmark. An open base model is a way to seed that ecosystem. ## What readers can do now - **If you build with Chinese large models, benchmark ERNIE 5.5 Flash Base against DeepSeek-V4-Flash-Base and MiMo-V2.5-Base** on your own long-context tasks before committing — Baidu's SOTA claim is self-reported and should be re-tested on real workloads. - **Treat "1M-token context" as a fine-tuning/inference capability, not a default setting.** Real latency and cost depend on how you serve it. - **Watch the license and the checkpoint.** Open-sourcing on Hugging Face signals accessibility, but read the terms before shipping a commercial product. ## Honest limitations This article relies on Baidu's official release (ernie.baidu.com), Xinhua, and technology-news reprints. The core specifications — 280B total / 14B active parameters, 3.35T training tokens, 128K/1M context, and the SOTA claim over DeepSeek-V4-Flash-Base, Hy3-preview-Base, and MiMo-V2.5-Base — are reported as Baidu states them and were not independently re-measured. The architectural details (Latent-Gated Sparse Attention, Multi-Router Latent MoE, ECHO, FlashMask-4) come from Baidu's own write-up and are not yet validated by third-party reproduction. I have not benchmarked the model myself, and I have not covered ERNIE 5.0/5.1's consumer-facing features, which are outside this column's scope. Figures from Chinese-language outlets are treated as corroborating, not as primary fact sources. --- Published by NeuroAI (https://neuroai.site/) — https://neuroai.site/a/na-corp-baidu-ernie-55 Free to quote with attribution and a link to the original.