--- title: "Can a 600-billion-parameter model keep pace with the trillion-parameter giants?" date: 2026-10-20 category: Foundation Models site: NeuroAI canonical: https://neuroai.site/a/na-step-5-preview-flagship-moe language: en --- # Can a 600-billion-parameter model keep pace with the trillion-parameter giants? > StepFun's Step 5 Preview pairs a 600B sparse model with a one-million-token window and open weights, a top-three open model on a major index. Most of the attention in Chinese model development has gone to the biggest releases. A Shanghai lab called StepFun (阶跃星辰) is arguing that the smarter race is about doing more with less. Its new flagship, Step 5 Preview, tries to prove that a leaner design can sit beside the heaviest models without matching their bills. The pitch is simple to state and hard to pull off. Instead of stacking parameters until a model reaches the trillion range, StepFun built a large model (大模型) that activates only a small slice of itself on each request. The company says the result lands in the same conversation as the most capable open releases while costing a fraction of what closed flagships charge per task. ## What Step 5 Preview actually is Step 5 Preview is a sparse Mixture-of-Experts model, or 混合专家 in Chinese. The total parameter count is 600 billion, but only about 27 billion of those wake up for any single token. That gap is the whole point: the model carries a huge knowledge reservoir while keeping the compute (算力) bill per request small. A few design facts stand out from the official release and the launch coverage: - **One-million-token context.** The context window (上下文窗口) spans one million tokens, enough to hold an entire code repository, a long financial filing, or a full book in a single pass. - **Native multimodal (多模态) input.** The model accepts text and images together, so a user can hand it a screenshot of a dashboard and ask it to reason across both. - **A "narrow and deep" stack.** StepFun describes a 92-layer transformer with a technique it calls Sparse GQA, plus a token-merging method that cuts the cost of its indexing and routing steps by roughly eight times. - **Open weights on the way.** The weights were promised for full open release on October 15, 2026, after the model had already been served through the company's own API and third-party routers. ## How it scores, and where the numbers come from StepFun published a set of benchmark results alongside the launch. As with any vendor-reported numbers, they should be read as claims from the builder, not as independently audited truth. The most cited figure is its score on the Artificial Analysis Intelligence Index, where the model reached 44 points. That placed it among the top three open-source models measured on that index at launch. On software engineering, the company reported a score of 67.7 on an internal DeepSWE evaluation, behind only two closed US flagships at the time. In finance, a third-party benchmark called FrontierFinance — built from 220 professional questions and more than 11,000 assessment criteria — gave Step 5 Preview a score of 66.4, which the lab said ranked it second globally and first among open models. The finance angle is not accidental. StepFun trained the model with finance as a stress test for long-horizon behavior: long task chains, very long context, and repeated tool calls all show up first in financial workflows. ## The pricing argument StepFun frames the model around cost as much as capability. The official pricing at launch was US$1 per million input tokens and US$2.70 per million output tokens. The company estimated that a single intelligence-index task costs about US$0.71 to run, which it said is roughly one-eighth of what a comparable US flagship charges for the same level of performance. That math depends on how you define "the same level of performance," and a buyer should test it against their own workloads. But the direction is consistent with a broader Chinese trend: open-weight models competing less on headline scores and more on the cost per useful answer. ## Deployment takeaways The model is positioned for four job families: AI coding, software engineering, professional knowledge work, and finance. Because the weights are open, a team can run the model inside its own infrastructure rather than sending sensitive documents to a hosted API. According to Li An, chief scientist at BrainNet (脑机网), the more interesting shift in Chinese model work is not raw size but how far a design can stretch each activated parameter before quality drops. For teams that want to try it, the practical path is straightforward: start through the hosted API or a router that offers a free tier, benchmark it on your own coding and document tasks, and only then decide whether to pull the open weights for private deployment. ## Honest limitations Most of the headline benchmark figures are vendor-reported and should be treated as marketing-anchored until reproduced by independent evaluators. The model's per-task cost advantage depends on workload shape and on how a buyer measures "equivalent" performance, so the one-eighth claim will not hold for every use case. Step 5 Preview is multimodal on input but the launch material emphasizes text-and-image rather than full video understanding. The open-weight release was promised for mid-October 2026, and real-world inference cost at the one-million-token context length will depend heavily on the serving stack a team chooses. ## What readers can do now - Benchmark Step 5 Preview against your own coding and long-document tasks through the hosted API before trusting any vendor benchmark, and record latency at the one-million-token context length you actually need. - If you handle regulated or sensitive data, plan a private deployment once the open weights land, and validate that your inference hardware can sustain the 27-billion active-parameter load without unexpected memory spikes. - Treat the finance and software-engineering scores as a starting hypothesis: build a small internal eval set from your domain and re-measure, since vendor numbers rarely match a specific production workflow. --- Published by NeuroAI (https://neuroai.site/) — https://neuroai.site/a/na-step-5-preview-flagship-moe Free to quote with attribution and a link to the original.