NeuroAI NEUROAINEUROAI.SITE
China AI, reported clearly
1,300+ verified stories on models, robotics, BCI and the business of AI.
Browse stories
ESC

StepFun Opens a 321B Reasoner Built Around the Cost of Thinking Slowly

A Shanghai lab open-weights a 321B multimodal model whose paper argues decoding cost, not parameter count, is the real frontier.

2026-11-17 · 917 words · NeuroAI
StepFun Opens a 321B Reasoner Built Around the Cost of Thinking Slowly

Ask an engineer what makes a thinking model expensive to run, and the answer is rarely its size. It is the slow, token-by-token stretch while the model works through a problem, burning GPU time a user never sees. StepFun (阶跃星辰), a Shanghai startup, has published a large model (大模型) built from the ground up to shorten that stretch.

The model, called Step-3, is a 321B-parameter (3210亿参数) multimodal reasoning model (推理模型). It handles both text and images. Only 38B parameters activate per token. StepFun released the weights under Apache 2.0 in late July 2025, after unveiling the model in Shanghai on the eve of that year's World Artificial Intelligence Conference. The accompanying paper, submitted to arXiv on 25 July 2025, has a title that reads like a thesis statement: "Step-3 is Large yet Affordable."

The argument: stop shrinking the model

The usual way to make a model cheaper is to make it smaller. StepFun's paper takes the opposite position. It argues that decoding cost is a property of how a model interacts with hardware, not simply of how many parameters it holds. Three things matter: the arithmetic intensity of attention, the sparsity of the mixture-of-experts (混合专家, MoE) layers, and exactly where attention and feed-forward computation are placed on a machine.

Step-3 therefore activates 38B parameters per token — more than DeepSeek-V3 or Qwen3 MoE 235B activate, according to the paper — yet StepFun reports lower theoretical decoding cost on the hardware it studied. The claim is not that the model is small. It is that the model is arranged so that each GPU does more useful work per watt.

Two changes to the plumbing

The design rests on two techniques, both named in the paper and both aimed at the mechanics of serving a model rather than at its intelligence.

  • Multi-Matrix Factorization Attention (MFA). A low-rank factorization of the query dimension — 2,048 rather than the full hidden size of 7,168 — reduces both the size of the KV cache and the amount of attention computation, while StepFun says it preserves expressiveness.
  • Attention-FFN Disaggregation (AFD). A distributed inference system that separates attention layers from feed-forward layers into specialized subsystems, so each can run on hardware suited to it.

The rest of the card is conventional but specific: 61 layers, of which five are dense; 48 experts per MoE layer with three selected per token plus one shared expert; a 65,536-token context window. Checkpoints ship in bf16 and block-fp8, and StepFun recommends vLLM or SGLang for serving.

The number worth reading twice

The paper reports a head-to-head test on Hopper GPUs. Under a 50 ms time-per-output-token service-level objective, with 4K context, FP8 and no multi-token prediction, Step-3 reached a decoding throughput of up to 4,039 tokens per second per GPU. DeepSeek-V3, in the same setup, reached 2,324.

Those figures come from StepFun's own implementation and its own chosen scenario; the paper explicitly says it picked a setting favorable to the comparison model. That caveat matters. A single throughput number on one accelerator generation is evidence of an engineering result, not a universal law. What survives the caveat is the design point: a team can spend parameters it once would have cut, if it spends them in the right place.

Why a Shanghai startup cares about cost per token

China's large-model labs have spent two years competing on benchmark scores and release cadence. Step-3 is a bet that the next constraint is economic. Reasoning models think longer, which means more decoded tokens, which means the cost of each token starts to determine whether a model can be offered cheaply at scale. A model that is "large yet affordable" is trying to win on the unit economics of inference rather than on a leaderboard alone.

The company's own history points the same way. StepFun (formally Shanghai Jieyue Xingchen Intelligent Technology Co., Ltd.) was registered in April 2023 and had already open-sourced a video-generation model and a speech model with Geely Auto before Step-3. Multimodal foundation models are its stated direction, and StepFun describes Step-3 as its first full-size, natively multimodal reasoner.

For developers outside China, the open weights are the practical story. A 321B model is not something most teams can run in a cupboard, but Apache 2.0 means it can be inspected, fine-tuned and deployed commercially without a licensing conversation. Whether the hardware-aligned design ports cleanly to accelerators other than the ones studied remains an open question.

Honest limitations

  • Step-3's cost and throughput numbers come from StepFun's own paper and its own hardware setup. No independent reproduction of the 4,039-versus-2,324 comparison was found.
  • Theoretical decoding cost is not the same as a production bill. Network topology, batch size, real context lengths and token prices all change the outcome.
  • The model is large. Open weights remove license friction, not the need for substantial memory and a multi-GPU serving stack.
  • This article did not benchmark the model, and the 'first full-size, natively multimodal reasoner' description comes from StepFun's own materials.

What readers can do now

  1. Separate training cost from serving cost when evaluating a model. A cheaper-to-serve model can be worth more than a slightly smarter one if you expect heavy, long-running traffic.
  2. If you serve open models, read the attention and expert layout before the benchmark table. KV-cache size and expert routing predict your hardware bill more directly than a score does.
  3. Treat the Hopper numbers as a hypothesis to test on your own stack. Download the fp8 checkpoint and measure tokens per second under your latency target before committing.

Related coverage

Cite this article

NeuroAI (脑机网), "StepFun Opens a 321B Reasoner Built Around the Cost of Thinking Slowly," 17 November 2026, https://neuroai.site/a/na-model-step3-decoding-cost

Machine-readable: BibTeX · RIS · JSON · Markdown

More in “Foundation Models” → · Back to home · Markdown version

China AI, reported clearly
1,300+ verified stories on models, robotics, BCI and the business of AI.
Browse stories