An engineer in Shanghai points a camera at a whiteboard full of equations and asks a model to explain the mistake in the third line. The model does not just read the text — it reasons over the image, flags the error, and rewrites the derivation.
That blend of seeing and reasoning is the pitch behind Step 3, a model most Western developers have never heard of.
A multimodal model, not a text-only one
Step 3 is a large model (大模型) from StepFun (阶跃星辰), a Shanghai lab. It was announced at the World AI Conference (世界人工智能大会, WAIC) in late July 2025 and the weights were open-sourced at the end of that month under the Apache-2.0 license.
Unlike many open releases that are text-first with vision bolted on, Step 3 was built as a native multimodal reasoning model. It accepts both text and images and is meant to handle cross-modal problems: math expressed in a photo, a chart that needs interpretation, a real-world scene that has to be analysed.
The architecture is a mixture-of-experts (混合专家, MoE) system:
- 321 billion total parameters, about 38 billion active per token.
- 48 expert networks, with each token routed to three plus a shared expert.
- A dedicated visual encoder (around 5B parameters) paired with a 316B language model.
The real engineering bet: cheaper decoding
The detail that separates Step 3 from a generic big MoE is what StepFun calls model-system co-design. Two pieces matter:
- MFA (Multi-Matrix Factorization Attention). A custom attention mechanism the team says reduces memory and compute pressure during generation.
- AFD (Attention-FFN Disaggregation). A system-level split that lets attention and feed-forward layers be placed and served separately.
Together, StepFun reports, these cut decoding cost relative to DeepSeek-V3 — the company's own paper, "Step3: Large yet Affordable: Model-system Co-design for Cost-effective Decoding" (arXiv 2507.19427), frames the whole release around that claim. The motivation is practical: a model is only useful if you can serve it without a warehouse of the most expensive GPUs.
Built to run on domestic hardware
Step 3 was also engineered for the Chinese accelerator ecosystem. StepFun says it adapted the model to Huawei Ascend (昇腾) and MetaX (沐曦), among others, and formed a "Model-Chip Ecosystem Innovation Alliance" (模芯生态创新联盟) with close to ten chip vendors.
That is a strategic choice as much as a technical one. An open multimodal model that runs well on domestically produced silicon is more deployable inside China's enterprises, hospitals, and device makers, where procurement and supply constraints shape what can actually ship.
What the numbers say
From StepFun's own ModelScope card, Step 3 posts these scores among open-source vision-language models:
- MMMU: 74.2% — college-level multimodal understanding.
- MathVision: 64.8% — visual math reasoning.
- SimpleVQA: 62.2% and HallusionBench: 64.2% — visual QA and hallucination resistance.
Against proprietary vision models the gap is real but narrowing: o3 and Gemini 2.5 Pro score higher on MMMU in the same table, while Step 3 sits near the top of the open column. The honest read is "best-in-class open multimodal," not "beats everything closed."
How to actually run it
Step 3 is primarily a self-host open-weight release. The checkpoint lives on Hugging Face (stepfun-ai/step3) and ModelScope. StepFun's own deployment guidance suggests it can run in a quantized form on a cluster such as 8×48GB GPUs — still datacenter hardware, but far from a flagship-only requirement.
Inference is supported through vLLM and SGLang, and the model takes image-plus-text inputs for vision-language reasoning.
Honest limitations
- It is huge. 321B parameters must stay resident in memory even when only 38B are active, so single-consumer-GPU serving is not realistic; multi-GPU or datacenter deployment is the norm.
- Context is 64K tokens. That is shorter than the 128K–1M windows of several 2025 peers, which limits whole-document or very-long-video analysis in one pass.
- Vision input only. Step 3 reads images; it is not an audio or video-native model in its open-weight form.
- Benchmarks are self-reported. The MMMU and MathVision figures come from StepFun's card; independent third-party evaluation of the open checkpoint is still limited.
- Primarily for self-hosting. Unlike some rivals, Step 3 is not heavily pushed as a hosted API product, so most users will meet it as weights, not a chat box.
What readers can do now
- Pull the weights and test the multimodal claim. Grab stepfun-ai/step3 from Hugging Face or ModelScope and run a few image-reasoning prompts — math from a photo, chart interpretation, scene QA — to see where it helps.
- Serve it through vLLM or SGLang on a multi-GPU box. Use the quantization guidance to get onto 8×48GB-class hardware instead of attempting the full-precision checkpoint on limited memory.
- If you deploy in China, benchmark on Ascend. Because Step 3 was co-designed for domestic accelerators, a proper Huawei Ascend (昇腾) run is the fairest test of its "affordable decoding" promise — not just an NVIDIA baseline.