NeuroAI NEUROAINEUROAI.SITE
ESC

1,000 Domestic Chips Carried a 1.6-Trillion-Parameter Model Through Training

A Shenzhen consortium including Huawei ran full-parameter post-training for DeepSeek's 1.6-trillion-parameter V4-Pro model on Ascend 910C chips, moving domestic compute from inference into training.

2026-11-05 · 994 words · NeuroAI
1,000 Domestic Chips Carried a 1.6-Trillion-Parameter Model Through Training

There is a quiet dependency hiding inside every impressive model demo. The model answers in seconds, but its intelligence was installed months earlier in data centres full of accelerators that, for Chinese labs, mostly had to come from abroad. Domestic chips could serve the finished product; teaching the product new skills was a different story.

That story changed in mid-2026. A research consortium led by Shenzhen Hetao College (深圳河套学院) — working with Harbin Institute of Technology's Shenzhen campus, the Shenzhen Research Institute of Big Data, Huawei and the Shenzhen smart-city computing platform — completed full-parameter post-training (全参数后训练) of DeepSeek-V4-Pro, a 1.6-trillion-parameter mixture-of-experts (混合专家) large model (大模型) and the largest DeepSeek has released, on a cluster of more than 1,000 Huawei Ascend 910C chips. The result was announced by the Shenzhen municipal government and covered by the South China Morning Post, and it marks the first time, according to the consortium, that a third-party team has run full-parameter post-training for this model on domestic Chinese compute.

Key takeaways

  • The consortium completed full-parameter post-training — not a lightweight fine-tune — of the 1.6T-parameter DeepSeek-V4-Pro on Ascend 910C hardware.
  • The run stayed stable for more than 1,500 steps at roughly 27 seconds per step with zero interruptions, reaching over 30% model FLOPs utilisation, a vendor-reported efficiency metric.
  • The project took about a month and treated the training run as a talent programme: 42 students worked on it directly.
  • The strategic shift is from inference to training: domestic chips can now sustain the most communication-heavy phase of the model lifecycle.

Why post-training is the hard part

The distinction between running a model and training one is easiest to see through the hardware's eyes. Inference is a one-way street: a prompt goes in, an answer comes out. Post-training is closer to building a multi-level interchange on top of that road — the model must reflect on its own outputs, and every parameter gets updated rather than a thin adapter layer.

For a mixture-of-experts architecture, the communication bill explodes. During inference only a few experts activate per token, but during training all of them must learn at once, forcing massive all-to-all data exchange across the cluster — by the consortium's own account, dozens of times the volume of a comparable dense model. Interconnect bandwidth, memory placement and scheduling become the bottleneck, which is precisely where domestic accelerators were least proven.

The team reported three engineering breakthroughs. First, a distributed "memory puzzle" solution that shards 1.6 trillion parameters across every card in the cluster with precisely orchestrated exchange timing. Second, a load-balancing scheduler tuned for MoE traffic so that no expert idles while another saturates. Third, full-chain monitoring with automated recovery — the run lasted more than 1,500 steps at about 27 seconds per step without a single interruption or error, over roughly a month of continuous operation. The consortium also reported model FLOPs utilisation above 30% and a 14% improvement in key training-operator efficiency. For context, utilisation in the 30% range is considered solid industrial-grade results even on top-tier overseas chips, where leading teams reach roughly 40%.

Why the rest of the world should care

Three separate audiences have skin in this game.

For anyone tracking the semiconductor standoff, this is evidence that export controls have not frozen Chinese AI at the inference stage. A trillion-parameter model updating its full weights on homegrown silicon closes the most-watched gap in the domestic compute story. It does not close the efficiency gap — 30% versus roughly 40% utilisation still means paying more for the same training — but "can it be done at all" was the question that mattered most, and the answer is now yes.

For cloud buyers and CIOs, the practical consequence is optionality. Post-training is what turns a general checkpoint into a model that knows your domain, your formats and your safety rules. If that phase can run on domestically controlled hardware, organisations that cannot ship data to overseas providers gain a path to customisation they previously lacked. DeepSeek-V4-Pro had already been deployed for inference on domestic chips; training was the missing half.

For the research community, the consortium published the failure modes that matter: MoE load balancing, memory sharding at trillion scale, and long-run stability engineering. Those are reusable lessons, and the project doubles as a training ground — 42 students from Shenzhen Hetao College worked directly on the production run, an unusual apprenticeship at this scale.

The trend line extends beyond one model. According to Li An, chief scientist at BrainNet (脑机网), the meaningful unit of competition is shifting from single chips to full training stacks — schedulers, interconnects, frameworks and operators — and this project is one of the first public proofs that China's stack can carry frontier-scale training workloads, not just serve them.

Honest limitations

  • The headline metrics — 1,500+ stable steps, over 30% utilisation, 14% operator improvement — come from the consortium's own announcements and have not been independently audited.
  • This was post-training (continued pretraining and supervised fine-tuning), not pretraining from scratch; the pretraining frontier on domestic chips remains untested in public.
  • Utilisation still trails the roughly 40% that leading teams report on top-tier overseas accelerators, so cost per unit of training remains higher.
  • The run demonstrates capability on one model architecture; generalisation to other trillion-parameter models is plausible but unproven.
  • No downstream benchmark comparison was published showing how the post-trained model performs relative to its pre-run state, so the quality of the update itself is not quantified.

What readers can do now

  • If you evaluate infrastructure for a large organisation, ask vendors of domestic-accelerator clouds specifically for full-parameter post-training references, not just inference SLAs — this run shows the distinction now matters.
  • If you study systems for distributed training, the consortium's three published bottlenecks (memory sharding, MoE load balancing, long-run monitoring) are a concrete checklist for what to benchmark.
  • If you track the chip story, watch for pretraining runs on domestic hardware next; that is the remaining frontier this result stops short of.

Related coverage

Cite this article

NeuroAI (脑机网), "1,000 Domestic Chips Carried a 1.6-Trillion-Parameter Model Through Training," 5 November 2026, https://neuroai.site/a/na-model-deepseek-v4pro-ascend-training

Machine-readable: BibTeX · RIS · JSON · Markdown

More in “Foundation Models” → · Back to home · Markdown version