At twenty to two in the morning, the operations desk of a Hangzhou live-commerce company is lit by four monitors and one desk lamp. A host is still selling on screen. Behind her, a counter tracks unanswered buyer questions, and it is falling faster than any human team could make it fall.
The replies are written by a model. Tonight the team's job is not to write them but to watch latency, catch the occasional wrong size chart, and keep cost per conversation under the number on the whiteboard.
This is where Chinese compute is mostly consumed: not in a training run that ends, but in a service that never quite stops.
Key takeaways
- National token volume reached 140 trillion a day by March 2026, from about 100 billion in early 2024 (National Data Administration).
- Ascend 950PR is specified at 2 PFLOPS FP4 with 128 GB of memory and 2.0 TB/s interconnect, a memory-and-bandwidth profile.
- Huawei claims CloudMatrix 384 reaches about 300 PFLOPS BF16 per system, a scale-out figure rather than a per-chip one.
- China had 42 ten-thousand-card compute clusters in March 2026 (CAICT); capacity is expected above 24 GW by end-2026 (SemiAnalysis).
- Training remains the gap: US chips are estimated about 5× Huawei's best now, widening toward 17× by late 2027 (CFR).
Serving is a different business
A training run is synchronous, bursty and finite. Thousands of chips work in lockstep for weeks, one straggler stalls the group, and the output is a checkpoint. A serving fleet is asynchronous, continuous and measured in milliseconds. Requests arrive in bursts, the model answers a token at a time, and the output is revenue.
Training rewards interconnect bandwidth and numerical precision across a large cluster. Serving rewards memory capacity, memory bandwidth, low-precision throughput, and the ability to keep thousands of small jobs busy.
Four levers follow:
- Memory capacity per package, so weights and caches sit close to the arithmetic.
- Memory bandwidth, because generating each token reads the model again.
- Low precision such as FP8 and FP4, which raises throughput per watt when a workload tolerates it.
- Interconnect, because a serving cluster is only as fast as its slowest cross-node hop.
The demand side pushes the same way. The national average reached 140 trillion tokens a day by March 2026, against roughly 100 billion in early 2024 — a more than thousandfold rise (National Data Administration).
A training cluster that runs for a month is a cost centre. A serving cluster that runs for a year is a revenue line. A chip programme under constraints would optimise for the second.
What the silicon says
The 950PR is specified at 2 PFLOPS FP4 with 128 GB of proprietary HiBL 1.0 memory and 2.0 TB/s interconnect. The 950DT carries 144 GB of HiZQ 2.0 at roughly 4.0 TB/s. The 960 is scheduled for 2027.
Read as ratios rather than headline numbers, the pattern is consistent: a great deal of memory and interconnect per unit of arithmetic. FP4 support is the clearest tell. It is the format you adopt when you intend to push very large numbers of tokens through a model, and when a small numerical error is an acceptable price for throughput.
That does not make the parts weak; it makes them specific.
The system is the unit
Huawei's answer to per-chip deficits is to stop arguing about chips. The company claims CloudMatrix 384 reaches about 300 PFLOPS BF16 — a rack-scale figure, and a company claim rather than an independently verified benchmark.
The surrounding numbers make the strategy legible. China had 42 ten-thousand-card compute clusters as of March 2026 (CAICT). Data-centre capacity is expected to exceed 24 GW by the end of 2026, against roughly 56 GW in the United States (SemiAnalysis).
That is not the behaviour of companies chasing benchmarks; it is the behaviour of companies buying capacity they expect to sell by the token.
The economics of a token
A training run is a fixed bet. A serving fleet is a business with a marginal cost per request, set by utilisation.
Prices show the range: Qwen3.7-Max lists at US$2.5 and US$7.5 per million input and output tokens; DeepSeek V3.1 at RMB 2 and RMB 8 (≈ US$0.28 and US$1.10); Doubao Lite at RMB 0.3 and RMB 0.6 (CGTN). At the bottom of that range a provider is selling arithmetic near cost and hoping volume covers the fixed base.
Agentic workloads change the arithmetic: coding agents consume 10–50× the tokens of plain chat for the same user, because they loop, retry and read files.
Capacity is tight. In March 2026, Tencent Cloud raised prices on some model inputs by 463%, Alibaba Cloud raised AI compute and storage by up to 34%, and Baidu Cloud by 5–30%. When the cheapest strategy is to cut prices and the market raises them instead, the constraint is real.
The weakness
The hole in the strategy is the training frontier. The Council on Foreign Relations estimates that the best US chips are about 5× Huawei's best on total processing performance, widening to around 17× by the second half of 2027. In the most Huawei-favourable scenario the company produces about 5% of Nvidia's aggregate AI compute in 2026; the median estimate is near 1%.
Serving someone else's frontier model cheaply is a good business until the frontier moves somewhere your hardware cannot follow. Chinese labs have compensated with post-training, distillation and inference (推理) engineering, and those gains are real. But they are gains on a base someone else is also advancing.
Honest limitations
Token volumes are administrative figures, not audited revenue, and say nothing about how much traffic is paid. The CloudMatrix 384 figure is a company claim with no independent benchmark attached. Capacity estimates come from one research house working from incomplete disclosures. Nobody publishes utilisation rates for Chinese inference fleets.
What readers can do now
- Operators: price workloads per completed task rather than per token, and measure utilisation before adding capacity.
- Investors: treat token growth as a demand signal and utilisation as the profit signal; the two can diverge for a long time.
- Researchers: publish latency and throughput at batch sizes that resemble production serving, not single-stream benchmarks that flatter every accelerator.