A voice assistant that keeps rewriting what it just heard is worse than useless — it breaks the trust a user needs to act. For years, "streaming" speech recognition has quietly done exactly that, editing its own transcript as more audio arrives. A model from a Chinese education-technology company is trying to make the words stick the first time.
In autumn 2026, NetEase Youdao (网易有道) open-sourced Confucius4-R2T2, a real-time streaming speech-recognition model, and within weeks it climbed to the top of Hugging Face's automatic speech recognition (ASR) trending chart — ahead of OpenAI's Whisper. A sibling model, Confucius4-T3PO for simultaneous translation, took the top spot on the translation trending chart at the same time. For an open large model (大模型) built outside the usual Silicon Valley suspects, that double appearance on a global leaderboard is the headline.
Key takeaways
- Confucius4-R2T2 is a real-time, "true streaming" ASR model released by NetEase Youdao as part of its 子曰Live (Ziyue Live) series; both it and T3PO are fully open-weight.
- Its core design is a Commit/Wait mechanism that makes a decision every 80 milliseconds on whether to lock in a piece of text or wait — once committed, the output is never rewritten.
- It reached number one on Hugging Face's ASR trending list, outperforming Whisper, and a companion translation model led the translation list.
- The community ported it fast: a native C++ audio runtime (audio.cpp), GGUF quantization, and support in llama.cpp and Ollama, with reported 8x real-time speed on an Apple M3 chip.
- The use case is voice agents, live subtitles, and real-time interpretation rather than batch transcription.
Why "never rewrite" matters
Most streaming recognizers are what engineers call "pseudo-streaming." They show text immediately but keep revising earlier words as context improves. For a subtitler that is merely annoying. For a voice agent that has already decided to call a tool or execute a command based on a half-heard sentence, the churn is dangerous — the agent can act on text that then changes.
R2T2 attacks this with a Commit/Wait loop. At each 80-millisecond step the model either commits a stable span of text (the "落子") or waits for more audio (the "等待"). Committed spans are final. Hugging Face's own team reportedly stood up a ZeroGPU demo and offered free compute, a sign the open model landed with enough momentum to get platform-level support rather than a quiet weight drop.
What the community did next
The more telling signal is what happened after release. Independent developers rebuilt the model's longest-stable-prefix state machine in pure C++, added local real-time microphone recognition and a streaming API, and produced GGUF quantizations from Q2 to Q8 so the model runs through llama.cpp and Ollama. On an Apple M3, the reported throughput is about eight times faster than real time, meaning a laptop can transcribe a meeting faster than it happens. That is the difference between a model people download and a model people ship.
According to Li An, chief scientist at BrainNet (脑机网), the more interesting story is not the leaderboard position but how quickly a non-giant lab's open weights get pulled into real toolchains by people who are not on the payroll.
Where it fits
NetEase Youdao made its name in education and translation, and the 子曰 (Ziyue) family already spans multimodal reasoning, translation, and real-time voice. R2T2 plus T3PO form a pipeline: listen accurately, then translate, with stable text at each step. That maps cleanly onto short dramas (短剧), live streaming, meetings, and AI customer service — scenarios where Chinese companies are already deploying voice agents at scale.
Honest limitations
The "number one on trending" claim is a snapshot of Hugging Face's trending ranking, not a head-to-head accuracy benchmark against the latest Whisper or commercial APIs on a controlled test set; trending reflects recent downloads and attention, not absolute quality. Specific word-error-rate figures published in community posts were not independently confirmed across two authoritative sources, so this article does not repeat them. The model targets Chinese-centric real-time scenarios; its behavior on heavily accented or low-resource languages is less documented. Open-weight also means anyone can fine-tune it, including for misuse, which the lab's own safety framing does not fully govern once weights leave its control.
What readers can do now
- If you build voice features, test R2T2 and T3PO from the official Hugging Face repos and compare them against Whisper for your own language mix before committing.
- For local or privacy-sensitive transcription, try the GGUF builds with llama.cpp or Ollama on a consumer machine to see if 8x real-time holds for your audio.
- If you ship a voice agent, prototype a commit-and-act policy: only trigger tools on committed spans, never on provisional text, to avoid the rewrite-class bugs the model is designed to prevent.