--- title: "InternLM3 and the 'thinking density' bet from Shanghai AI Lab" date: 2026-11-06 category: Foundation Models site: NeuroAI canonical: https://neuroai.site/a/na-model-internlm3-thinking-density language: en --- # InternLM3 and the 'thinking density' bet from Shanghai AI Lab > InternLM (书生·浦语), from Shanghai AI Laboratory (上海人工智能实验室), released InternLM3-8B-Instruct on 15 Jan 2025, trained on 4T tokens with a dual fast/slow thinking mode. A data-efficiency story. Most labs answer the cost problem by making models bigger. Shanghai AI Laboratory (上海人工智能实验室) answered it by making the data cleaner. The result, InternLM3-8B-Instruct, is a small model that behaves like two models at once. Released in January 2025, it is less a headline grabber than a quiet argument about how much compute a useful large model (大模型) actually needs. ## Key takeaways - InternLM3-8B-Instruct, from the InternLM (书生·浦语) series, was released on 15 January 2025 with 8 billion parameters. - The lab states it was trained on about 4 trillion (4T) tokens and that this cut training cost by more than 75% versus comparable-scale models. - It is the first open general dialogue model to merge a "deep thinking" long chain-of-thought mode with a normal response mode inside one model. - It is also described as the first open-source general model to support browser use, performing more than 20 web-navigation steps. - On the official model card (evaluated with OpenCompass), it posts CMMLU 83.1, MMLU 76.6, GPQA-Diamond 37.4, MATH-500 83.0 and AIME2024 20.0. ## The "thinking density" thesis The official framing is "thinking density" (思维密度): rather than cramming more parameters, the lab refined its data pipeline so each token taught the model more. According to the release materials, InternLM3 was trained on roughly 4T tokens, far fewer than the 10T-plus corpora common for 8B-class models, and the lab claims this reduced training cost by over 75% relative to peers of similar scale. That is a vendor claim about internal accounting, so treat the exact percentage as indicative. But the direction is credible: data curation, not raw scale, is where much of the recent efficiency gain in open models has come from. If a lab can reach similar benchmark territory on a third of the tokens, the marginal dollar moves from pretraining to data engineering. ## Two modes, one weights file The more interesting design choice is the dual reasoning mode. InternLM3 supports a "deep thinking" mode that uses long chain-of-thought to work through hard problems, and a normal mode for fluent everyday chat. The lab describes this as the first time an open general model fused the two in a single model rather than shipping a separate reasoning variant. For developers, the practical upside is simple: one deployment serves both fast conversational traffic and slow, deliberate reasoning, switched by a system instruction rather than by hosting two models. That halves serving complexity for teams that need both. ## Browser use as a built-in capability InternLM3 is also positioned as the first open-source general dialogue model with native browser-use support, able to chain more than 20 sequential web-navigation steps. This points at a trend beyond chat: models that act, fetching and reasoning over live web pages, rather than answering from a frozen snapshot. The claim is vendor-stated; whether 20 robust steps hold up in messy real websites is something only deployment will confirm. ## The benchmark picture Per the official model card, evaluated with the open-source OpenCompass framework, InternLM3-8B-Instruct scores: - CMMLU (Chinese multilingual understanding): 83.1 - MMLU (0-shot): 76.6 - GPQA-Diamond (0-shot): 37.4 - MATH-500 (0-shot): 83.0 - AIME2024 (0-shot): 20.0 These numbers are sourced from the model card and have not been independently re-run here. They place the 8B model competitively against Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct on several axes, and the MATH-500 and CMMLU results are the strongest evidence for the data-efficiency thesis. AIME2024 at 20.0 in thinking mode is modest and shows the ceiling for a small model on hard competition math. ## Where it fits InternLM is a research-led series from Shanghai AI Laboratory (上海人工智能实验室), a public research institution, not a commercial front. That lineage matters: the releases tend to emphasize reproducibility, open evaluation, and tooling (the lab also produces the OpenCompass benchmark suite and the LMDeploy serving toolkit). For a buyer, that means the model comes with a transparent evaluation chain rather than a black-box marketing sheet. ## Honest limitations - The "over 75% training cost reduction" and "4T tokens" figures are vendor-stated and relate to internal cost accounting; they are not independently auditable from outside. - Benchmark scores are taken from the official model card (OpenCompass); we did not re-run them, and OpenCompass versions can shift results slightly. - At 8B parameters, the model's ceiling on the hardest reasoning and math tasks (AIME2024 20.0) is well below frontier models; it is a small, efficient model, not a frontier one. - Browser-use over 20 steps is a controlled-capability claim; real-world robustness on arbitrary sites is unverified here. ## What readers can do now - If you need a small, cheap, self-hostable model for Chinese and English reasoning, benchmark InternLM3-8B-Instruct against Qwen2.5-7B on your own tasks before choosing. - Use the deep-thinking toggle deliberately: route hard queries to thinking mode and keep casual traffic in normal mode to save compute. - Pull the model via the official repository and run OpenCompass yourself if you want the evaluation to be reproducible rather than vendor-reported. --- Published by NeuroAI (https://neuroai.site/) — https://neuroai.site/a/na-model-internlm3-thinking-density Free to quote with attribution and a link to the original.