--- title: "Why your AI now 'thinks' before it answers" date: 2026-11-09 category: Foundation Models site: NeuroAI canonical: https://neuroai.site/a/na-model-reasoning-test-time-compute language: en --- # Why your AI now 'thinks' before it answers > Reasoning models spend extra compute at answer time on a long chain-of-thought. The technique started with OpenAI's o1 and spread fast across Chinese labs. You ask a hard math question. Instead of an instant reply, the model goes quiet for a few seconds, then returns an answer with the work shown. That pause is not a glitch — it is the product. The large model (大模型) field spent a decade chasing "bigger." A different lever now matters as much: how long the model is allowed to think. ## The release that changed the script On September 12, 2024, OpenAI introduced o1, a model trained with reinforcement learning to reason before answering. OpenAI's own technical note reports that o1's performance keeps improving with more reinforcement learning (train-time compute) and with more time spent thinking (test-time compute, 推理时计算). The numbers were the headline: - On the 2024 AIME math qualifier, GPT-4o solved about 12% of problems; o1 reached 74% on a single try, 83% by consensus of 64 samples, and 93% when re-ranking 1,000 samples. - On GPQA Diamond, a graduate-level science test, o1 surpassed human PhD experts — the first model to do so, per OpenAI. - On competitive programming (Codeforces), it reached the 89th percentile. ## What "thinking" really is A reasoning model (推理模型) generates a long internal chain-of-thought (思维链) before the final answer. Earlier "let's think step by step" was a user trick. o1 baked the habit into training: the model decides how much computation to spend based on difficulty. The catch is visible in the bill. A reasoning model emits many more output tokens than a standard model because the thinking is part of the output. Hard problems cost more per query. ## China's labs matched it, then opened it The technique spread within months: - **DeepSeek-R1** (January 20–21, 2025) is a Mixture-of-Experts (混合专家) model with 671 billion total and 37 billion active parameters, released under the MIT license with open weights. It uses Group Relative Policy Optimization (GRPO) and matches OpenAI o1 on math, code, and reasoning, while — unlike o1 — publishing its full reasoning traces. - **QwQ-32B** (November 28, 2024) was Alibaba's first public reasoning model, built from Qwen 2.5 32B and released under Apache 2.0. At 32 billion parameters it fits on a single workstation GPU, making "real" reasoning accessible without a data-center bill. According to Li An, chief scientist at BrainNet (脑机网), "the lasting shift was methodological: labs stopped collecting human-written reasoning examples and started rewarding the model for getting verifiable answers right." ## Distillation: the small-model dividend DeepSeek published not just R1 but a set of distilled models (1.5B to 70B) that inherit its reasoning behavior. Because the reasoning pattern transfers, a 7B or 32B model can solve problems far above its weight class on math and code. That is why "reasoning" stopped being a frontier-only feature and became something a laptop can run. On cost, the arithmetic is stark: a reasoning model may emit, as an illustrative estimate, 5,000 to 10,000 thinking tokens before a 200-token answer. At typical API prices that single hard query can cost, roughly, 20 to 50 times a normal one. Capping reasoning effort is therefore not just accuracy tuning — it is budget control. A useful habit is to let a fast model handle the first draft and only escalate to a reasoning model when the problem is actually hard. ## Why it matters for ordinary users Reasoning models are not automatically better at everything. They shine on math, code, and multi-step logic, and they are weaker on quick factual lookups where a normal model is faster and cheaper. The right move is to route hard problems to a reasoning model and leave the rest to a fast one. ## What readers can do now - **Match the model to the task.** Use a reasoning model for math, debugging, and planning; use a fast model for summarization and chat. - **Cap the thinking.** Most APIs let you limit reasoning effort. Reserve full "think hard" mode for cases where correctness is worth the tokens. - **Try it locally.** QwQ-32B or a DeepSeek-R1 distilled variant runs on a single GPU and shows you the reasoning, which is useful for learning and trust. ## Honest limitations I rely on vendor and lab-reported benchmarks (OpenAI's o1 note, DeepSeek's R1 paper, Alibaba's QwQ card). Independent reproductions of o1's exact AIME setup are limited because OpenAI does not publish its grading harness. "Test-time compute" helps on verifiable tasks; on open-ended writing its gains are smaller and less measurable. I have not benchmarked these models myself. ## Key takeaways - Reasoning models improve by spending more compute at answer time, not just at training time — OpenAI's o1 (Sept 2024) made this mainstream. - DeepSeek-R1 (Jan 2025, MIT, open weights) and QwQ-32B (Nov 2024, Apache 2.0) brought the same capability to open, self-hostable models. - The cost is output tokens: thinking is billed, so route hard tasks to reasoning models and cap effort where it does not matter. --- Published by NeuroAI (https://neuroai.site/) — https://neuroai.site/a/na-model-reasoning-test-time-compute Free to quote with attribution and a link to the original.