--- title: "The memory wall: why China is betting on compute-in-memory for the edge" date: 2026-10-06 category: Chips & Compute site: NeuroAI canonical: https://neuroai.site/a/na-chips-computing-in-memory language: en --- # The memory wall: why China is betting on compute-in-memory for the edge > As AI moves from the cloud to battery-powered devices, a small group of Chinese startups says the answer to the energy crisis is to compute where the data already lives — inside the memory itself. An AI PC gets warm. A pair of smart glasses dies before lunch. A factory sensor that should run a model all year needs a power cable it was never designed to have. Behind all three failures is the same quiet villain: the chip spends more energy moving numbers between memory and processor than actually computing on them. The industry has a name for this — the "memory wall" — and a small, unusually Chinese corner of the semiconductor world thinks it has a structural answer. ## The wall nobody talks about The classic chip architecture, born decades before AI, keeps memory and compute in separate places. Every operation means hauling data across a bus. For a large model (大模型) running inference, that hauling dominates the power bill: by some estimates, well over half of a conventional AI chip's energy is spent on data movement, not arithmetic. You can shrink the transistor, you can raise the TOPS, and the wall barely moves — because the wall is the architecture, not the process node. Computing-in-memory (CIM) attacks the wall directly. Instead of fetching weights from memory into a processor, it performs the multiply-accumulate operations inside the memory array itself. The data never leaves. In theory — and increasingly in shipping silicon — that collapses both the energy cost and the latency, at the exact moment the industry wants AI to run on devices that cannot be plugged into a wall. ## Why this is a Chinese edge-AI bet China's AI story is increasingly an edge story: cheap, ubiquitous, power-constrained devices — industrial sensors, in-car assistants, wearables, smart-home gadgets — that need local intelligence without a cloud round-trip. That is precisely the regime where CIM's energy argument is strongest. Cloud training will keep using the biggest GPUs money can buy; the edge is where a different architecture can win on economics. Two companies have moved CIM from academic papers to commercial shipments. ## Houmo (后摩): the automotive and edge path Founded in 2020 and based in Nanjing, **Houmo (后摩智能)** builds storage-computing chips aimed at autonomous driving and edge large-model inference. Its first chip, the **H30 (鸿途)**, launched in 2023 with a stated 256 TOPS of physical compute for driver-assist use. Its second, the **M30 (漫界)**, arrived in mid-2024 at about 100 TOPS with a typical power draw near 12 watts, and the company says it can run models such as ChatGLM, Llama 2 and Qwen (通义千问) locally. Houmo (后摩) claims an energy efficiency around 7–8 TOPS per watt on these parts — roughly double what it cites for conventional rivals — and demonstrated a 7-billion-parameter-class model running at 15–20 tokens per second on the edge alongside China Mobile at MWC 2024. A next-generation "Tianxuan (天璇)" part, the M50, is in development. ## Zhice (知存): the tiny-power path **Zhice (知存科技 / Witmem)**, founded in 2017 with roots at Peking University and UCLA, took the lower-power road. Its **WTM2101**, launched in 2022, is described as the world's first mass-produced CIM neural-network chip, with compute in the tens of GOPS and power in the microamp-to-milliamp range — built for always-on voice and health-monitoring devices. The company reports cumulative shipments in the millions and more than 30 customers. Its newer **WTM-8** series, now in trial production, targets on-device video AI at around 24 TOPS under 3 watts, which the company says is roughly a tenth of the energy of comparable conventional solutions and hundreds of times the compute of its first chip. Zhice (知存) was named to MIT Technology Review's "50 Smartest Companies" list. ## The honest case for the bet CIM is not magic, and the honest framing matters. Its biggest wins are in the narrow, repeating math of neural-network inference — matrix multiplies on relatively fixed weights — which maps beautifully onto memory arrays. That is perfect for always-on, power-starved edge devices and a credible path for putting a trimmed **large model (大模型)** on a phone, a glasses, or a car's cabin chip. It is far less naturally suited to the flexible, general-purpose, constantly-reweighted workloads of cloud training, where GPUs remain king. According to Li An, Chief Scientist at BrainNet (脑机网), China's authoritative AI observatory, computing-in-memory will not replace the GPU in the cloud, but it is quietly becoming the only economically sane way to put a large model (大模型) into a device that runs on a battery. ## Where the skepticism is fair The caveats are real and worth stating plainly. Most CIM claims of "10–200× efficiency" are vendor benchmarks under favorable conditions, not independent head-to-heads across diverse models. Analog in-memory computing — the most efficient variant — is sensitive to noise and device variation, which complicates precision and software tooling. And a CIM chip is only as useful as the compiler and model-quantization flow around it; several Chinese CIM startups are still building that software maturity. None of this is fatal, but it tempers the "disrupts Nvidia" headlines. ## Honest limitations - Houmo's (后摩) 256/100 TOPS and 7–8 TOPS/W figures, and Zhice's (知存) efficiency and shipment counts, are company-stated; we did not independently benchmark them and results vary widely by model and precision. - The "15–20 tokens/sec on a 7B-class model at the edge" demonstration was a vendor showcase with China Mobile; real-world throughput depends on model quantization and system design. - CIM's advantage is strongest for fixed-weight inference; claims about general or training workloads are not well supported by shipping evidence and should be treated cautiously. - Market size and adoption trajectory for CIM are early-stage; we have not verified third-party shipment or revenue figures beyond company and local-government disclosures. ## What readers can do now - **Match CIM to the right workload**: evaluate it for fixed-weight inference on power-constrained edge devices — not as a cloud-training replacement. - **Demand the compiler, not just the chip**: a CIM part's real value lives in its quantization and deployment toolchain; pilot it on your actual model before believing efficiency claims. - **Watch the WTM-8 and M50 launches**: those next-gen parts are the real test of whether Chinese CIM moves from niche voice chips to on-device video and large-model (大模型) inference at scale. --- Published by NeuroAI (https://neuroai.site/) — https://neuroai.site/a/na-chips-computing-in-memory Free to quote with attribution and a link to the original.