Von Neumann Bottleneck

Cross-cuts: Memory
last updated 2026-06-16
The Memory WallIn-Memory ComputingProcessing-in-Memory (PIM)Charge-Domain ComputeVon Neumann…

The classic computer keeps memory and processing in separate places, so instructions and data must shuttle back and forth across a bus. When the work is data-heavy and arithmetic-light, that shuttling, not the arithmetic, sets the cost.

Why it dominates AI inference

Neural-network inference is mostly matrix-vector multiplication: stream a large weight set from memory, do one cheap multiply-accumulate per weight, repeat. The arithmetic is nearly free; fetching the operands is not. The canonical figures (Horowitz, “Computing’s Energy Problem”, ISSCC 2014): a floating-point operation costs on the order of a picojoule (~0.4-3.7 pJ depending on type/precision), while an off-chip DRAM access costs ~1.3-2.6 nJ, i.e. hundreds to ~1,000x more energy to fetch the value than to compute with it. So the energy and latency of inference are set by data movement, not FLOPS. The bandwidth-shaped sibling, quantified, is the The Memory Wall (compute +60,000x vs DRAM bandwidth +100x over 20 years, 2024 Gholami Ai And Memory Wall).

Why it matters here

This is the root justification for the whole in-memory and near-memory family: if moving the data is the cost, stop moving it. In-Memory Computing and Charge-Domain Compute fuse the multiply into the memory array; Processing-in-Memory (PIM) puts logic next to the memory banks; Near-Memory Compute stacks memory on the compute die. The bandwidth-shaped sibling of this problem is the The Memory Wall.

Connected ideas

Related concepts

Frontier questions