Physics / mechanism
Conventional compute separates memory and logic, forcing data across the von Neumann bottleneck—dominant cost in AI inference workloads. In-memory computing (IMC) embeds arithmetic directly in the memory array, typically using analog crossbar structures where Ohm’s law and Kirchhoff’s current summation perform matrix-vector multiplication (MVM) in O(1) time per layer. Key device substrates: SRAM-based digital IMC (Samsung, TSMC N5/N3), and analog non-volatile variants using PCM, RRAM, or OTS-based selectors. SRAM IMC achieves ~10–50 TOPS/W at 28–5nm; analog NVM claims 100+ TOPS/W but suffers weight precision limits (~4–6 bit effective), conductance drift, and write endurance constraints. Closest to productization: Mythic (RRAM), Syntiant (SRAM), Untether AI (SRAM near-memory).
Competitive landscape
SRAM digital IMC is conservative but foundry-compatible and production-ready. Analog NVM IMC promises highest efficiency but requires custom fab and error-correction overhead. Near-memory compute (HBM + logic die, e.g., SK Hynix AiMX) is a softer alternative—no array-level compute but cuts bandwidth cost. PIM (Processing-in-Memory) from Samsung/SK overlaps significantly. Neuromorphic (Intel Loihi, IBM NorthPole) shares the co-location principle but targets spiking networks.
| Approach | TOPS/W | Precision | Foundry risk |
|---|---|---|---|
| SRAM IMC | 10–50 | 8-bit native | Low |
| Analog NVM IMC | 100+ | 4–6 bit effective | High |
| Near-memory (HBM+PIM) | 5–20 | Full | Low |
Investment relevance (all routes)
In-memory compute attacks the The Memory Wall and Von Neumann Bottleneck directly; the open question is who captures the value, and through which vehicle.
Companies using
Connected ideas
Sources
Frontier (open questions)
- Does digital IMC plus 3D stacking close the energy gap before analog CIM scales out of the edge?
- Which analog-CIM device family (SRAM-analog, RRAM, flash, memcapacitor) reaches volume first?