HBM is stacked DRAM bonded over through-silicon vias and co-packaged with an accelerator on a silicon interposer, and it is now the single component that sets the bandwidth, capacity and cost ceiling of AI compute, with revenue rising from roughly $4B in 2023 to around $35B in 2025.
Summary
High-Bandwidth Memory is DRAM turned on its side. Instead of placing memory chips on a motherboard and talking to them over a narrow, fast serial bus, HBM stacks 8 to 16 DRAM dies vertically over a base die, connects them with through-silicon vias, and sits the whole stack next to the processor on a silicon interposer (in the dominant case, TSMC’s CoWoS). Bandwidth then comes from brute bus width rather than pin speed: HBM3E uses a 1,024-bit interface, HBM4 doubles that to 2,048-bit. Per-stack figures run from roughly 460 GB/s at 16GB for HBM2E, through about 1.2 TB/s at 24-36GB for HBM3E, to roughly 2.0-3.3 TB/s at 36-64GB for HBM4, with HBM4E at 3+ TB/s expected in 2027. Energy is about 3-4 pJ/bit.
The parameters that decide HBM are not really the DRAM cells. Three matter. First, packaging: HBM only exists as part of an advanced package, and CoWoS supply is widely argued to be the true gating constraint, with lead times of 50-104 weeks and Nvidia reported to have booked around 55% of 2026 capacity. Second, the base die: from HBM4 onward it becomes a logic die, which splits the industry along a foundry fault line, with SK Hynix and Micron using TSMC N12-class and Samsung using in-house SF4. Third, wafer intensity: 1GB of HBM consumes roughly 3-4 times the wafer capacity of 1GB of DDR5, so HBM growth cannibalises commodity DRAM supply, with AI absorbing about 20% of global DRAM wafer capacity in 2026.
HBM’s importance is best read from the papers that treat it as the thing to be worked around. LLM inference is described as bottlenecked by the capacity and bandwidth of GPU HBM; reasoning models with long output sequences and low arithmetic intensity make the memory wall worse; mixture-of-experts weights strain HBM capacity and cost efficiency; and billion-scale vector search cannot fit in HBM at all. A whole research literature now proposes to displace or supplement it: UCIe-attached on-package memory, high-bandwidth flash, monolithic-3D on-chip cache, photonic memory appliances and even optical delay-line “fiber memory”. None of it ships. HBM does.
Evidence base
- Per-stack bandwidth scales from ~460 GB/s (HBM2E, 16GB) to ~2.0-3.3 TB/s (HBM4, 36-64GB) and 3+ TB/s (HBM4E, 2027), driven by bus width doubling from 1,024-bit to 2,048-bit, at ~3-4 pJ/bit.
- HBM revenue path: ~$4B (2023), ~$17B (2024), ~$35B (2025 Micron TAM), ~$32.7B (2026 UBS), ~one third of DRAM value, ~$100B TAM by 2028; bit demand +130% in 2025, +70-77% in 2026, +68% in 2027.
- CoWoS is widely argued to be the gating constraint rather than the DRAM die, with 50-104 week lead times and Nvidia reported to have booked ~55% of 2026 capacity; from HBM4 the base die becomes a logic die split between TSMC N12-class (SK Hynix, Micron) and Samsung SF4.
- At ISSCC 2026 Samsung demonstrated HBM4 running well past JEDEC pin speeds, alongside LPDDR6 from Samsung and SK Hynix ref.
- HBM4E signal integrity at ECTC 2026: ~67% unit-interval eye at 12 Gb/s without receiver equalisation, ~72.5% with a one-tap DFE, and >60% at 12.8/14/16 Gb/s ref.
- A UCIe-based on-package memory proposal claims up to 10x bandwidth density, 3x lower latency, 3x lower power and lower cost than HBM4 and LPDDR on-package memory, using either a logic die in front of LPDDR6/HBM or DRAM that natively speaks UCIe.
- A photonic memory appliance integrating HBM3E, an on-module photonic switch and external DDR5 reports up to 32 TB shared memory, 115 Tbps all-to-all switching and simulated 3.66x throughput and 1.40x latency improvement in LLM inference, aimed at removing the silicon beachfront limit on memory-to-compute ratio.
Open questions
- Does the HBM4 logic base die decide market share? Specifically, does Samsung’s in-house SF4 base die yield and perform well enough against SK Hynix and Micron on TSMC N12-class to hold three-supplier parity?
- Which constraint actually binds through 2027: CoWoS interposer capacity and its 50-104 week lead times, or DRAM wafer capacity given HBM’s 3-4x wafer intensity versus DDR5?
- Can large-granularity memory (high-bandwidth flash) close the read-amplification gap in production, given it can degrade tiled GEMM by up to an order of magnitude at kilobyte access granularity?
- Does custom HBM fragment the JEDEC standard, and if so does that concentrate or dilute supplier pricing power ref?
Assessment drafted 2026-08-31 from up to 18 KB sources using the technology-scorecard framework; scores are a draft read pending review.