Theses›Datacentre & AI compute›Retrieval and memory tier

Retrieval and memory tier

Storage value migrates to the retrieval, index and memory tier, off the commodity bytes

  • Bear
  • Closing
  • $8.1B by 2035

The callMemory that computes is the right answer for agent retrieval at scale, but today's bottlenecks are capacity and tool execution, not retrieval: right market, too early.

D1D2R1R2R3C1foragainst D1D2R1R2R3C1foragainst
Fig. 1The balance. Drivers and catalysts against restraints, each block sized by its weight; catalysts hang dashed because they have not happened yet, sized by their odds. It tips toward the restraints.
  1. D1As agents scale against large live data stores, retrieval moves onto the critical path of every step: tool execution and retrieval are already 65-89% of agent latency while the GPU idles.
  2. D2Merchant memory sockets form when incumbents need the part: XCENA's chip ships inside SK hynix's CXL memory product, the Astera route.
  3. R1Too early: the bottlenecks measured in production today are capacity and tool execution, and retrieval latency comes later.
  4. R2Incumbents are closing the KV-cache half with capacity and software: NVIDIA's BlueField-4 context-memory storage ships in H2 2026 and Samsung's CXL memory pool ran at 92% of DRAM.
  5. R3Memory-silicon exits in 2025-26 went for $0.1-0.9B, small for venture outcomes, and UPMEM sold to Qualcomm after a decade.
  6. C1A production deployment of computing memory publishes its economics, proving the socket pays.end-2027
Rests onthe beliefs underneath
Supported by · confidence
What would prove itdated tests, open and resolved
202720282029today1234predictionswatch 202720282029today1234predictionswatch
Fig. 2Each ring is a dated test: dashed while open, struck through when overdue, ticked or crossed once resolved.
  1. 1By end-2027 at least one major agent platform ships a durable, queryable, metered persistent-memory product tier, evidence that agent memory becomes a retained substrate rather than ephemeral context.Dec 2027 · open
  2. 2Atlas's equal-BOM benchmark at 10x or better (floor 3x) against a tuned CPU and graph database.Dec 2027 · open
  3. 3Through 2028 the data/retrieval software layer grows materially faster than the storage-medium makers, confirming value migrates off the bytes toward the layer that makes them usable to agents.Dec 2028 · open
  4. 4The retrieval/memory layer's value is captured predominantly by SOFTWARE on commodity memory, not by dedicated retrieval silicon, through 2028.Dec 2028 · open
  • ·NVIDIA's context tier or software KV-offload handling multi-hop graph retrieval; that would end the case.
Expressionslisted, private, technologies
Private · 10 private companies, 5 of them in the US.

Engram · Deepset (Deepset GmbH) · turbopuffer · Pinecone · Zilliz · Weaviate · Qdrant · Chroma · Mem0 · deepset

1 more private company tracked, not shown

Technologies
Who has already bet on itprivate rounds, from public sources

$554M raised by 9 companies, $148M of it in the last three years. Wave under way: Large rounds in the last three years; the category is funded and getting crowded.

Backed by a top-tier venture firm: Pinecone, Zilliz, Engram, Weaviate, Qdrant, deepset, Mem0, turbopuffer. Backed by a strategic investor: Zilliz, Qdrant.

1 more funded company in the market.

What changedthe record, newest first
  • 4 Sep 2026When two funds independently ask you the same two questions about a bet, those questions are the bet. The useful signal is not that they are sceptical - it is that nobody has yet arrived at your answer, which is what an edge looks like from the inside and what an unfundable round looks like from the outside.
  • 24 Jun 2026A software 'memory layer' startup raised ~$100M to cut AI token costs with claimed 100x fewer tokens. Reads as confirmation that value is migrating onto the retrieval/memory layer, and a warning that software on commodity hardware may capture it before any specialised chip does.
Reading
Show the reading