Theses›Datacentre & AI compute›Fog computing middleware

Fog computing middleware

As compute fragments across GPUs, custom ASICs, photonic, neuromorphic and probabilistic substrates, a middleware layer forms between applications and silicon.

  • Bear
  • Closing
  • $187B by 2035

The callThe layer that runs any AI workload on any chip gets bought by the chipmaker every time; absorption is now the consensus and the survivors are priced at $13-15B.

D1D2R1R2R3C1foragainst D1D2R1R2R3C1foragainst
Fig. 1The balance. Drivers and catalysts against restraints, each block sized by its weight; catalysts hang dashed because they have not happened yet, sized by their odds. It tips toward the restraints.
  1. D1AI shifts to inference and agents, so workloads spread across GPUs, custom ASICs, photonic and analogue chips, clouds, neoclouds and edge boxes, each needing its own compiler and kernels.
  2. D2Chipmakers and hyperscalers keep paying for the layer: Pensando $1.9B, Modular $3.9B, Groq about $20B.
  3. R1Absorption by the silicon vendors is now the consensus, and take rates are going to zero with free gateways from Cloudflare, Vercel and NVIDIA's Dynamo.
  4. R2The survivors are already priced: Baseten at $13B, Fireworks at about $15B, Inferact's $150M seed at $800M, Callosum's $100M seed at about $400M.
  5. R3The two biggest non-NVIDIA fleets, Google's TPUs and Amazon's Trainium, stay captive, so the mixed fleet an independent can reach is small.
  6. C1No catalyst event found. The nearest dated call: another compiler, portability or routing company bought by a chipmaker or hyperscaler above $1B.end-2028
Rests onthe beliefs underneath
Supported by · confidence
Threatened by · confidence
What would prove itdated tests, open and resolved
20272028202920302031today12345678910predictionswatch 20272028202920302031today12345678910predictionswatch
Fig. 2Each ring is a dated test: dashed while open, struck through when overdue, ticked or crossed once resolved.
  1. 1A heterogeneous-compute routing or AI-compiler middleware company is acquired, or raises $100M or more, between 6 Nov 2026 and 31 Jan 2027.Jan 2027 · open
  2. 2A hardware-routing design win at GTC, and routing becoming a named layer in hyperscaler stacks.Dec 2027 · open
  3. 3By 2028 heterogeneous-compute routing is a named layer in hyperscaler or enterprise AI stacks (not just in-house glue).Dec 2028 · open
  4. 4Absorption at rising prices continues: by end-2028 at least one more independent compiler / hardware-portability / heterogeneous-routing company is acquired by a silicon vendor or hyperscaler at >$1B.Dec 2028 · open
  5. 5The neutral model-router survives bundling: through end-2028 OpenRouter remains independent AND its weekly token volume does not decline for 2 consecutive quarters attributable to hyperscaler-bundled routers.Dec 2028 · open
  6. 6Merchant heterogeneity stays niche: by end-2028 non-hyperscaler-captive, non-NVIDIA-rack accelerators remain <15% of merchant/multi-tenant inference capacity by revenue, so the addressable surface for independent hardware-routing stays narrow.Dec 2028 · open
  7. 7Another $1B+ absorption by a chipmaker or hyperscaler; none, while independents keep growing, would reopen this.Dec 2028 · open
  8. 8Merchant non-NVIDIA accelerators passing 15% of multi-tenant inference capacity on an AWS external-Trainium ramp.Dec 2028 · open
  9. 9By 2030 a workload-routing / compute-middleware company reaches a >$1B outcome on owning the chips-to-applications layer.Dec 2030 · open
  10. 10The market is large enough that BOTH the software middleware AND a dedicated control-plane/agent ASIC reach venture-scale outcomes (not mutually exclusive).Dec 2030 · open
Expressionslisted, private, technologies
  • NVDANVIDIA
    • cheap
     Dominant share of AI training compute.
  • AMZNAmazon Web Services
    • fair
     Sells managed model inference across hundreds of models, with in-catalogue prompt routing, to enterprises (Amazon subsidiary; parent ticker).
  • MSFTMicrosoft
    • full
     Sells managed model inference and a model router across its Foundry catalogue to enterprises.
  • GOOGLGoogle Cloud
    • cheap
     Sells managed model serving for Gemini and third-party models to enterprises (Alphabet subsidiary; parent ticker).
  • QCOMQualcomm
    • not screened
     Sells the Modular inference stack (MAX serving, Mojo kernels) it bought in Jul 2026, alongside its own AI chips.
Private · 11 private companies, 8 of them in the US.

Callosum · Modular · Gimlet Labs · AlphaTango · ZML · FlexAI · OpenRouter · Baseten · Fireworks AI · Together AI · Inferact

1 more private company tracked, not shown

Technologies
Who has already bet on itprivate rounds, from public sources

$4.4B raised by 10 companies, $3.6B of it in the last three years. Wave under way: Large rounds in the last three years; the category is funded and getting crowded.

Backed by a top-tier venture firm: Together AI, Gimlet Labs, Modular, Fireworks AI, Inferact, OpenRouter, Callosum, FlexAI, ZML. Backed by a strategic investor: Together AI, Gimlet Labs, Fireworks AI, OpenRouter.

2 more funded companies in the market.

What changedthe record, newest first
  • 22 Oct 2026Nvidia's own filings flag that open models could end up on rival hardware; software that makes chips interchangeable is how that happens.
  • 22 Oct 2026By buying Hugging Face, Nvidia is moving to own the front door to open models, which could make its hardware the default even without requiring it.
  • 1 Aug 2026The routing-layer question has always been: durable toll or acqui-hire feedstock? The best-funded player in the category is about to announce a nine-figure seed with three revenue lines that don't need the moat to exist — and a deepening single-vendor tie that looks like how absorption stories start.
1 earlier
  • 4 Jul 2026Every company that abstracts AI hardware keeps getting bought by a chip vendor: Qualcomm just paid $3.9B for Modular, after AMD took Pensando and NVIDIA absorbed Groq. The layer keeps failing as an independent toll booth and keeps paying anyway, because the acquisition price rises roughly 10x each cycle. Meanwhile the neutral model-router (OpenRouter, now $1.3B) survives precisely because the clouds' own routers only route within their own catalogues.