Theses›Datacentre & AI compute›Small models

Small models

Applied AI fragments from a few large frontier models toward many small, specialised, on-device models; this proliferation, not bigger models, is the binding demand driver for sub-watt edge-inference silicon

  • Bear
  • Shut
  • $18B by 2035

The callAI moves onto glasses and earbuds, but the phone chip already runs small models for free; only a memory-device or state-space chip the phone chip can't absorb is open.

D1D2R1R2R3C1foragainst D1D2R1R2R3C1foragainst
Fig. 1The balance. Drivers and catalysts against restraints, each block sized by its weight; catalysts hang dashed because they have not happened yet, sized by their odds. It tips toward the restraints.
  1. D1AI moves onto glasses, earbuds and rings that must listen and see all day on a tiny battery, with AI glasses forecast at 45M to 112M units by 2030.
  2. D2Models get small enough to matter: a 4B model rivals a 72B one on reasoning, and 40-70% of agent calls could be served by small models.
  3. R1The phone neural processor already runs small models above 100 tokens a second, so the general chip absorbs the workload as a feature.
  4. R2The same shift opens the door to Apple, Qualcomm and Arm before any newcomer, and a newcomer's benchmarks have to beat an optimised kernel on a merchant chip.
  5. R3Listed edge-vision chip companies are modest: Ambarella is worth about $2.78B.
  6. C1Always-on inference becomes the default on smart glasses and earbuds, a 100x step in duty cycle a phone neural processor cannot serve on a 200mAh battery.2030
Rests onthe beliefs underneath
Supported by · confidence
What would prove itdated tests, open and resolved
20272028202920302031today123watch 20272028202920302031today123watch
Fig. 2Each ring is a dated test: dashed while open, struck through when overdue, ticked or crossed once resolved.
  1. 1Syntiant's S-1 segment disclosure, which settles the biggest swing in sizing the merchant slice.Dec 2027 · open
  2. 2A named device where on-device inference needs more power efficiency or latency than a general phone chip delivers.Dec 2028 · open
  3. 3Smart-glasses volumes against the 45M to 112M forecasts, and whether the memory-device and state-space companies become components.Dec 2030 · open
Expressionslisted, private, technologies
  • 603893.SSRockchip Electronics Co., Ltd.
    • full
  • AMBQAmbiq Micro, Inc.
    • full
  • NVDANVIDIA
    • cheap
     Dominant share of AI training compute.
  • MSFTMicrosoft
    • full
     Makes the Phi family of small language models and sells them through Azure to developers and enterprises.
  • GOOGLAlphabet (Google)
    • cheap
     Releases the open Gemma small models, including on-device variants for phones and IoT, and runs Gemini Nano on Android devices.
  • METAMeta
    • cheap
     Releases open-weight Llama models, including lightweight sizes for on-device use, to developers.
  • IBMIBM
    • full
     Makes the open Granite small models (3B and 8B language models) and sells them through watsonx and Red Hat to enterprises.
  • BABAAlibaba
    • not screened
     Releases the open-weight Qwen model family, including 2B-4B sizes widely deployed on devices.
Private · 6 private companies, 3 of them in the US.

Automaton Engine · Cartesia · Mistral AI · Liquid AI · Arcee AI · Multiverse Computing

2 more private companies tracked, not shown

Technologies
Who has already bet on itprivate rounds, from public sources

$5.7B raised by 5 companies, $5.6B of it in the last three years. Wave under way: Large rounds in the last three years; the category is funded and getting crowded.

Backed by a top-tier venture firm: Mistral AI, Cartesia. Backed by a strategic investor: Mistral AI, Liquid AI, Cartesia, Arcee AI, Multiverse Computing.

Reading
Show the reading