Agentic AI is the pattern where one user goal triggers multi-step LLM orchestration with tool calls, retries and failure recovery; it now works well enough to run instrumented laboratories and chip-design loops unattended, but its cost profile, energy accounting and security model are all measurably unresolved.
Summary
Agentic AI describes systems in which a language model is not asked to produce a single answer but to pursue a goal over many steps: planning, calling external tools, reading results, retrying on failure and maintaining state between steps. The definition used in the systems literature is operational rather than philosophical, namely “a single user goal triggers multi-step orchestration, tool calls, retries, and failure recovery”. The step from stateless generation to stateful autonomy is the whole point and also the source of most of the difficulty.
The engineering reality is that the model is often not the expensive part. Measurement of tool-heavy agentic workloads finds tool execution dominating end-to-end latency: 81-89% for retrieval in Haystack, up to 65% for Bash/Python in SWE-Agent, 85-88% for conformer generation in ChemCrow, while a tool-light workload such as Toolformer stays roughly 88% GPU-bound, so the split is workload-dependent 2511.00739 Cpu Centric Agentic Execution. A separate audit reports CPU-side processing accounting for up to 90.6% of latency and 44% of dynamic energy, and orchestration structure dominating energy cost, with workflows consuming 4.33x more energy per successful goal than linear baselines and up to 7.63x for multi-step reasoning. The proposed remedies so far are software scheduling, not new silicon: CPU-aware overlapped micro-batching and mixed agentic scheduling deliver up to 1.7-3.9x lower latency from existing CPU-GPU resources 2511.00739 Cpu Centric Agentic Execution.
The second parameter that decides economics is model size. NVIDIA Research argues that agents need narrow task competence rather than broad conversation, that small language models are “sufficiently powerful, inherently more suitable, and necessarily more economical” for most agentic invocations, that 40-70% of LLM calls in existing agentic systems could be served by specialised SLMs, and that paying 10-30x for unneeded capability is unsustainable at scale 2025 06 Slm Future Agentic Ai Nvidia.
The third parameter is trust. Agents that hold state, credentials and tool access break the assumptions of conventional access control. Role-based access control is described as insufficient for stochastic agents in critical cloud infrastructure; memory-based attacks such as memory injection decouple the compromise from the eventual harmful action, defeating endpoint content filters; and a survey of 21 proposed agent permission systems finds no settled way to express, derive or enforce user-level policy. Where these three parameters land, cost per successful goal, model size mix, and enforceable permissions, decides whether agentic AI is a laboratory technique or an enterprise substrate.
Evidence base
Open questions
- Does the 40-70% SLM substitution estimate from 2025 06 Slm Future Agentic Ai Nvidia hold in a deployed production agent, and what is the measured effect on task success rate rather than on cost alone?
- Do the CPU-side latency shares of 81-89% for retrieval and up to 65% for code execution 2511.00739 Cpu Centric Agentic Execution persist once the scheduling fixes are applied, or does the bottleneck move back to the model?
- Will any of the 21 surveyed permission approaches converge into an enforceable cross-vendor standard, or will each agent platform keep its own product-level policy?
- Can agentic performance be shown outside domains with an executable acceptance predicate, given that HORIZON’s authors call their 100% completion result a controlled proxy?
Assessment drafted 2026-08-31 from up to 18 KB sources using the technology-scorecard framework; scores are a draft read pending review.