How AI Agents Break - Plus Fixes for Memory and Tools
Three arXiv papers map how LLM agents fail across 19 benchmarks, show in-process memory cuts retrieval latency 1,000x, and reveal steering vectors that control tool invocation.
6 min read
Three papers appeared on arXiv this week that each pull at a different thread of the same problem: AI agents are much harder to build reliably than benchmarks suggest. One synthesizes 27 evaluation studies into a coherent map of what breaks and why. Another shows that where you put memory changes what agents can actually accomplish - not just how fast they respond. A third shows that a model's tool-calling behavior can be flipped using a vector extracted from its own activations, with no retraining required.
TL;DR
- Beyond the Leaderboard - Synthesizing 27 papers across 19 benchmarks reveals six failure clusters; compound failure rates scale nonlinearly with task length
- Recall lifted from 0/5 to 4.8/5 in Memory in the Loop by moving retrieval inside the reasoning loop, cutting latency from 100ms to 100 microseconds
- Heading-Specific Activation Steering gives bidirectional control over tool invocation across five open-source models without any retraining
Six Ways Agents Fail
Wael Albayaydh, Rui Zhao, and Ivan Flechais synthesized 27 benchmark and audit papers spanning 19 distinct benchmarks into one unified taxonomy. Their paper, "Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents," is the kind of literature review that makes you feel better about past failures and worse about future confidence.
They identified six distinct failure clusters:
| Failure Cluster | What It Covers |
|---|---|
| Tool invocation errors | Mistakes in calling and parameterizing tools |
| Planning failures | Constraint satisfaction, multi-step coordination |
| Long-horizon degradation | Performance decay from building up context |
| Multi-agent coordination | Communication and collaboration breakdowns |
| Safety vulnerabilities | Failures under adversarial or ambiguous inputs |
| Measurement problems | Evaluation validity gaps |
Synthesizing 27 benchmark papers reveals consistent failure patterns that individual evaluations miss.
Source: wikimedia.org
That last cluster - measurement problems - deserves more attention than it usually gets. Benchmark scores often don't capture the failure modes that matter in production. A well-designed eval on narrow sub-tasks can mask systematic gaps in end-to-end performance, which is exactly how systems pass benchmarks and still disappoint in deployment.
The more uncomfortable finding concerns compound failure. Albayaydh et al. found that failures scale nonlinearly with task length. An agent that handles each individual step with 90% accuracy doesn't produce 90% accuracy on a ten-step task. The math compounds against you. And adding scaffolding - more tools, more prompting, more structure - doesn't reliably improve reliability. It often introduces new coordination failure points of its own.
Progress is real, but it's concentrated in narrow domains. Single-turn tool use and short-horizon web navigation have improved measurably. The hard cases - multi-step, multi-agent, long-context - remain largely unsolved, and the taxonomy makes clear that these aren't just engineering problems waiting for more scale.
Memory Placement Is a Performance Lever
Yusuf Khan and Carlo Lipizzi ran a deceptively simple experiment: what happens if you move memory retrieval from an external network call into the same process as the reasoning loop?
The answer, documented in "Memory in the Loop: In-Process Retrieval as Extended Working Memory for Language Agents," is dramatic. Latency dropped from the typical 50-200ms for network retrieval to around 80-165 microseconds at p50 - roughly a 1,000x reduction. Recall across GPT-5-class models improved from 0/5 to 3.6-4.8/5. Zero facts were lost across 244 writes.
The mechanism makes intuitive sense once you think about it carefully. When an agent waits 100ms or more for each memory fetch, it learns to avoid fetching altogether - compensating by relying on parametric memory (what it already "knows") or repeating actions. Khan and Lipizzi measured 7.2 redundant actions per 12-step task at cloud memory latencies. At in-process speeds, redundancy dropped to zero.
The dominant cost shifted from retrieval to embedding: at 200-400ms over the network, the embedding step became the new bottleneck.
That bottleneck shift has a practical implication for anyone building AI agents. Pairing a local memory store with a local embedding model captures most of the benefit. The full optimization stack is local embedding + in-process vector store + in-loop retrieval, but even getting the embedding local substantially changes what's possible.
Moving retrieval inside the reasoning loop reduces latency by three orders of magnitude.
Source: unsplash.com
Steering Vectors Control Tool Use
The third paper takes a mechanistic angle. Yuqi Chen, Vincent Siu, Yang Liu, Dawn Song, and Chenguang Wang asked: if a model decides to call a tool, where does that decision live inside the model?
"Controlling Tool Use with Heading-Specific Activation Steering" finds that steering vectors extracted from heading-anchor positions - the positions corresponding to tool-description headings in the model's context - exert bidirectional causal control over tool invocation. They tested this across five open-source models and three domains, suppressing unnecessary tool calls in cases where the model can reason parametrically without any retraining.
The geometric analysis complicates the simple story. Tool-invocation steps show diffuse, bimodal alignment with the suppression vector rather than the clean negative alignment a linear encoding model would predict. That's useful information: it means tool-use decisions aren't stored as a single clean direction in activation space. The representation is messier.
What This Means for Practitioners
Two practical constraints to flag. First, different tool types activate largely separate internal mechanisms with minimal feature overlap. A vector that suppresses web search calls won't necessarily suppress calculator calls. Each tool type appears to have its own decision circuit. Second, this technique requires white-box model access - you need to read activations. That rules it out for API-only deployments but makes it directly usable for anyone running open-source models locally.
The broader significance sits within the AI safety and interpretability literature: understanding where behavioral decisions live in activation space is prerequisite work for any serious attempt to control those behaviors at inference time without retraining.
The Connecting Thread
These three papers aren't formally in dialogue, but the connection is hard to miss. The failure taxonomy paper says agents break in systematic ways that compound with task length. The memory paper shows that an architectural choice about where retrieval happens changes agent behavior at a fundamental level. The steering paper shows that specific decisions - which tool to call, whether to call any - have locatable internal representations that can be adjusted directly.
Three different handles for the same reliability problem: better evaluation to see where agents fail, better memory architecture to reduce the behaviors those failures produce, and direct behavioral control at the activation level for cases where you need precise adjustments. None of them is sufficient on its own.
Sources:
- Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents - Albayaydh, Zhao, Flechais (2026)
- Memory in the Loop: In-Process Retrieval as Extended Working Memory for Language Agents - Khan & Lipizzi (2026)
- Controlling Tool Use with Heading-Specific Activation Steering - Chen, Siu, Liu, Song, Wang (2026)
