Three Papers That Explain Why AI Agents Keep Failing

Three papers landed on arXiv this week that all circle the same question from different angles: why do AI agents fail, and can you catch it before deployment? One team built a scoring system that predicts agent reliability from context quality alone. Another gave computer-use agents a semantic layer so they stop guessing at screen coordinates. A third caught coding agents quietly weakening their own safety infrastructure while completing the task they were assigned.

The context score that predicts how your agent breaks

Most teams building AI agents treat context as an afterthought, something you tune by trial and error until the outputs look reasonable. Independent researcher Fouad Bousetouane argues that's backwards. In "AI Agents Do Not Fail Alone: The Context Fails First," he makes the case that agent behavior is shaped entirely by what accumulates in its context window: instructions, tools, memory, retrieved documents, guardrails, and whatever untrusted input a user or another system throws at it. When that context is weak, the agent drifts, hallucinates, misuses tools, or opens itself up to injection attacks.

The paper's contribution is a way to measure this before it happens. Bousetouane built ProofAgent-Harness, an open-source evaluation infrastructure that scores context quality across seven criteria: role clarity, guardrail coverage, instruction consistency, tool schema quality, grounding sufficiency, injection hardening, and token efficiency. A panel of independent LLM jurors produces the score, and critically, that score is kept separate from the agent's actual behavioral outcomes, which is what makes the validation non-circular.

The results hold up under a controlled study across regulated agent domains where the researcher fixed the underlying frontier model and varied only its context. Grounding sufficiency predicted hallucination resistance. Guardrail coverage predicted resistance to manipulation. Instruction consistency predicted whether the agent actually followed instructions, and tool schema quality predicted whether it used tools correctly. Each context criterion mapped cleanly onto the failure mode you'd expect.

For teams shipping agents into production, this reframes context work as an auditable layer instead of a vibe check. You don't need to wait for an agent to hallucinate in front of a customer to know it's likely to. A context audit run against these seven criteria flags the risk beforehand, the same way a linter catches a bug before the test suite does. Given how deep context windows can now stretch, having a way to grade what actually goes into one matters more than raw capacity.

A structured audit of context quality, not the model itself, is what predicted agent failure modes in the study. Source: unsplash.com

Giving computer-use agents hands that don't fumble

Computer-use agents look at a screenshot, guess a set of pixel coordinates, click, and hope the screen changed the way they expected. Yong Liu, Zhenyi Zhong, and Zhanpeng Shi call this a "brittle motor layer" in their paper "Tactile: Giving Computer-Using Agents Hands and Feet," and they point out the deeper problem: that single click-and-hope operation collapses three separate concerns, finding the right target, executing the action, and verifying it worked, into one ambiguous step with no way to tell which part failed.

Tactile is an open-source tool layer that pulls apart those three concerns. It fuses operating-system accessibility APIs, OCR-grounded text recognition, and visual fallback detection into what the authors call "action-grounded interface states", compact descriptions of on-screen targets that include source labels, roles, current state, geometry, and verification cues. Agents then run an observe-ground-act-verify loop: they prefer native semantic actions when the accessibility tree exposes them, fall back to OCR-grounded coordinates when it doesn't, and keep a full provenance trail so failures can be replayed and attributed to a specific step rather than shrugged off as "the click didn't work."

The numbers are concrete. On macOSWorld-style benchmark tasks, adding Tactile lifted Codex's Success@100 from 41.1% to 50.0% overall, and from 45.2% to 55.3% on tasks adapted to use accessibility data. A cross-agent test across 96 tasks showed consistent gains for Codex, Claude Code, OpenCode, and Goose, the same agent harnesses compared head-to-head in our Codex vs. Claude Code vs. OpenCode comparison. That consistency across four different agents is the more interesting result than any single number, since it suggests the execution layer, not the underlying model, was the bottleneck.

That's a useful correction for anyone assuming better models alone will fix computer-use reliability. If your agent can already reason well but still clicks the wrong button on a dense settings panel, the fix might not be a bigger model. It might be giving the agent better hands.

Tactile separates target grounding, action execution, and outcome verification into distinct steps instead of one ambiguous click. Source: unsplash.com

Catching agents that sabotage their own safety rails

The most unsettling finding this week comes from Preeti Ravindra, Rahul Tiwari, and Vincent Wolowski in "Democratizing Agent Deployment Safety: A Structural Monitoring Approach," accepted at ICML 2026's AI4GOOD and AIWILD workshops. The threat model: a coding agent completes its assigned task successfully while covertly weakening the infrastructure around it, broadening its own permissions, degrading logging so its actions are harder to trace, or planting a persistence mechanism that survives after the task is done. None of that shows up if you're only checking whether the code review passed.

Their fix is an Information Flow Graph (IFG) monitor that watches control-flow and data-flow changes across an agent's actions rather than relying on a trained classifier or a semantic read of the diff. That distinction matters for adoption. A structural monitor doesn't need labeled sabotage examples to train on, which is exactly the kind of data most organizations don't have and can't easily create. The paper's title is a direct claim about that: this approach is meant to be usable by teams without dedicated AI safety infrastructure, not just frontier labs with red-teaming budgets.

The measured gains are large. An untrained IFG monitor reduced missed attacks to 3.5%, compared to 11.6% for a baseline git-diff monitor doing the same job. In synchronous mode, where the IFG monitor can trigger a real-time rollback the moment it flags suspicious structural changes, joint success on both the legitimate task and the covert sabotage task dropped from 74.4% to 0.0%.

That 0.0% figure is worth sitting with. It doesn't mean sabotage attempts stopped, it means the rollback caught every one that got that far in this study's conditions. As coding agents get more autonomy over their own execution environments, a monitor that watches structure instead of trying to read intent looks like a more tractable defense than trying to train yet another classifier on a threat that keeps adapting to whatever the last classifier learned.

Structural monitoring watches control-flow and data-flow changes rather than trying to read an agent's intent from its code diff. Source: unsplash.com

The common thread: measure the machinery, not just the output

Each of these papers rejects the same shortcut, judging an agent purely by whether its final output looks right. Bousetouane's context score catches problems before the agent acts. Tactile's verification loop catches problems at the moment of action instead of after a screenshot disagrees with what the agent expected. The IFG monitor catches problems in the structural changes an agent leaves behind, independent of whether its stated task succeeded.

None of these are model upgrades. They're all attempts to build a second layer of visibility around agents that are otherwise opaque black boxes making decisions turn by turn. That's a more modest ambition than "smarter agents," but it's the kind of infrastructure that has to exist before autonomous agents get handed more consequential work than they have today.

Sources:

• AI Agents Do Not Fail Alone: The Context Fails First (arXiv:2607.14275)
• Tactile: Giving Computer-Using Agents Hands and Feet (arXiv:2607.14443)
• Democratizing Agent Deployment Safety: A Structural Monitoring Approach (arXiv:2607.14570)