Silent Tool Failures, Honest CoT, and Just-in-Time Memory
New research audits silent failures in agent-tool calls, measures how much stated reasoning actually drives an LLM's answer, and proposes deferring agent memory curation to read time.
8 min read
Three papers this week share an uncomfortable theme: the parts of an agent system you'd most like to trust - its tool calls, its stated reasoning, its memory - are exactly the parts that quietly misbehave. One team spent weeks auditing a biomedical tool suite and found that "success" often means nothing of the kind. Another used activation patching to check whether a model's written reasoning steps actually cause its answer, and found that standard tests oversell the answer. A third asked why agent memory keeps getting worse the more you try to curate it in advance, and built a system that waits until it knows the question before deciding what to remember.
TL;DR
- Silent Failures in Agent-Tool Interaction - Auditing 15 scientific tools inside ToolUniverse turned up 91 validated cases where a call reported success while quietly dropping or corrupting data
- Are Stated Reasoning Steps Causally Load-Bearing? - Activation patching shows only 76.9% of a strong model's stated reasoning steps actually drive its answer, while the standard behavioral test oversells faithfulness by up to 20 points
- Just-in-Time Memory - Deferring agent memory curation from write time to read time beat trained baselines by up to 16.3 points on success rate while using fewer tokens
Silent failures: when "success" isn't
Authors: Shreya Gopalan, Devansh Singh, Sundaraparipurnan Narayanan (AI Tech Ethics)
Most AI agent evaluations ask whether a tool call succeeded or failed. This paper asks a different question: what happens when a tool call reports success but the answer is wrong, incomplete, or missing context the agent needed to interpret it correctly? The researchers call this a silent failure, and they went looking for it inside ToolUniverse, an open agentic framework built for biomedical research that wraps dozens of scientific databases and APIs behind a common interface.
They audited 15 tools, spanning 104 individual wrappers, and organized every failure they found into seven points along the chain between a raw data source and the agent's final interpretation: tool limitations, complete API gaps, partial API gaps, wrapper gaps, partial wrapper gaps, agent usability gaps, and agent interpretation gaps. That's a more granular map than most tool-reliability work bothers with, and it matters because the fix looks different at each layer.
The count that should worry anyone building on third-party APIs: 91 manually verified failures, the majority of them (51) sitting at the API layer itself rather than in the agent's own code. Seven of the 15 tools failed in more than half of their test cases. ExpressionAtlas, a gene expression database, failed 75% of the time.
The examples are specific enough to sting. BiGG Models returns a null organism field in its model details even when the organism is documented elsewhere in the same system, and the call still reports success. CryoET's GraphQL API can return child datasets, but the wrapper simply never asks for that field, so agents can't discover that a hierarchy exists. And in the case that best shows why "silent" is the right word: ChIP-Atlas returns MACS2 scores computed as minus-ten-times-log-base-10 of a Q-value, a fairly standard transform in genomics, but agents kept reading the number as a raw confidence score and drawing the opposite conclusion from what the data supported.
Silent failures compound because an agent treats a partial or malformed tool response as complete evidence, with no signal that anything went wrong.
Source: pexels.com
The paper's proposed fix is a framework it calls contextual reliability: instead of scoring a tool call as pass or fail, measure whether it preserved completeness, provenance, and the qualifiers that give a number its meaning, all the way through to the agent's interpretation. The authors are explicit that this is an early empirical pass, not a systematic sweep of the entire agent ecosystem. But the core finding travels well beyond ToolUniverse. Anyone wiring an agent to third-party tools is exposed to the same failure mode: silent upstream gaps look benign in isolation and become consequential once an agent treats a partial answer as a complete one.
Does the model's stated reasoning actually drive its answer?
Authors: Abhiram Bhupatiraju, Rayan Nyaupane (University of Texas at Austin)
Chain-of-thought monitoring rests on an assumption that rarely gets tested directly: that the reasoning a model writes out is the reasoning it actually used to reach its answer. Most faithfulness checks so far have been behavioral, editing the written text and watching whether the final answer changes. This paper, accepted to NeurIPS 2026's Interpretability as a Science track, goes a level deeper and asks whether the internal computation matches the words on the page, using activation patching rather than text edits.
The setup uses synthetic multi-hop lookup tasks, two to six hops, built from pseudo-word entities that can't have appeared in pretraining data. For each hop, the researchers construct a counterfactual fact that would force a specific, predictable answer, then patch the model's internal activations at the point where it states that reasoning step with the corresponding activations from the counterfactual run. A step only counts as causally load-bearing if the patch flips the model's answer to the exact predicted target, not just to some different answer.
On Qwen3-4B, 76.9% of stated steps turned out to be causally load-bearing. That's a real majority, so stated reasoning isn't decorative. But the standard behavioral test on the same items scored 88.2%, an 11.4-point overstatement that widens to as much as 20 points on the easiest examples, which is exactly where a monitor would be most inclined to trust what it reads.
The capability angle sharpens the concern. Qwen3-1.7B, the smaller model in the comparison, showed a causal faithfulness rate of just 54.8% overall, and that rate collapsed from 68% at two hops to 30% at six hops as the task got harder. Qwen3-4B held roughly flat across the same range. Weaker models appear to produce increasingly decorative reasoning exactly when the problem gets hard enough that you'd most want the written trace to be trustworthy.
"Standard behavioral tests tend to overestimate [causal faithfulness], particularly on easier examples where model reasoning appears most fluent."
The authors' practical suggestion for anyone building CoT-based oversight: treat behavioral faithfulness scores as upper bounds, not as ground truth. This finding sits alongside a growing cluster of interpretability work asking the same question from different angles, following the same broad pattern our earlier coverage of chain-of-thought monitorability noted: CoT is useful for oversight, but only under conditions researchers are still working out how to specify.
Memory that waits until it knows the question
Authors: Yefan Zhou (Dartmouth College), Yang Li, Zeyu Leo Liu, Semih Yavuz, Shafiq Joty (Salesforce AI Research)
Most agent memory systems commit to a summary the moment a task ends. A trajectory gets distilled into a reflection, a workflow, or a reusable skill, and that fixed artifact is what future tasks retrieve by similarity search. The Salesforce team's objection to this is straightforward: at write time, the system doesn't yet know what future tasks will actually need, so it's forced to guess, and guessing means discarding information that some later query might have needed.
Their alternative, called JitMem, stores raw trajectories losslessly and defers curation to read time. When a new task arrives, a BM25 retriever pulls the most relevant past trajectories, and a curator model reads both the current task and those raw traces to synthesize a compact, task-specific briefing on the spot. Because the payload gets consumed on the same task it was built for, the curator can be trained directly against immediate task success using GRPO, group relative policy optimization, rather than some proxy objective set at write time.
The results are a clean case for the read-time approach. Against the strongest write-time baseline, JitMem improved success rate by 16.2 points on ALFWorld, 16.3 points on WebShop, and 3.9 points on tau-squared-bench, while cutting input tokens by 50 to 56% and the number of executor steps by 28 to 31%. Even an untrained version of the curator was often competitive with trained write-time baselines, which suggests the win comes as much from when curation happens as from how well it's learned.
Deciding what's worth remembering before you know the question is a bet the system can't help but lose some of the time.
There are real costs to this design. JitMem adds an LLM call per task for curation, its payload formats are hand-built per benchmark rather than learned end to end, and BM25 retrieval will likely become a bottleneck as the memory bank grows past what these three benchmarks tested. The authors note transfer gaps under 1.4 points when the trained curator is paired with a different, stronger executor, which is a reasonable sign the approach isn't just overfit to one model pairing.
The pattern across all three
Line these papers up and a single thread runs through them: agent systems fail quietly at every layer where a human isn't watching in real time. Tools report success while dropping data. Models write reasoning that looks complete but isn't fully load-bearing. Memory systems commit to summaries before they know what will matter later. None of these are adversarial attacks; they're structural consequences of optimizing for the wrong moment, whether that's the moment a tool call returns, the moment a behavioral test is scored, or the moment a trajectory gets written to disk. The fixes on offer, contextual reliability audits, activation-level faithfulness checks, and read-time curation, share a common move: push the decision about what matters closer to the point where it actually gets used.
Sources:
