AI Agent Failures Need Escrow, Not Just Safety Training

A cross-institutional research team ran 5,000 AI agent transaction simulations and found one consistent result: financial safeguards borrowed from insurance and capital markets can cut user losses by up to 61%.

The paper - "Quantifying Trust: Financial Risk Management for Trustworthy AI Agents" - was published April 8 on arXiv by researchers from Microsoft Research, Columbia University, Google DeepMind, t54 Labs, and Virtuals Protocol. It proposes what the authors call the Agentic Risk Standard, or ARS, a settlement-layer protocol for autonomous AI systems that handle financial tasks for users.

The idea is disarmingly simple. Existing AI safety research tries to make models fail less. ARS assumes they'll fail, and asks what happens to the user's money when they do.

TL;DR

5,000 simulation rounds show escrow and underwriting can cut AI agent financial losses by 24-61%
15-20% of risky transactions were deterred before execution when agent providers had to post collateral
The "guarantee gap": AI safety reduces failure probability but never reaches zero - and probability is not a guarantee
FINRA's 2026 oversight report issued its first-ever warning to broker-dealers about AI agent hallucinations in financial settings
The standard is open-source, available via T54 Labs on GitHub

The Guarantee Gap

The paper opens with a framing that should be uncomfortable for anyone building agentic AI products for consumer finance: no amount of safety training removes hallucination risk. Language models are stochastic systems. The probability of failure can be reduced - but it can't be removed by technical means alone.

The researchers call this the "guarantee gap." Users in high-stakes settings need enforceable guarantees. AI systems can only offer probabilities. That gap, left unaddressed, produces a rational outcome: users limit what they delegate to AI agents, constraining the whole market.

"Most trustworthy AI research aims to reduce failure probability. That work is essential, but probability is not a guarantee. ARS takes a complementary approach - instead of trying to make the model perfect, we formalize what happens to users when systems fail."
Wenyue Hua, Microsoft Research, lead author

The point isn't that AI agents are dangerous. It's that the financial plumbing hasn't caught up with the product ambitions. Visa, Mastercard, Stripe, and Google are actively building protocols to let AI agents spend money for users. Anthropic launched its Managed Agents platform this week. Anthropic's own research found AI agents now run autonomously for an average of 45 minutes without human input. None of that infrastructure includes a loss-recovery mechanism.

The Numbers

The ARS framework splits AI tasks into two categories and applies different protection layers to each.

Task Type	Example	Protection Layer	Mechanism
Fee-only	Produce a report, write code	Escrow	Service fee held until verified
Fund-handling	Currency trading, financial API calls	Escrow + underwriting	Risk priced, collateral posted, user reimbursed on failure

In simulations, the results varied markedly depending on configuration:

Escrow alone produced consistent but limited protection for fee-only tasks
Underwriting reduced losses by 24-61% in fund-handling scenarios, depending on how accurately underwriters estimated failure rates
Collateral requirements independently deterred 15-20% of risky transactions before execution - because fraud or misexecution now carried a direct cost to the agent provider

The deterrence effect is worth sitting with. The framework doesn't just compensate users after failures. It changes the incentive structure for agent providers, who now have skin in the game.

The ARS framework developed by T54 Labs and Google DeepMind researchers The ARS architecture showing how escrow vaults, collateral requirements, and underwriting layers interact in the proposed settlement protocol. Source: globalcrypto.tv

The Escrow Mechanism

For standard service tasks, escrow is the baseline. Payment is held in a vault and released only after verified task delivery. This is a solved problem in traditional commerce - construction, legal services, and software licensing all use escrow routinely. The paper applies the same logic to AI outputs.

The Underwriting Layer

For capital-involving tasks - trading, currency conversion, calls to financial APIs - escrow isn't enough. The outcome isn't known at payment time. The ARS solution adds an underwriting layer: a risk-bearing third party assesses the task, prices the failure risk, may require the agent provider to post collateral, and commits to reimbursing the user under specified failure conditions.

This mirrors how financial clearinghouses work. It's not a novel idea. What's new is applying it to AI.

The Collateral Effect

The 15-20% deterrence figure from collateral requirements is arguably the most interesting result. It suggests that requiring agent providers to post capital before accessing user funds would filter out a meaningful share of risky operations before they execute. Insurance and derivatives markets learned this lesson decades ago. ARS proposes importing it.

What the Numbers Don't Say

The simulation environment is stylized. Five thousand rounds is enough to show directional effects, not to calibrate precise real-world loss figures. A few limitations are worth flagging.

Failure-rate estimation is the hard problem. The paper found that zero-loading premiums - where underwriters don't mark up risk to cover their own costs - produced insolvency. Accurate failure-rate estimation is the difference between a functioning market and one that collapses under its first major event. That estimation problem isn't solved in this paper or anywhere else.

Non-financial harms are excluded. ARS covers losses denominated in money. It says nothing about hallucinations that cause defamation, incorrect medical advice, or legal missteps. Those remain entirely unaddressed.

Who bears the cost? The paper doesn't model who actually pays underwriting premiums in equilibrium. In financial markets, these costs eventually pass to end users. Whether ARS makes AI agents more expensive for retail users - and by how much - isn't analyzed.

Regulatory alignment is absent. The standard is open-source and voluntary. There is no current regulatory requirement for AI agent providers to implement anything like it.

FINRA's 2026 Regulatory Oversight Report on generative AI and agent risks in financial services FINRA's 2026 Annual Regulatory Oversight Report included its first dedicated section on generative AI risks, warning broker-dealers to develop procedures targeting hallucinations and unauthorized agent actions. Source: finra.org

That last point matters. FINRA's 2026 Annual Regulatory Oversight Report - published late last year - included its first-ever section on generative AI. It warned broker-dealers to develop procedures targeting hallucinations and flagged that agents may act "beyond the user's actual or intended scope and authority." FINRA stopped short of mandating any specific technical safeguard. The SEC and other agencies are reportedly monitoring but have issued no guidance.

The ARS paper gives regulators a framework to point at. Whether any regulator will require it is a different question.

"The industry is building increasingly autonomous AI agents but hasn't addressed what happens when they fail with someone's money. That's the problem t54 Labs was founded to solve."
Chandler Fang, co-founder, t54 Labs

So What?

If you're building financial products on top of AI agents, the relevant takeaway is the deterrence data, not the loss-reduction headline. A 15-20% reduction in risky transactions through collateral requirements is meaningful even before underwriting enters the picture. That's a design choice available today, without waiting for regulatory mandate.

For everyone else: the paper's real contribution is naming a problem the industry has been awkward about. Every major AI lab is announcing agentic platforms. Very few are explaining what users are entitled to recover when those agents go wrong. The ARS won't fix that on its own - but it makes the silence harder to maintain.

Sources: