Nobody Will Underwrite Your Agent

Your agent works. It resolves the ticket, issues the refund, updates the record, closes the loop — unattended, at a quality your metrics say beats the median human doing the same job. You are ready to take the human out of the loop. And then the deployment stalls, not in engineering, but in a meeting where someone from legal or finance asks a single question you cannot answer: when it's wrong, who eats the loss?

This is the part of agent autonomy that the capability curve doesn't touch. You can push accuracy from 95% to 99% and the question doesn't move. Because the blocker was never "is the model good enough." The blocker is that a probabilistic system taking irreversible actions is a risk somebody has to hold, and right now nobody wants to hold it. Your errors-and-omissions policy was written for human mistakes and increasingly carves out automated ones. Your model vendor's contract disclaims the whole thing. And no carrier has an actuarial table for a system whose failure rate quietly drifts every time you edit a prompt.

The uncomfortable truth is that "can we deploy this autonomously" is turning into an insurance and indemnity question far more than a machine-learning one. The teams that figure out how to make an agent underwritable — legible, bounded, and provably supervised — are going to ship autonomy while everyone else is still stuck in the liability meeting. This is a piece about how to be the first kind of team.

The liability gap nobody scoped

Start with what the law already says, because it's less ambiguous than most engineers assume. In Moffatt v. Air Canada, a customer asked the airline's website chatbot about bereavement fares, got a confidently wrong answer, booked on that basis, and later sued. Air Canada's defense was almost poetic in its wrongness: the chatbot, it argued, was "a separate legal entity" responsible for its own statements. The tribunal rejected that outright. The bot was part of Air Canada's website; Air Canada was responsible for everything on it, including the parts that hallucinated. The damages were trivial — a few hundred Canadian dollars — but the principle is not. You own your agent's outputs the same way you own a typo on your pricing page. There is no "the model did it" defense.

Now scale that principle from a wrong sentence to a wrong action. A chatbot that misstates a policy costs you the fare difference. An agent that issues the refund, moves the money, cancels the order, or emails the customer a commitment you didn't authorize costs you the action plus the cleanup plus, potentially, a pattern of identical actions across every session that ran the same faulty logic before anyone noticed. The exposure isn't one bad outcome. It's a correlated fleet of them.

That correlation is exactly what makes the risk hard to place. Traditional liability is a law of large numbers: independent, uncorrelated errors that average out, which is what lets an actuary price them. An agent doesn't fail independently. One bad model update, one poisoned retrieval source, one prompt edit that shifts behavior three degrees, and every instance fails the same way at the same time. Insurers have a name for that shape, and it isn't "professional liability." It's catastrophe — the same bucket as floods and wildfires, where a single event triggers thousands of simultaneous claims. Carriers price catastrophe risk very differently, when they agree to price it at all.

Why your existing coverage quietly excludes it

Most teams assume their existing policies stretch to cover this. In most cases, they don't, and the language is moving in the wrong direction.

Professional indemnity and errors-and-omissions insurance were built around human error — a person exercising professional judgment and occasionally getting it wrong. AI inserts a non-human decision-maker into that workflow, and carriers have noticed. Through 2025 and into 2026, insurers began attaching endorsements that fully exclude any claim arising out of AI use, output, training, advice, or automated decision-making. Not a sub-limit. A carve-out. Underwriters describe these systems, not unfairly, as black boxes producing outputs that are neither deterministic nor consistent — and quantifiability is the entire foundation of underwriting. If they can't bound the loss distribution, the rational move is to exclude the risk rather than misprice it.

There's a second trap in the disclosure clauses. Some policies now condition coverage on disclosing AI use, which means a team that quietly wired an agent into a customer-facing workflow may have voided its own protection under a misrepresentation or failure-to-inform clause. The coverage you think you have may already be gone, and you'd only find out at claim time.

And don't expect the vendor to backstop you. One review of AI vendor contracts found that 88% cap liability at the monthly subscription fee and only 17% offer any regulatory-compliance warranty. Read that literally: if a model you pay a few thousand dollars a month for causes a six-figure loss, the contract caps the vendor's exposure at a few thousand dollars — and many of those same contracts include indemnification clauses that require you to defend them. The liability doesn't stop at the vendor. It flows straight through to you, by design.

What a market is starting to form around

Loading…