2 posts tagged with "liability"

An agent books the wrong flight. It sends an apology email to the wrong customer. It writes a database migration that drops a column three services still read from. In each case the model produced a plausible-looking action, executed it, and moved on. And in each case somebody absorbed a real cost — a rebooking fee, a damaged relationship, an incident bridge at 2 a.m.

Here is the uncomfortable part: most AI products have no answer for who that somebody is. The question never comes up in the design review. It surfaces later, one ticket at a time, in a support queue where an agent improvised a $40 credit because the customer sounded angry and the rep had no policy to point at. Multiply that by a few thousand tickets a month and the unit economics quietly rot — not from a dramatic failure, but from a slow leak nobody scoped.

"The model made a mistake" is not a support escalation. It is a billing event. And the products that survive the agentic era will be the ones that designed for that event before the first angry ticket, not the ones that improvised refunds by vibes until the gross margin went negative.

A customer's general counsel sends you a one-line email: "When the model invents a fact in our compliance workflow next week, whose insurance is paying?" You forward it to your VP of Engineering, who forwards it to Legal, who forwards it back to you. By the time the chain closes, three people have separately assumed that someone else read the model provider's terms carefully. None of them did. The contracts don't actually connect — and you are the layer in the middle that finds out first.

This is the AI indemnification gap. It exists because every enterprise AI product sits in a three-link liability chain — end customer, your product, model provider — where each link silently assumes the layer underneath is carrying the weight. The model provider's terms cap damages at roughly the last twelve months of fees and explicitly exclude output accuracy. Your MSA inherits those exclusions through a flow-down clause your customer's lawyer didn't read carefully. Your customer's contract with their downstream user — the actual end-of-chain victim when an output goes wrong — names your product as the responsible party with no clear upstream recourse.

The first claim discovers the gap. Until then, everyone in the chain is operating on a hopeful shrug.