2 posts tagged with "user-trust"

The McDonald's drive-thru AI didn't fail because users complained. It failed because users stopped using the drive-thru. For three years the system logged healthy "acceptance rates" while viral videos showed customers pleading with it to remove 260 chicken nuggets from their order. When the partnership ended, the official reason was that the technology "wasn't yet ready." The real signal had been sitting in foot traffic data the whole time — unread, unmeasured, unreported.

This is the shape of most AI feature failures in production. Users don't disable your feature. They don't file tickets. They don't leave one-star reviews. They quietly route around it, and your dashboards keep showing green.

The metrics tell you to kill it. Three percent of monthly actives. The eval refresh has slipped two cycles. The prompt has a // TODO: revisit when we move off the legacy ticket schema from a year ago. Your senior AI engineer spends a full week per month babysitting the thing — model upgrades, label drift, the one tool integration that flakes whenever the upstream API changes its date format. Every quarterly review, somebody asks why this assistant still exists, and every quarter the answer is "we haven't gotten to it yet."

So you write the deprecation memo. You copy the structure from the API sunset playbook your platform team perfected: T-minus-six-months announcement, a migration guide, a banner in the product, a webhook for partners, the usual Sunset: HTTP header. You ship it on a Tuesday. By Thursday afternoon, your CSMs are forwarding emails that don't sound like API deprecation complaints. They sound like breakup letters.

That's the moment most teams realize they took a category error to production. The thing you're retiring isn't an API. It's a relationship the user formed with something that talked back.