The Accuracy Threshold Problem: When Your AI Feature Is Too Good to Ignore and Too Bad to Trust

McDonald's deployed its AI voice ordering system to over 100 locations. In testing, it hit accuracy numbers that seemed workable — low-to-mid 80s percent. Customers started posting videos of the system adding nine sweet teas to their order unprompted, placing bacon on ice cream, and confidently mishearing simple requests. Within two years, the partnership was dissolved and the technology removed from every location. The lab accuracy was real. The real-world distribution was not what the lab tested.

This is the accuracy threshold problem. There is a zone — roughly 70 to 85 percent accuracy — where an AI feature is precise enough to look like it works, but not reliable enough to actually work without continuous human intervention. Teams ship into this zone because the numbers feel close enough. Users get confused because the feature is just good enough to lure them into reliance and just bad enough to fail when it matters.

Why 70–85% Is the Worst Zone to Deploy In

A naive intuition says partial accuracy is better than no accuracy. The research says otherwise.

A 2024 meta-analysis published in Nature Human Behaviour reviewed 106 experimental studies on human-AI combinations. Across decision-making tasks — which made up 85% of the studies — human-AI teams performed worse than the better of the two alone, with an effect size of g = -0.27. The finding that surprises practitioners: when AI outperformed humans, the combined team did significantly worse (g = -0.54) than when humans outperformed AI. Combining a 75% accurate AI with a 65% accurate human produces outcomes worse than using the AI alone, because the human brings their errors without bringing proportional corrective value.

The same dynamic shows up in a different form with calibration. In controlled experiments, when AI expressed 80% confidence despite only being 70% accurate, 69.6% of users adopted AI suggestions even when they disagreed with their own judgment. Well-calibrated AI (60% confidence, 70% accurate) led to 11.9% decision improvement over baseline. The overconfident version? 7.2% — barely better than chance. The critical failure: 64.3% of users couldn't tell the AI was overconfident at all.

This is why 70–85% accuracy is specifically dangerous. It's good enough to generate habitual use. It's bad enough that confident presentation causes visible failures. And users, at scale, cannot detect the difference between a 75% accurate system and a 95% accurate one from interaction patterns alone.

What Actually Happens When You Ship in This Zone

The failure modes cluster into three patterns:

The viral failure pattern. When accuracy is high enough to be deployed at scale, visible errors reach millions of users. Google AI Overviews had an error rate around 10% at launch. That sounds manageable until you recognize that "10% error rate at millions of queries per day" means tens of millions of wrong answers delivered confidently, with no uncertainty signal. Users who see AI recommend eating rocks for nutrition don't conclude "this is a calibrated 90% accurate system." They conclude "this system is broken." Only 8% of users double-check AI answers — the other 92% accepted what they saw.

The hidden labor pattern. Amazon's "Just Walk Out" checkout technology was marketed as AI-powered cashierless shopping. Internally, roughly 70% of transactions required manual review by workers watching video footage. The AI wasn't delivering 70% accuracy on its own — it was delegating 70% of its work to hidden humans. The practical outcome was higher labor cost than conventional checkout, delivered under a headline that implied the opposite. Amazon quietly removed the technology from its grocery stores in 2024.

The catastrophic edge case pattern. Klarna's AI customer service handled 2.3 million conversations per month and initially reported satisfaction "on par with humans." What the headline metrics didn't capture: complex financial disputes, billing errors requiring judgment, and frustrated customers who couldn't escalate to a human. The aggregate accuracy was fine; the tail of cases requiring nuanced judgment was not. By 2025, Klarna's CEO acknowledged "lower quality" and the company reversed course, rebuilding its human customer service team. The cost of removing the human fallback wasn't visible until the fallback was needed.

The Automation Bias Trap

Automation bias — the tendency to follow AI suggestions even when they're wrong — gets worse, not better, in the 70–85% zone. A systematic review of 35 studies found that higher perceived benefit of the AI system was significantly associated with false agreement with wrong AI recommendations. The people most likely to benefit from AI assistance — non-specialists, lower-experience workers — are also the most vulnerable to automation bias.

This creates a paradox. You deploy in this zone because the feature is valuable to users who need help. The users who need the most help are also the ones least likely to catch the errors. And those users are also the ones most likely to form habits around the AI before they've seen enough failures to calibrate their trust appropriately.

UnitedHealthcare's nH Predict AI determined hospital stay lengths for Medicare Advantage patients. The system had an accuracy problem: more than 90% of its coverage denials were overturned on appeal. By the time patients discovered errors, some had already been discharged earlier than medically appropriate. The consequence of getting calibration wrong in that context wasn't user frustration — it was patient harm and litigation.

The throughline across these cases is the same: an accuracy number that looked workable in a lab, a deployment that reached scale before calibration failures were understood, and a feedback loop that surfaced errors too slowly to correct behavior before damage accumulated.

What Confidence-Calibrated UI Actually Means

The instinct when accuracy is partial is to add a disclaimer. "AI-generated content. Verify important information." This doesn't work. Users anchor on the output, not the caveat. A 2025 Frontiers in Computer Science study found uncertainty visualization increased trust calibration for 58% of initially skeptical users — but only when the visualization was contextual, not generic.

What works is not communicating uncertainty as an afterthought. It requires architectural decisions:

Tiered confidence labeling, not raw probabilities. "Likely" / "Uncertain" / "Low confidence" outperforms "73% confident" in user studies. False precision — "99.73% confidence" — destroys trust faster than honest uncertainty because it implies calibration the system doesn't actually have. The frequency format ("7 out of 10 times this would be correct") outperforms percentages for most users.

Progressive disclosure, not front-loaded explanations. Google PAIR's updated guidelines are explicit: "The perfect time to show explanations is in response to a user's action." Explainability surfaced reactively, when a user challenges or reviews an output, is more useful than prefacing every output with a confidence panel. Users tune out static disclaimers. They engage with uncertainty that's responsive to their behavior.

Domain-specific accuracy, not aggregate scores. An AI agent may be 92% accurate on scheduling tasks and 65% accurate on financial analysis. Showing a blended 78% tells users nothing useful. Showing per-task accuracy lets users calibrate their review behavior appropriately. Waymo's approach — surfacing what the car sees in real-time and publishing peer-reviewed safety papers — gives passengers the inputs needed to form correct expectations, not just reassuring aggregate statistics.

Routing to humans as a first-class feature, not an escape hatch. In every case study above, the human fallback was either removed or treated as a backup that users had to fight to access. Designing a confidence threshold that triggers human escalation — before the user hits a visible failure — is what prevents the "I tried AI and it failed me" moment that collapses trust. Air Canada discovered this after a tribunal ruled the company liable for its chatbot's incorrect policy advice. The chatbot had no escalation path; the company had no way to prevent incorrect information from being treated as authoritative.

The Ship/Hold Decision Framework

The question of when to ship into partial accuracy isn't binary. It depends on two variables: reversibility and task stakes.

Irreversible decisions — coverage denials, legal documents, records that get acted upon — require accuracy close to the human baseline before deployment. The consequence of error isn't "user retries"; it's a decision that persists in the world after the AI has moved on. In these domains, 80% accuracy is not a starting point to optimize from; it's a contraindication.

Reversible decisions with low stakes — content drafts that users review, scheduling suggestions they can ignore, search results ranked by AI — tolerate partial accuracy because users are already in a review loop. The AI can be wrong 20% of the time without catastrophic consequences, as long as the UI makes clear that review is expected and easy.

The failure mode most teams hit is not shipping too early or too late. It's shipping with the wrong mental model: treating a reversible-review workflow as if it were a trust-and-deploy workflow. Klarna removed human fallback from complex cases. Amazon marketed the hidden labor as AI autonomy. McDonald's deployed to 100 locations without testing the actual distribution of accents, background noise, and overlapping voices. In each case, the accuracy number was real; the deployment context didn't match the evaluation context.

Three concrete signals that the deployment context is likely to diverge from the evaluation context: evaluation data was curated rather than sampled from real traffic; the human fallback requires user effort rather than being triggered automatically; and aggregate accuracy metrics aren't broken down by the tail cases most likely to cause visible failures.

Designing for the Trust Build-Up

Partial accuracy is not inherently a disqualifier. Gmail's Smart Compose has always been partial — a suggestion engine, not an autocomplete. It earned trust incrementally by starting conservative (short suggestions, frequently accepted) and becoming more elaborate as users built acceptance patterns. Users learned the system's confidence through behavioral feedback, not explicit scores. The feature is workable at partial accuracy because the UI design never implied otherwise.

Spotify Discover Weekly takes calculated risks with later recommendations only after safer, more certain recommendations have built up a history of success. The system earns the right to surprise you. It doesn't demand trust upfront.

The common thread is that trust builds slowly and breaks quickly. A single visible failure after a streak of correctness should trigger proportional trust reduction, not total collapse — but only if the product design has been preparing users for partial accuracy rather than papering over it with confident presentation.

The 70–85% accuracy zone is not the end of the road. It is a deployment context that requires specific UI design choices, explicit human routing thresholds, and honest calibration of what users should expect. The failure isn't shipping in this zone. The failure is shipping as if you were at 95%.

The teams that get this right start with the question: "What does this system need to look like for users to calibrate their trust correctly?" Not "How do we make the accuracy number look good enough to ship?"