3 posts tagged with "ux-design"

Watch a developer use an agentic coding tool for an hour and you will see the same gesture forty times: a dialog appears, "Allow the agent to run git status?", and a hand moves to the approve button before the eyes finish reading. By the fortieth prompt the prompt is not being read at all. It is a speed bump the user has learned to take at full speed.

This is the quiet failure of human-in-the-loop. The architecture diagram still shows a human gating every dangerous action. The audit log still records an explicit approval for every command. But the human has stopped evaluating anything. They have become a biological "yes" function wired into the control flow — present in the loop, contributing no judgment to it. The permission prompt was supposed to be a safety control. It has degraded into latency with a confirmation dialog attached.

A feature that fails 70% of the time is harmless. The user learns within a week that they have to verify every output, treats the system as an unreliable assistant, and adjusts. A feature that succeeds 70% of the time is worse than that. It is right often enough that the user stops verifying, and wrong often enough that the failures are concentrated, visible, and personal. The user's mental model collapses into "I cannot tell when to trust this" — which, as a product experience, is strictly worse than "I know not to trust this."

This is the 70% uncanny valley, and it is where most AI features built in the last two years live. The team measures aggregate accuracy, watches the number cross some "good enough" threshold, and ships. The realized user experience does not improve monotonically with that number. Between roughly 60% and 85% accuracy, the product gets worse as it gets more accurate, because the cost of a wrong answer the user did not think to check exceeds the value of a right answer they no longer have to verify.

The team that ships at 70% without designing for the predictability problem is not shipping a worse version of a 95% product. They are shipping a different product entirely: one whose primary failure mode is silent.

Most teams building ambient AI ship something users immediately turn off.

The pattern is consistent: the team demos the feature internally, everyone agrees it's useful in theory, and within two weeks of launch the disable rate exceeds 60%. This isn't a model quality problem. It's an architecture problem — and specifically an interrupt threshold problem. Teams design their ambient agents around what the AI can do rather than what users will tolerate when they didn't ask for help.

The gap between explicit invocation ("ask the AI") and ambient monitoring ("the AI watches and acts") is not just a UX question. It demands a fundamentally different system architecture, a different event model, and a different mental model for when an AI agent earns the right to speak.