2 posts tagged with "automation-bias"

Put a human in the loop and you have a control. Put a hundred approval requests a day in front of that human and you have a rubber stamp wearing a control's badge. The two look identical on an architecture diagram. They behave nothing alike under load, and the gap between them is where most "responsible AI" deployments quietly fail.

The pattern is familiar to anyone who has watched a security operations center drown. Industry surveys put the share of alerts that go uninvestigated somewhere between a quarter and two-thirds — one frequently cited figure is that 62% of alerts are simply ignored, and 55% of teams admit to regularly missing alerts they would classify as critical. The analysts aren't lazy. They are processing a few thousand alerts a day with a false-positive rate that often exceeds 50%, and the human brain responds to that ratio exactly the way you'd expect: it stops looking. Agentic AI is now recreating this exact failure mode, one confirmation dialog at a time.

The single most expensive sentence in a 2026 code review thread is "the agent wrote it this way." Not because it's wrong — sometimes it isn't — but because it ends a conversation that used to start one. The reviewer types a question, the author quotes the model's reasoning back at them, and the thread resolves before anyone has actually argued about the change. The social cost of disagreeing with a confident, well-spoken model has quietly become higher than the cost of merging a subtle bug, and most teams won't see the trade in their metrics for another two quarters.

This is not a story about whether AI writes good code. It writes code, some of it good. This is a story about what happens to a quality gate when the friction at composition time collapses. Review velocity rises, defect rate rises in lockstep, and the correlation isn't obvious because nobody is tracking review-time-to-defect with the author class attached. The senior engineer who used to be the gravity well of taste in the codebase becomes the lone holdout in a culture quietly recalibrating around model deference.