Acceptance Sampling for Agent Output: What Manufacturing QA Knows That Code Review Doesn't
Your agent fleet opened forty pull requests this week. You reviewed the six that touched payment code, skimmed a few that happened to land while you had the tab open, and merged the rest on green CI. If someone asked you what your review policy is, you would describe something like this — and it is not a policy. It is a mood.
The numbers say most teams are in the same place. A recent large-scale study of agent-authored pull requests in popular open-source repositories found that 61% receive no recorded review activity at all, and of the ones that do get reviewed, most are reviewed only by other agents. Meanwhile the volume keeps climbing: agents now produce PRs, documents, support responses, and tickets at a rate no human review process was designed for. Reviewing everything is impossible. Reviewing nothing is malpractice. So teams improvise somewhere in the middle, with no stated rule, no measured coverage, and no way to say whether the current level of scrutiny is paranoid or reckless.
Manufacturing solved this exact problem a century ago. When Western Electric was churning out telephone equipment in the 1920s, inspecting every unit was economically impossible and shipping uninspected lots was unacceptable — so Bell Labs statisticians built acceptance sampling: a mathematically grounded discipline for deciding how much of a batch to inspect, when to reject the whole batch, and when a supplier has earned lighter scrutiny. It became MIL-STD-105 during World War II, then ANSI/ASQ Z1.4 and ISO 2859-1, and it still governs how a container of goods gets accepted at a port today. The mapping onto agent fleets is almost embarrassingly direct — and almost nobody in AI engineering has picked it up.
