20 posts tagged with "developer-productivity"

Your next calibration meeting has a measurement problem nobody wants to name. The packet in front of the committee says one engineer shipped 340 PRs this half and another shipped 90. Five years ago that gap meant something. Today it mostly tells you who has the better agent harness, the more permissive review culture on their team, and the higher tolerance for merging generated code. The number on the slide measures the fleet. The committee is supposed to be evaluating the human.

This isn't a hypothetical drift that will arrive someday. Industry analyses estimate AI now writes on the order of 30% of committed code at companies that have adopted agents seriously, and a longitudinal study of 300 engineers found teams generating 98% more pull requests after agent adoption. Every output metric your review process inherited from the pre-agent era — lines changed, PRs merged, story points, velocity — is now a blended measurement of human judgment and machine throughput, with no attribution boundary between them. Calibration committees are comparing blended numbers as if they were still measuring people.

The pitch for overnight coding agents is seductive: you go to sleep, the fleet works, and you wake up to finished pull requests. What actually happens is subtler and more expensive. You wake up to a queue — six branches, two failed runs, a dependency bump you didn't ask for, and a refactor that is either brilliant or subtly wrong. The agents did produce code. But the deliverable that landed on your desk isn't code. It's a triage problem, and most teams have no workflow for it.

The numbers say this isn't a niche complaint. A telemetry study of more than 10,000 developers across 1,255 teams found that high-AI-adoption teams merged 98% more pull requests — while review time rose 91% and average PR size grew 154%. Follow-up data from 2026 is worse: production incidents per PR roughly tripled, and 31% more PRs now merge with no human review at all. The bottleneck didn't disappear when agents started working the night shift. It moved to 9 a.m., concentrated into the first ninety minutes of your day, and got a name: the morning review queue.

A polished eight-page design doc used to mean something before anyone read a word of it. The existence of the artifact was evidence: someone spent two weeks thinking about failure modes, arguing with themselves about tradeoffs, and pre-empting the objections they knew reviewers would raise. The document was a receipt for the thinking. Reviewers could triage by polish alone, because polish was expensive to fake.

That correlation is now dead. An agent produces a comprehensive, well-structured, diagram-laden RFC in under an hour — complete with an alternatives-considered section for alternatives nobody actually considered. The artifact survived; the signal it carried did not. And most engineering organizations are still running a review process that was designed, implicitly, around the old cost of writing.

"What did you do yesterday?" is the first question of every standup, and on a team that runs agent fleets overnight it has become impossible to answer honestly. The literal answer is: I wrote three prompts, went home, and woke up to eleven pull requests, four of which I have not read yet. The person reciting their update is not lying on purpose. The ritual is lying for them, because it was built around an assumption that no longer holds — that the unit of work is a human doing one thing at a time, serially, during business hours.

That assumption is load-bearing. It holds up the burndown chart, the sprint commitment, the velocity number, the "blocked / in progress / done" columns, and the whole choreography of who-tells-whom-what-when. Pull the assumption and the artifacts don't gracefully degrade. They keep producing numbers that look authoritative and mean nothing. A team can have a beautiful burndown and a green sprint while half its actual throughput happened between midnight and 6 a.m., attributed to no one, reviewed by no one, and reflected in no ceremony.

Your dashboard has never looked better. Pull requests merged are up almost double. Commits per engineer are climbing. Lines of code are pouring in. Every activity chart points up and to the right, and the AI coding tool you rolled out six months ago looks like the best money the org has spent all year.

Then you check the one number nobody puts on a slide: how long it takes for a customer to actually get something they asked for. It hasn't moved. In some quarters it's worse. The team is producing more of everything except the thing the business buys.

This is the velocity mirage. The metrics that are easy to count went up, and the outcome that matters quietly went the other way. It is one of the most expensive measurement failures in engineering right now, because it looks exactly like success.

A year ago your team adopted a PR description template. It had a ## Summary, a ## Changes, a ## Test plan, and a row of checkboxes. Reviewers loved it: every PR had context, every PR had a test plan, every PR had structure. Six months later the coding agent learned to fill it in. Now every PR has a ## Summary, a ## Changes, a ## Test plan, and a row of checkboxes — and reviewers no longer read past the title. The format that once focused attention now signals that there is nothing worth focusing on. The structure outlived the signal it carried.

This is not a code-quality problem. The code in those PRs is often fine. The problem is that the act of writing a description has been amputated from the act of thinking about the change, and the description is the artifact reviewers used to triage what to spend their finite attention on. When that artifact becomes uniformly formatted, plausibly worded, and indistinguishable from every other PR, the reviewer's attention triage breaks. The system that used to surface the unusual now flattens everything into the same shape.

The productivity claim around coding agents is that they remove the typing bottleneck. The bottleneck the engineer actually hits in practice is different. The engineer can no longer hold the system in their head, because the agent is editing files faster than the engineer can read them, writing tests faster than the engineer can reason about coverage, and refactoring abstractions faster than the engineer can verify they still type-check at the design level rather than just the compiler level.

The tight inner loop — hypothesize, change, observe, refine — that defines competent engineering quietly collapses into a different loop. The engineer is now reviewing agent output rather than building intuition about the system. A METR randomized controlled trial from mid-2025 found experienced open-source developers were 19% slower on familiar codebases when using AI assistants, while reporting they felt 20% faster. The 39-point gap between perceived and actual productivity is not a measurement error. It is the sound of comprehension being silently traded for throughput.

For two decades the bottleneck in software engineering was writing code. We optimized IDEs, autocompletion, refactoring tools, and frameworks to make typing cheaper. We won. Now the bottleneck moved one step downstream: writing is cheap, and reading is expensive. The PR-bot can spin up ten implementation attempts in parallel and open ten pull requests against your repo before you finish your morning coffee. Your reviewers cannot.

The rate limiter for AI-assisted software delivery is no longer the model's tokens per second. It is the number of human eyes you can put on a diff per day. And when those eyes get overwhelmed, you do not get a graceful degradation — you get rubber stamps. Code merges with LGTM 🚀 on top of code that nobody actually read. A senior engineer approves an AI-written patch that another AI tool already reviewed, and three weeks later a data-inconsistency bug eats forty hours of someone's life. Surface correctness is not systemic correctness, and a green pipeline is not understanding.

The ROI pitch for AI coding tools is irresistible on paper: developers complete tasks 55% faster in controlled experiments, ship 98% more pull requests, and report saving 3.6 hours per week. But when organizations look at their actual delivery metrics — bug rates, release cycle times, incident frequency — the numbers barely move. Something is absorbing all those gained hours, and it's not hard to find.

AI generates code in seconds. Engineers still review it at the same pace they always have.

The benchmark numbers tell a strange story. On SWE-bench Verified, multiple agent products running the same underlying model — Auggie, Cursor, Claude Code, all on Opus 4.5 — produced wildly different results. Auggie solved 17 more problems out of 731 than its closest peer despite the identical brain. The gap was scaffolding: how the agent was prompted, what context it was given, which tools it could call, and what the harness did when it got confused. The model is a commodity. The scaffolding around it is the product.

This is the same realization mature engineering teams reached about junior engineers a decade ago. A bright graduate doesn't ship value because the model is good. They ship value because the README is current, the test suite is fast, the code review rubric catches the same six mistakes every time, and someone wrote a CONTRIBUTING.md that names the constraints. Strip that scaffolding away and the same person produces locally coherent, globally wrong code that breaks production invariants the team didn't know to write down.

A senior engineer pulls up an agent-authored PR. The diff is clean. The tests pass. The naming is consistent. They skim it, leave a thumbs-up, and merge. Two months later, a different senior engineer is rewriting that module because the abstraction it introduced quietly leaks state across three call sites and the test suite never noticed because it asserted what the code does, not what the spec required.

This pattern is the dominant failure mode of code review in 2026. The reviewer instincts that worked on human-authored PRs — probe the author's intent, look for the bug they didn't think of, check whether the test reflects the design — break down on agent PRs because the bugs cluster in different places and the artifacts the reviewer sees are no longer the artifacts that matter.

The data backs the intuition. CodeRabbit's December 2025 analysis of 470 GitHub PRs found that AI-co-authored code produces about 1.7× more issues than human-authored code, with logic and correctness errors at 1.75×, security findings at 1.57×, and algorithmic and business-logic errors at 2.25× the human rate. Critical issues climb 1.4× and major issues 1.7×. The diffs read fluent, and that fluency is precisely the problem.

The dashboard says your engineers accepted 45% of AI suggestions last quarter. Leadership reads that as "45% of a developer's time saved" and signs the renewal. The engineers, meanwhile, are quietly rewriting half of what they accepted, debugging the other half, and wondering why their sprints still feel the same length. Both sides are looking at the same number. Only one of them is looking at the right number.

The most quoted study of 2025 should have ended the vendor-dashboard era on its own. METR measured experienced open-source maintainers working on real issues in their own repos, with and without AI. The developers predicted AI would speed them up by 24%. After the experiment they still believed AI had sped them up by 20%. The stopwatch said they were 19% slower. A thirty-nine-point gap between the story and the data — and the story is what went into the quarterly review.