I’ve been tracking our engineering velocity closely for the past nine months, and something isn’t adding up. Our entire team is using GitHub Copilot. PRs are flying. Commits are up 40%. Developers tell me they feel more productive than ever. Yet when I look at our actual release velocity—features shipped, customer value delivered—the needle has barely moved. Maybe 8-10% improvement, tops.
Then I saw the data, and it clicked: we’re not alone.
The Paradox in Numbers
Here’s what the research shows across the industry:
- 92.6% of developers use an AI coding assistant at least monthly
- 27% of production code is now AI-generated
- Individual coding tasks are 30-55% faster in controlled experiments
- Yet organizational productivity gains are stuck at ~10%
So what’s happening? How can near-universal adoption, with massive individual speed gains, translate to such minimal organizational impact?
The Bottleneck Just Moved
I think the answer is that coding was never really the constraint. Or rather, AI solved the coding speed problem but exposed everything else.
When developers write code 50% faster, what happens?
- PRs increase by 98% (creating review backlogs)
- Average PR size grows by 154% (making reviews harder)
- Bugs increase by 9% (creating QA bottlenecks)
- Security vulnerabilities spike—322% more privilege escalation paths in AI-generated code
We sped up one part of the assembly line and created pile-ups everywhere else.
Perception vs. Reality
Here’s the kicker: a recent METR study found that AI-assisted developers were actually 19% slower on real-world tasks—while they estimated they were 20% faster. That’s a 39-point perception gap.
This matches what I’m hearing from our team. They feel more productive. The dopamine hit of instant code suggestions is real. But are we actually shipping faster? Delivering more value? The data says no.
Are We Solving the Wrong Problem?
This makes me wonder: was coding speed ever the real bottleneck?
In my experience, the constraints are usually:
- Unclear requirements and product specs
- Architectural decisions and technical design
- Cross-team coordination and integration
- Customer validation and iteration
AI tools help me write code faster, but they don’t tell me what to build or why. They don’t clarify ambiguous requirements. They don’t resolve competing stakeholder priorities. They don’t help me say no to feature requests.
The Business Lens
From a product strategy perspective, this has huge implications:
-
ROI calculations are broken. We can’t assume “AI = faster shipping” anymore. The costs (quality issues, review overhead, technical debt) might outweigh the speed gains.
-
Process redesign is mandatory. If you speed up development without upgrading review, QA, and deployment processes, you just create bottlenecks downstream.
-
Measurement matters. Lines of code and PR velocity are vanity metrics. What matters is customer value delivered—and that requires a different measurement framework.
My Questions for You
I’m genuinely curious what others are experiencing:
- Are you seeing this pattern in your organizations—more code, same velocity?
- Have any teams successfully redesigned their processes to capture the AI speed gains?
- Is this just a learning curve (give it 2-3 more years) or a fundamental mismatch?
- Are we measuring productivity wrong, or are AI tools just not as transformative as we thought?
Would love to hear especially from engineering leaders who’ve been tracking this closely. What does your data show?
Sources: