93% of Developers Use AI Daily, But Productivity Plateaued at 10% Since 2023—What Are We Actually Measuring?

General
1

Body:

I’ve been tracking our engineering team’s AI adoption metrics for 18 months now at a Fortune 500 financial services company, and I’m seeing a paradox that’s making me question everything we think we know about productivity measurement.

The Numbers Don’t Add Up

Here’s what the data shows across the industry:

  • 93% of developers now use AI coding assistants daily (source)
  • 84% report they use or plan to use AI tools in development
  • Yet productivity gains have plateaued at roughly 10% since 2023

At my company, we’ve rolled out GitHub Copilot, Claude, and other AI tools to all 40+ engineers on my teams. Adoption hit 95% within 3 months. Developers love these tools and report feeling faster. But when I look at our delivery metrics—sprint completion rates, feature shipping velocity, incidents per release—they’re essentially flat.

What Are We Measuring Wrong?

I keep coming back to this: if nearly everyone is using AI daily, why isn’t productivity moving? Three hypotheses:

1. We’re measuring outputs, not outcomes

  • More PRs merged ≠ more customer value delivered
  • Faster code writing ≠ faster product delivery
  • Lines of code is a vanity metric we should have abandoned years ago

2. We’re ignoring the hidden costs

  • Code review times are up ~90% in our team
  • Debugging AI-generated code takes longer than writing from scratch
  • Junior engineers report lower comprehension of what they’re shipping

3. Productivity was never the right word

  • Maybe AI makes us more capable, not more productive
  • We’re tackling more ambitious features with the same team size
  • The bottleneck shifted from writing code to defining what to build

The Financial Services Reality

In regulated industries like ours, there’s another dimension: AI code requires more scrutiny. We can’t just ship code we don’t fully understand. Our compliance team requires human review of all financial logic, regardless of how it was generated.

So AI helps us write the boilerplate faster, but the critical path—understanding business requirements, ensuring regulatory compliance, validating edge cases—hasn’t changed.

What Should We Be Measuring Instead?

I’m starting to think we need different metrics entirely:

  • Time from idea to validated customer value (not just shipped code)
  • Developer satisfaction and learning (not just velocity)
  • Capability expansion (what new problems can we tackle now?)
  • Quality per unit of effort (not volume of output)

But I’m genuinely curious: for those of you who’ve adopted AI coding tools, what metrics actually moved in your organizations? And for those where productivity gains plateaued like mine, what do you think we’re missing?

Is this an AI measurement problem, or a fundamental misunderstanding of what productivity means in software engineering?

2

Luis, this resonates hard from the product side. I’ve been seeing the exact same paradox at our Series B SaaS startup, but from a different angle.

Engineering Is Faster, Product Discovery Isn’t

Here’s what I’m observing: our engineering team is shipping 40% more features per quarter since we rolled out AI coding assistants. But our product success rate—features that actually drive customer adoption and retention—has stayed flat or maybe even declined slightly.

The problem? Engineering outpaced product discovery.

We’re shipping faster than we can validate. Our customer research cycle, beta testing, and feedback loops run at the same speed they always did. But now we have 3 features in market before we’ve learned whether feature #1 was even the right thing to build.

The JTBD Lens

From a Jobs-to-be-Done perspective, the job of “writing code” got faster, but the job of “solving customer problems” didn’t. And that second job is what actually matters for business outcomes.

Your hypothesis #3 hits the nail on the head: the bottleneck shifted. In 2023, engineering capacity was the constraint. In 2026, product clarity and customer validation are the constraints.

What We Changed

After recognizing this, we made two changes:

  1. Feature flagging everything - We ship code fast but release it slowly, using feature flags to control rollout while we validate
  2. Measuring learning velocity, not shipping velocity - How fast can we learn whether this feature solves a real customer problem?

The uncomfortable truth: AI made engineers faster, but it didn’t make us smarter about what to build. That’s still a human problem, and it’s where we should be focusing our measurement and investment.

Maybe the 10% productivity plateau is actually the ceiling until we solve the product discovery bottleneck?

3

This is the conversation every CTO should be having right now. I’ve been wrestling with this exact question at my company for the past year.

The Executive Expectation Gap

Here’s what makes this especially challenging from the CTO seat: executives expect 3× productivity gains, engineers report 10%, and business outcomes show 0%.

Our board saw headlines about “AI will 10x developers” and now they’re asking why we haven’t reduced headcount or dramatically accelerated our roadmap. Meanwhile, I’m seeing exactly what Luis described: high adoption, developer satisfaction up, but delivery velocity essentially unchanged.

Where the Time Actually Went

I had our engineering managers track this for 6 weeks. Here’s where the “productivity gains” disappeared:

  1. Code review burden → +90% time per PR (matches Luis’s experience)
  2. Debugging AI-generated edge cases → +45% time in testing phase
  3. Architecture discussions → More complex features attempted = more design time needed
  4. Compliance and security reviews → We can’t skip these regardless of how code was generated

The net result? Individual developers write code faster, but team velocity is unchanged because the bottlenecks are elsewhere.

The Metric I Actually Trust

The one number that did move for us: time-to-hello-world for new engineers. Onboarding time dropped by 30% because new hires can lean on AI to understand our codebase faster.

But that’s a one-time efficiency gain, not ongoing productivity improvement.

My Controversial Take

I think we’ve been sold a productivity story, when what AI actually delivers is capability expansion. We can tackle more ambitious technical challenges with the same team. We can explore more architectural options. We can refactor more aggressively.

But if you measure productivity as “output per engineer,” you’re going to be disappointed.

The right question isn’t “Why didn’t productivity improve?” It’s “What new capabilities do we have that we didn’t before?” And that’s much harder to quantify.

Luis, I’d love to hear if your finance industry peers are seeing similar patterns. Is this a regulated industry thing, or universal?

4

Okay this is wild because I’m seeing the exact same pattern but from a completely different angle—design systems and tooling.

The “Faster But Not Better” Problem

I lead design systems for 3 product teams, and we adopted AI tools (GitHub Copilot, v0, Claude for code generation) about 9 months ago. Here’s what happened:

What got faster:

  • Component scaffolding (basically instant)
  • Converting designs to code (40% time reduction)
  • Writing documentation and prop tables (60% faster)

What didn’t get faster:

  • Design-engineering alignment conversations
  • Accessibility audits and fixes
  • Performance optimization
  • Understanding why a component behaves a certain way

It’s like we automated the easy parts but left the hard parts untouched. And in software, the hard parts are where the actual value is.

The Comprehension Debt Problem

There’s this concept I’ve been reading about called “comprehension debt” (source)—code that works but nobody understands.

I’m seeing this in our design system components. Developers can generate a new component variant in minutes using AI, but when it breaks in production or doesn’t meet accessibility standards, they can’t debug it because they didn’t learn the underlying patterns.

So we’re building faster but not smarter. And that catches up with you eventually.

My Failed Startup Parallel

This reminds me of my failed SaaS startup. We had amazing no-code tools that let us ship features incredibly fast. But we were shipping the wrong things because we weren’t spending time understanding our customers.

Speed without direction is just… flailing.

Michelle’s point about capability expansion vs productivity is spot on. AI tools let us attempt more ambitious features, but attempting more ≠ succeeding more.

What Actually Matters

Maybe the 10% productivity plateau is because we’re still measuring the wrong thing. In design, we learned that speed of iteration matters less than quality of iteration.

Could the same be true for engineering? Maybe the right metric is “How many customer problems did we actually solve?” not “How many lines of code did we write?”

I know that’s harder to measure, but if 93% AI adoption only gives us 10% productivity gains, then either:

  1. The tools aren’t as good as advertised
  2. We’re measuring the wrong thing

I’m betting on #2.