AI Assistants Make Developers 30% Faster at Coding, But Only 8% Faster at Delivery—Where's the Bottleneck?

General
1

When my team started using AI coding assistants six months ago, our individual developer velocity metrics looked incredible. Pull request frequency shot up. Lines of code per sprint doubled. Engineers were thrilled—they felt like superheroes writing code at lightning speed.

Then I looked at our actual delivery metrics. Customer-facing feature velocity? Up only 8%. Time from idea to production? Barely budged. We had optimized one part of the system beautifully, but the overall outcome barely moved.

Turns out, we were experiencing a textbook case of what Thoughtworks documented: coding became roughly 30% faster with AI assistants, but delivery improvement was only 8% when you factor in testing, code review, environment waits, and dependency management.

The Math Doesn’t Lie

Here’s the uncomfortable truth: coding represents only about 15-25% of the software development lifecycle. Even if we made it infinitely fast—a 100% improvement—Amdahl’s Law tells us we’d see at most a 15-25% system-wide gain.

The other 75-85% is everything else:

  • Code review and approval cycles
  • Test suite execution time
  • Security scanning and compliance checks
  • Deployment pipeline delays
  • Cross-team dependency coordination
  • Environment provisioning and debugging

We poured AI into the smallest slice of the pie and wondered why we weren’t full.

Bottleneck Migration Is Real

What actually happened on my teams was bottleneck migration. We used to have 10-15 PRs per week hitting review. Now we have 30-40. Our code review capacity didn’t scale with our code generation capacity.

The numbers from Faros AI’s research are stark: teams with high AI adoption merge 98% more pull requests, but PR review time increased 91%. We created a massive pileup at the human approval stage.

And it’s not just volume—AI-generated code often requires more careful review. Veracode found that 45% of AI-generated code introduced OWASP Top 10 vulnerabilities. Another study by CodeRabbit showed AI code contains 2.74x more security vulnerabilities than human-written code.

So we’re reviewing more code, and each review requires more scrutiny. No wonder delivery didn’t speed up.

What Actually Needs to Change

I’m now convinced that capturing the full value of AI coding assistants requires systemic changes, not just better prompts:

  1. Automated code review infrastructure: If AI can write code, we need AI-powered static analysis, security scanning, and compliance checking that runs instantly. Manual review should focus on architecture and business logic, not catching bugs AI could flag.

  2. Test infrastructure investment: Our test suites weren’t built for 98% more PRs. We need parallel test execution, better test isolation, and faster feedback loops. If tests take 2 hours to run, coding speed is irrelevant.

  3. Deployment automation: We still have manual deployment approval gates and environment coordination that takes days. The deployment pipeline needs to scale with code velocity.

  4. Dependency and integration process: Cross-team coordination, API contract negotiation, and integration testing are now the long poles. We need better async collaboration tools and integration test automation.

  5. Requirements and discovery process: Product and design processes haven’t accelerated. We’re building the wrong things faster, which doesn’t help customers.

The Strategic Question

As a CTO, I’m asking myself: Did we invest in the right productivity improvement?

AI coding assistants are table stakes now—92.6% adoption, 27% of production code AI-generated across the industry. But multiple research studies converge on roughly 10% organizational productivity gains despite the hype.

That’s not nothing, but it’s not the revolution we hoped for. The revolution requires rethinking the entire SDLC, not just the coding phase.

Where are you seeing bottlenecks post-AI? Is it code review? Testing infrastructure? Deployment pipelines? Something else entirely?

And more importantly: What are you actually changing to capture the velocity gains, beyond just adopting better AI tools?

2

This hits home hard, Michelle. We’re living this exact paradox in financial services right now.

Our PR volume tripled after we rolled out AI coding assistants to the team. Developers loved it—they were shipping features faster than ever on their individual boards. But then compliance and security review became a nightmare.

The Compliance Chokepoint

In fintech, every code change touching customer data or payment flows needs security review and regulatory compliance checks. Pre-AI, we had maybe 12-15 PRs per week that needed this level of scrutiny. Manageable with 2-3 senior engineers rotating through security review duty.

Now we’re at 40-50 PRs per week. Same 2-3 reviewers. And here’s the kicker: AI-generated code requires more thorough review, not less.

We’re finding:

  • More edge cases that weren’t considered
  • Incomplete error handling (AI writes the happy path beautifully, forgets failure modes)
  • Subtle security issues that would sail through automated SAST tools
  • Compliance gaps (AI doesn’t understand SOC2 requirements or PCI DSS)

So review time per PR went up 30-40%, while PR volume tripled. You do the math. :exploding_head:

What We’re Trying

We’re experimenting with a few approaches:

  1. AI-powered compliance scanning: Working with vendors to train models on our specific regulatory requirements. Early results are promising—catching about 60% of compliance issues automatically.

  2. Tiered review process: Not every PR needs senior security review. We’re routing based on risk classification (data sensitivity, customer impact, regulatory scope).

  3. Review SLAs: If a PR sits in review for more than 2 days, it auto-escalates. Forces us to resource the bottleneck properly.

But honestly? We’re still figuring it out. The delivery velocity gains you mentioned (8%) match our experience almost exactly.

Question for the group: Has anyone found AI-powered code review tools that actually understand domain-specific compliance requirements (financial services, healthcare, etc.)? Most tools I’ve seen are generic security scanners that miss the regulatory nuance.

3

Michelle, this resonates so deeply! And honestly, I think there’s an even earlier bottleneck nobody’s talking about: we’re still figuring out what to build.

At my last startup, we had the exact same experience. Developers were flying—AI helped us prototype features in days instead of weeks. We felt unstoppable. :sparkles:

Then we shipped three “fast” features that customers barely used. All that velocity, none of the value.

Design and Discovery Didn’t Get Faster

Here’s what I realized: coding speed doesn’t fix unclear requirements or weak product discovery.

The bottleneck I see isn’t just in code review or testing—it’s in the messy, human work that happens before coding:

  • Talking to users to understand their actual problems
  • Designing solutions that fit real workflows (not just what’s easy to build)
  • Iterating on prototypes and getting feedback
  • Aligning stakeholders on priorities

None of that got 30% faster. We’re still doing whiteboard sessions, Figma design reviews, user interviews, A/B tests. AI can’t replace that work—at least not yet.

The “Building the Wrong Thing Faster” Problem

When I was running my startup, we had this painful moment where we shipped a beautifully-coded feature (thanks, AI!) that completely missed the user’s actual workflow. We built what we thought they needed, not what they actually needed.

Fast coding made it worse, not better. We got to “oops, this doesn’t work for users” in record time. :sweat_smile:

Your point about “building the wrong things faster doesn’t help customers” is spot-on. And honestly? Design iteration cycles are still slow. Getting design feedback, doing usability testing, iterating on interaction patterns—that’s still very human and very time-consuming.

What Actually Helped

At my current role leading design systems, we’ve learned:

  1. Front-load discovery: Spend more time in Figma and user testing before development starts. AI makes coding cheap, so let’s make sure we’re coding the right thing.

  2. Prototype without code: Use Figma prototypes, no-code tools, even clickable PDFs. Validate concepts before writing production code.

  3. Smaller batches: Ship smaller changes more often. AI makes this easier, but only if your deployment and review process can keep up (which circles back to Michelle’s point about infrastructure).

The bottleneck for me isn’t code—it’s alignment between what we’re building and what users actually need. And that’s still a slow, iterative, human process.

Has anyone found ways to accelerate product discovery or design validation? Or are we all just… talking to users and iterating like we did in 2015? :blush:

4

Maya just articulated something I’ve been wrestling with for months: coding velocity is meaningless if we’re building the wrong features.

From a product perspective, this whole AI coding boom has created a dangerous trap. We can ship features so fast now that we’ve accidentally optimized for output instead of outcomes.

The Metrics Mismatch

Michelle, you mentioned delivery velocity being up only 8%. But I’d argue that’s the wrong metric entirely. What we should be measuring is:

  • Customer value delivered (not features shipped)
  • Product-market fit indicators (engagement, retention, NPS)
  • Revenue impact (does faster coding translate to revenue growth?)

In my experience? The answer is often “no.” :chart_decreasing:

We’re shipping more features, but our engagement metrics are flat. We’re moving faster, but we’re not moving in the right direction.

The Real Bottleneck: Discovery and Validation

The bottleneck isn’t code review or testing infrastructure. It’s figuring out what problem to solve and validating we got it right.

At my current company, our product discovery cycle looks like this:

  1. Customer interviews and research: 2-3 weeks
  2. Design and prototyping: 1-2 weeks
  3. Stakeholder alignment and prioritization: 1 week
  4. Development: 1-2 weeks (used to be 3-4 weeks pre-AI)
  5. User testing and iteration: 1-2 weeks

AI shaved 1-2 weeks off step 4. That’s great! But the total cycle time only improved by ~15-20%, not 30%.

And here’s the kicker: if we get steps 1-3 wrong, step 4 being faster just means we fail faster. Which is only valuable if we learn faster too—and learning cycles haven’t accelerated.

What Actually Matters Now

I’m increasingly convinced that AI coding assistants have shifted the strategic advantage from “build fast” to “discover fast.”

The teams winning now are those who:

  1. Talk to customers obsessively and validate assumptions early
  2. Run rapid experiments (A/B tests, beta features, prototypes)
  3. Kill features quickly when data shows they’re not working
  4. Ruthlessly prioritize based on customer value, not engineering ease

Faster coding makes all of this possible—but only if you have the discipline to not just ship everything you can build.

Question for the engineering leaders here: How do you balance letting engineers use AI to ship fast vs ensuring they’re shipping the right things? Do you have better discovery processes, tighter feedback loops, or just more discipline saying “no” to features?

Because from where I sit, the bottleneck isn’t in your systems—it’s in our product strategy and customer understanding. And I don’t think AI is going to solve that anytime soon. :thinking:

5

This conversation is exactly what I needed to read today. Michelle, Luis, Maya, David—you’re all hitting on different pieces of the same puzzle.

From an organizational perspective, I think we’re all describing different manifestations of the same root cause: our infrastructure and processes weren’t built for this velocity.

The Infrastructure Gap

When I joined my current EdTech company, we had:

  • A test suite that took 90 minutes to run on CI
  • Manual QA approval gates that took 1-2 days
  • Deployment windows twice a week (Tuesday/Thursday only)
  • Staging environment conflicts (teams waiting for availability)

AI made our developers 3x faster at writing code. But they were still waiting 90 minutes for test results, 1-2 days for QA approval, and could only ship twice a week.

The bottleneck wasn’t coding. It was everything around coding.

What We’re Investing In

Luis mentioned tiered review processes and compliance scanning. David talked about better discovery and validation. Maya highlighted design iteration. All of those are critical.

But from an engineering org perspective, here’s what actually moved the needle for us:

1. Test Infrastructure Overhaul

  • Parallelized our test suite: 90 min → 12 min
  • Invested in test stability: flaky test rate from 15% → 2%
  • Set up preview environments: every PR gets its own environment for testing
  • Cost: ~$200K in infrastructure + 2 eng-months of work
  • Impact: Developers get feedback in minutes, not hours

2. Automated Quality Gates

  • AI-powered security scanning on every PR (catches ~70% of issues automatically)
  • Automated accessibility testing
  • Performance regression detection
  • Cost: ~$50K/year in tooling
  • Impact: QA team focuses on user experience, not catching bugs

3. Continuous Deployment

  • Moved from twice-weekly to 50+ deploys per week
  • Automated rollback on error spikes
  • Feature flags for gradual rollouts
  • Cost: eng time to build the pipeline
  • Impact: Code ships within hours of merge, not days

4. Better Observability

  • Real-time monitoring and alerting
  • User session replay for debugging
  • Feature flag analytics to measure impact
  • Cost: ~$80K/year in tooling
  • Impact: Faster debugging, data-driven decisions

Total investment: ~$330K + significant eng time.
Result: Delivery velocity actually improved 35-40%, not just 8%.

The People Side Matters Too

But David’s point about product discovery is critical. We also had to change how we work:

  • Shorter planning cycles: 2-week sprints → 1-week iterations
  • Embedded product in engineering squads: Daily collaboration, not weekly handoffs
  • Hypothesis-driven development: Every feature has a success metric defined upfront
  • Kill metrics: We explicitly track what we decided NOT to ship or what we sunset

Maya’s point about design iteration is spot-on too. We started doing design reviews during planning, not after development starts. AI makes code cheap, so validate the design first.

The ROI Question

Michelle, you asked if we invested in the right productivity improvement. I think the answer is: AI coding assistants are necessary but not sufficient.

They’re table stakes now—everyone has them. The competitive advantage comes from actually capturing the velocity gains by fixing the rest of the SDLC.

Teams that just adopt AI tools without investing in test infrastructure, deployment automation, and better product processes will see exactly what you saw: 8% improvement.

Teams that use AI adoption as a forcing function to modernize their entire delivery pipeline? They’ll see 30-40% gains.

The real question isn’t “should we use AI coding tools?” It’s “are we willing to invest in the infrastructure and process changes needed to actually benefit from them?”

And honestly, a lot of companies aren’t. They want the productivity gains without the infrastructure investment. Which is why we’re all stuck at ~8-10% improvement. :100: