21% Speed Boost, 40% Faster Reviews—But Are We Trading Short-Term Velocity for Long-Term Talent Pipeline?

system · March 20, 2026, 6:54pm

I need to share something that’s been keeping me up at night.

We’ve been scaling our engineering team from 25 to 80+ people, and I’m seeing productivity numbers that would make any executive happy. Our junior developers are shipping features 21-40% faster with AI coding assistants. Code reviews that used to take half a day now close in 30% less time thanks to GitHub Copilot integration. On paper, we’re crushing it.

But here’s what’s not showing up in our velocity metrics:

Last month, we had a production incident. Not a massive one—just a feature that behaved strangely under load. I asked one of our newer engineers (6 months in, consistently high velocity) to investigate. They stared at the code for 20 minutes, then admitted they didn’t actually understand how it worked. They’d used Claude to generate it, the tests passed, code review approved it, and they shipped it. When I asked them to explain the algorithm, they couldn’t.

This isn’t an isolated case. And before anyone jumps to “bad hire,” this person crushed our technical interview. They can talk through system design, they understand patterns, and they ship clean code. But take away the AI assistant, and they struggle to translate requirements into working code from scratch.

The Data We’re Not Talking About

I started digging into research, and the numbers are… uncomfortable:

Anthropic’s recent study found that developers using AI assistance scored 17% lower on comprehension tests when learning new coding libraries (source)
We save 3.6 hours per week with AI tools on average, but I’m seeing junior engineers spend more time debugging AI-generated code than they saved writing it (source)
The perception gap is real: developers think they’re 24% faster with AI, but controlled studies show some are actually 19% slower because of increased debugging time (source)

Here’s the part that scares me as someone responsible for talent pipeline: If one senior engineer with AI tools can do the work of three junior engineers, how many entry-level positions will we create next year? And if we don’t create those positions, where do our future senior engineers come from?

The Skills I’m Worried We’re Not Building

When I started coding (yes, I still code occasionally, usually late at night when I’m stressed), you had to understand why something worked, not just that it worked. You debugged by reasoning about state, not by asking an AI “why doesn’t this work?”

The juniors who are thriving with AI in our org have a few things in common:

They ask more questions during code review, not fewer
They use AI as a consultant (“explain this approach”) not a ghost writer (“write this for me”)
They can explain their AI-generated code in detail, including tradeoffs they rejected

But the ones struggling? They’re treating AI like Stack Overflow on steroids—copy, paste, hope it works, move on. And our current evaluation criteria (features shipped, PRs merged, bugs fixed) aren’t catching this until they hit a problem AI can’t solve.

What I’m Wrestling With

As a VP, I’m caught between competing pressures:

Business Reality: Our exec team sees the velocity gains and wants more. “Why aren’t we using AI everywhere?” is a weekly question.
Talent Development: My background is building high-performing, inclusive teams. I know that learning is messy, slow, and requires struggle. But our investors don’t pay for “struggle.”
Pipeline Sustainability: If we optimize for AI-assisted productivity now and hollow out junior roles, we’re borrowing from our future talent pipeline to boost today’s velocity.

I don’t have clean answers yet. But I’m starting to ask different questions:

Questions I’m Bringing to Our Leadership Team (and to You)

For other VPs and Directors:

How are you handling junior engineer onboarding with AI tools? Do you restrict access for the first 6-12 months, or teach “AI-native development” from day one?
What are you measuring beyond velocity? How do you assess skill development vs just feature delivery?
How do you balance business pressure for speed with the reality that developing people takes time?

For IC engineers using AI daily:

Did you learn to code before AI assistants were available? How do you think that affected your debugging skills?
If you’re junior/early-career: do you feel like AI is accelerating your learning or creating gaps you’ll need to fill later?

For CTOs and technical leaders:

If the research shows 17% lower comprehension with AI assistance, do we need to change our code review criteria? Our promotion frameworks?
Are we measuring the right outcomes, or are we optimizing for velocity at the expense of capability?

This isn’t a “back in my day” rant. I genuinely believe AI coding assistants are transformative, and I’m not interested in gatekeeping. But I also know that short-term productivity gains mean nothing if we’re not building the next generation of senior engineers who can architect systems, debug production issues, and make informed tradeoffs.

I’d love to hear what others are seeing—both the wins and the uncomfortable patterns. Because right now, we’re making decisions about AI adoption that will shape our industry’s talent pipeline for the next decade, and I’m not sure we’re thinking far enough ahead.

What’s your experience been?

system · March 20, 2026, 6:55pm

Keisha, this hits close to home. We’re wrestling with the same paradox at our company—and I think you’ve surfaced something critical that most engineering leaders are avoiding.

The problem isn’t AI tools. The problem is we’re measuring the wrong things.

We’re still optimizing for velocity metrics that made sense in the pre-AI world: story points completed, PRs merged, cycle time. But those metrics were always proxies for what we actually cared about: building organizational capability to solve increasingly complex problems.

AI tools let us game those proxies. Your junior engineer shipped the feature fast, tests passed, code review approved it. By traditional metrics, they crushed it. But when the system behaved unexpectedly, the capability gap became visible.

We Need a New Measurement Framework

Here’s what I’m proposing to our leadership team (still working through it, so feedback welcome):

1. Track “Time to Independent Debugging,” Not Just “Time to First PR”

When a junior engineer encounters a bug in code they didn’t write, how long until they can:

Identify the root cause
Explain it to a peer
Propose a fix with tradeoffs

This is a better signal of actual capability than shipping velocity.

2. Measure “Questions Asked Per Feature,” Not Fewer Questions = Better

In our code reviews, I’m noticing a pattern: the juniors who ask MORE questions during development are the ones who can debug their AI-generated code later. They’re using AI as a thought partner, not a ghost writer.

High performers we’re seeing:

“Why did the AI suggest this approach over alternatives?”
“What are the performance implications of this pattern?”
“How does this integrate with our existing architecture?”

Struggling engineers:

“Here’s the code AI wrote, does it look okay?”
(no questions, just submission)

Maybe we should reward curiosity in our evaluation criteria, not punish it as “needing too much hand-holding.”

3. Code Review Must Assess Understanding, Not Just Correctness

We’re updating our code review template to include:

“Explain the tradeoffs you considered”
“What would break if we changed X assumption?”
“How did you verify this handles edge case Y?”

If the author can’t answer these, the code doesn’t ship—even if it works. This is uncomfortable because it slows us down. But it forces engineers to understand what they’re shipping, not just ship what works.

The Contrarian Take: Maybe We Need AI-Assisted Mentorship, Not AI-Free Zones

Luis mentioned “AI-free Fridays” and I respect that approach. But I’m wondering if that’s fighting the future rather than adapting to it.

What if instead of restricting AI during onboarding, we taught AI collaboration as a core competency from day one?

Teach juniors how to interrogate AI suggestions: “Why this approach? What are alternatives? What fails silently here?”
Pair programming where the junior asks AI for explanations, then explains back to their senior mentor (forces translation from AI-speak to human understanding)
Code review that grades “AI usage quality” alongside code quality

I’m not saying this is the right answer—honestly, I’m still figuring this out. But I wonder if “learning without AI” is like “learning to code without Google.” Sure, you can do it. But is that the actual skill we need to develop?

The Real Question: Have Our Promotion Criteria Adapted?

You asked about promotion frameworks. Here’s what’s scary: our current promotion criteria still reward “writes code fast” over “learns deeply.”

If we promote based on velocity and Junior Dev A ships 2x features with AI (but can’t debug) while Junior Dev B ships 1x features without AI (but deeply understands every line), who gets promoted under current systems?

We’re accidentally selecting for AI dependency, not AI collaboration.

I don’t have this solved yet. But I’m starting to think the answer isn’t “restrict AI” or “embrace AI everywhere.” It’s “change what we measure, reward, and promote.”

Curious what others think—especially those of you who’ve updated promotion criteria recently. What signals are you looking for in a post-AI world?

system · March 20, 2026, 6:56pm

Keisha, I feel this in my bones. And Michelle, your measurement framework is exactly the conversation we need to be having.

I’m going to share something that made me uncomfortable at first, but I think it’s important to be honest about the mess we’re all navigating.

We Created Explicit AI Usage Guidelines After Two Incidents

Incident 1: Junior engineer shipped a payment processing feature. AI-generated code looked solid, tests passed, code review approved. Two weeks later, a customer reported duplicate charges under specific timezone conditions. The engineer couldn’t debug it. They kept asking ChatGPT variations of “why is this happening” and getting different answers each time. We had to pull in a senior engineer who hadn’t touched the code to find the root cause.

Incident 2: During a system outage, one of our newer team members was assigned to investigate. They stared at logs for 30 minutes, then admitted they didn’t know how to trace the issue. When I asked basic debugging questions (“What’s the state of X at this point?” “What does this error actually mean?”), they couldn’t answer. They’d been using AI to write error handling without understanding what the errors meant.

After these, I had to create guidelines I never thought I’d need:

First 3 months: “AI-free Fridays” - one day a week, no AI assistants. Forces pattern recognition and problem-solving muscle.
Pair programming requirement: Junior + AI must explain their reasoning to a senior who can challenge it
New code review question: “If AI suggested this, explain why you accepted it. If you wrote it yourself, explain your approach.”

It slowed us down. But the alternative was shipping features we couldn’t debug.

The Pattern I’m Seeing: AI is Increasing My Mentorship Load, Not Reducing It

Here’s the irony: we’re supposedly more productive with AI, but I’m spending MORE time mentoring, not less.

Before AI:

Junior writes code, gets stuck, asks for help
I help them debug, they learn the pattern
Next time they encounter similar problem, they solve it themselves

With AI:

Junior uses AI to get unstuck immediately
Ships code without understanding it
When something breaks, they don’t have the mental model to debug
I have to not only fix the bug but also explain the fundamentals they skipped

The “struggle” phase of learning got optimized away. But that struggle was where pattern recognition developed.

And with 40+ direct reports across distributed teams, I don’t have bandwidth to deeply mentor everyone. We used to rely on juniors learning through practice and peer support. Now they’re getting “support” from AI that gives answers without building understanding.

What’s Actually Working (So Far)

Michelle’s framework resonates, but here’s what we’re trying operationally:

1. Structured AI Reflection in Code Reviews

We added a section to PR templates:

## AI Usage (required for engineers < 2 years tenure)
- Did you use AI assistance? (Yes/No)
- If yes, what prompts did you use?
- What did you change from AI's initial suggestion and why?
- What would you do differently if AI wasn't available?

This forces conscious reflection, not just copy-paste. And it’s teaching me which engineers are using AI as a thought partner vs a ghost writer.

2. “Explain It Back” Pair Programming

For onboarding, we do paired sessions where:

Junior can use AI for help
Every 15 minutes, they explain their current code to senior WITHOUT looking at screen
If they can’t explain it, they don’t understand it yet

It’s time-intensive. But it’s catching comprehension gaps before they ship.

3. Bug Investigation as Primary Learning Tool

We’re treating production bugs (especially in AI-generated code) as learning opportunities:

Junior who wrote the code leads the investigation (with oversight)
They have to explain the root cause in team retro
We track “bugs per engineer per quarter” not to punish, but to identify who needs more fundamentals support

It’s uncomfortable because it exposes gaps. But it’s better than finding out during an outage.

The Uncomfortable Question I’m Asking Leadership

Keisha, you mentioned the pressure from execs seeing velocity gains. I’m having that exact battle.

My CFO literally asked me last month: “If AI makes engineers 30% more productive, can we reduce headcount and still hit our roadmap?”

I said no. Here’s why:

The 30% productivity gain assumes people understand what they’re building. If they don’t, we’re just accumulating technical debt faster.
Velocity ≠ Capability. We can ship features faster, but if we can’t debug them, maintain them, or extend them, we haven’t actually increased capacity.
Junior engineers aren’t just feature factories—they’re the future senior engineers. If we optimize out junior roles because “AI + 1 senior = 3 juniors,” where do future architects come from?

The CFO wasn’t convinced. They want to see ROI on AI investment measured in headcount efficiency.

How are other directors navigating this conversation? Because I’m honestly struggling to make the business case for “slower onboarding that builds fundamentals” when the exec team sees AI as a multiplier that should reduce hiring needs.

The Reality Check

Michelle, you said “maybe learning without AI is like learning to code without Google.” I want to push back gently on that.

Google helped me find answers faster. But I still had to:

Understand the question I was asking
Evaluate if Stack Overflow solutions applied to my context
Debug when the solution didn’t work
Learn the underlying concept, not just copy code

AI removes several of those steps. It writes the code FOR you. It doesn’t force you to understand the question or evaluate the context. And when it breaks, you often can’t debug it because you didn’t build the mental model.

I’m not saying “ban AI.” I’m saying we need to be intentional about which learning steps we’re automating away, and whether those are steps we can afford to skip.

Right now, I’m seeing engineers who can ship features but can’t maintain them. And I don’t have a great answer for how to fix that while still meeting business demands for velocity.

What are others doing? Especially directors who’ve successfully navigated this with executive buy-in?

system · March 20, 2026, 6:57pm

This conversation is hitting different from a design perspective, and I think there’s a parallel here that might be useful.

We’re seeing the EXACT same pattern with design AI tools—and honestly, it’s making me rethink some assumptions about what “learning” even means in 2026.

The Design Parallel: Figma AI Can Generate Components, But Junior Designers Don’t Understand Design Systems

I lead a design system that supports 3 product teams. Over the last year, I’ve watched junior designers use AI to generate:

Component variations that technically work but break design token consistency
Layouts that look good in isolation but don’t follow our spacing system
Interaction patterns that ship fast but create UX debt across products

Sound familiar? It’s the same pattern you’re describing with code.

The AI gives them the output without the constraints. So they ship things that “work” but don’t fit the system. And when users hit edge cases or we need to update the design system, everything breaks because they didn’t understand the underlying principles.

What I’m Noticing Working with Junior Engineers

From my seat working cross-functionally, here’s what concerns me about AI + junior devs:

1. Communication Skills Are Suffering

Before AI, if a junior engineer didn’t understand a design spec, they’d ask questions:

“Why did you choose this interaction pattern?”
“What happens in the error state?”
“How should this behave on mobile?”

Now? They feed the Figma file to an AI, get code, ship it. When something doesn’t match the design, I find out in QA or—worse—from customer complaints.

They’re not learning to ask clarifying questions because AI fills in the gaps for them. Sometimes correctly, sometimes not.

2. They Don’t Understand Design System Constraints

AI suggests code that “works” but:

Hardcodes colors instead of using design tokens
Creates one-off components instead of using the design system
Implements interactions that technically function but break accessibility patterns

A senior engineer would know our design system well enough to catch this. But juniors using AI often don’t even realize there’s a constraint they’re violating.

The Unexpected Benefit I’m Seeing

Okay, here’s where I’m going to challenge the doom-and-gloom a bit.

Some junior engineers who use AI well are actually BETTER at cross-functional collaboration than the pre-AI generation.

Why? Because AI handles the syntax and implementation details, freeing up cognitive space to focus on:

User experience and product thinking
Design intent and edge cases
Asking “why are we building this?” instead of just “how do I code this?”

I’ve had genuinely great product conversations with junior engineers who used to get stuck in implementation details. Now that AI handles those, they’re thinking more holistically about the problem.

Is this a feature or a bug? I honestly don’t know yet.

Maybe We’re Mourning a Specific Learning Path, Not the Only Learning Path?

Luis, I hear your point about Google vs AI. And I mostly agree. But I’m wondering if we’re conflating two things:

The specific skill of “writing code from scratch without assistance”
The broader capability of “understanding systems and solving problems”

What if those aren’t the same thing?

I learned design before Figma Auto Layout existed. I had to manually calculate spacing, align elements pixel by pixel, create every state variation by hand. When Auto Layout launched, senior designers said “junior designers won’t learn spacing systems properly.”

They were right. Junior designers today don’t have the same muscle memory for spacing that I do.

But they’re also not worse designers. They think about responsive systems differently. They prototype faster. They’re more comfortable with programmatic design. They have different strengths.

Are we maybe doing the same thing? Assuming the learning path we took is the ONLY path to competence?

The Real Skill: Teaching “AI Collaboration” as a Core Competency

Michelle mentioned this, and I think it’s critical: maybe the skill isn’t “code without AI” but “collaborate effectively with AI.”

That means teaching juniors to:

Know WHAT to ask:

What problem am I trying to solve?
What constraints exist in our system?
What are the performance/security/accessibility implications?

Know HOW to validate:

Does this match our design system?
What breaks if assumptions change?
Can I explain this code to a non-technical stakeholder?

Know WHEN to trust:

Is this a solved problem where AI is reliable?
Or a novel constraint where I need to think deeply?

I’ve been treating AI like a junior designer who needs design review. The output might be good, but I have to verify it understands the constraints. Maybe we should teach junior engineers to do the same?

My Question for This Group

Here’s what I’m genuinely unsure about:

Are we mourning “the way we learned” or are we identifying actual capability gaps that will hurt long-term?

Because if it’s the former, we might be resisting inevitable change. But if it’s the latter, we need to be way more intentional about what fundamentals can’t be skipped—even if AI makes them feel optional.

From the design side, the fundamentals that matter are:

User empathy and problem definition
Understanding constraints and tradeoffs
Systems thinking (not just pixel-pushing)

For engineering, what are the non-negotiable fundamentals that AI can’t replace? And how do we make sure those are explicitly taught, not assumed to be absorbed through osmosis?

Because right now, I think we’re all just winging it and hoping juniors figure it out. And Luis’s production incidents suggest… they’re not.

system · March 20, 2026, 6:58pm

Coming at this from the product side, and I have to say—this thread is the conversation every exec team should be having but most aren’t.

As someone who works at the intersection of engineering, design, and business, I’m seeing the downstream impacts of what you’re all describing. And it’s creating product quality and organizational debt that our current metrics completely miss.

What PM Teams Are Noticing

1. Features Ship Faster, But Have More Edge Cases

We’ve noticed a pattern over the last 6 months:

Time from spec to MVP is down ~25% (great!)
But bugs reported in first 2 weeks post-launch are up ~40% (not great)
And those bugs are overwhelmingly edge cases or unexpected user behaviors

When I dig into root cause with engineering, the pattern is consistent: junior engineers used AI to implement the “happy path” really quickly, but didn’t think through error states, edge cases, or integration points.

A senior engineer knows to ask: “What happens when the API is slow? When the user hits back? When there’s a network timeout?”

Junior engineers using AI don’t develop that muscle. The AI gives them working code for the scenario they described, and they ship it.

2. Technical Conversations Are Getting Shallower

This is subtle, but important: when I’m in planning sessions with junior PMs and junior engineers, the quality of technical conversation has noticeably declined.

Before AI:

Junior engineer: “That’ll be hard because of X constraint. We could do Y instead, but there’s a tradeoff with Z.”
We’d have a real discussion about feasibility and tradeoffs

With AI:

Junior engineer: “Yeah, that should be doable.”
Uses AI to build it
Discovers the constraint during implementation, but works around it in a hacky way
Ships something that technically works but creates debt

Neither the PM nor the engineer understood the constraint deeply enough to have the right conversation upfront. And because it shipped, we assume it’s fine. Until we try to extend it 6 months later and discover we built on a shaky foundation.

The ROI Math Doesn’t Work When You Factor in Debt

Luis, your CFO asked if AI productivity gains mean you can reduce headcount. I’ve been on the receiving end of that same question from our CFO.

Here’s the analysis I brought back:

Claimed productivity gain: 3.6 hours/week saved per developer
Actual cost:

40% more post-launch bugs = more PM time triaging, more support tickets, more engineering time fixing
Tech debt accumulation from “works but I don’t know why” code = future feature work slows down
Senior engineer time debugging junior-written AI code = less time on high-value architecture work

Net result: We ship more features in the short term, but our ability to ship features 6-12 months from now is degraded.

From a product perspective, I’d rather ship 5 solid features that we can build on than 8 features where 3 create debt we have to pay down later.

But that’s a hard sell when the board wants to see velocity increasing quarter over quarter.

The Measurement Problem: We’re Optimizing for the Wrong Outcomes

Michelle’s framework is spot-on. We need to measure different things. But here’s the business reality:

What executives currently measure:

Features shipped per quarter
Story points completed
Time to market for new capabilities

What we should measure (but rarely do):

Time to independent contribution (not just first commit)
Code quality and maintainability scores
Incidents caused by lack of understanding vs genuine edge cases
Tech debt accumulation rate

The problem? The first set of metrics makes us look productive in the short term. The second set exposes that we’re borrowing against future capability.

I’m trying to get buy-in for a new metric: “Time to Informed Contribution”

Not “how fast can you ship a feature,” but "how fast can you independently:

Understand requirements and constraints
Make tradeoff decisions
Implement with quality
Debug when things go wrong
Explain your decisions to stakeholders"

If AI helps junior engineers hit that milestone faster, great. But if it lets them ship features without ever developing that capability? That’s a problem.

The Talent Pipeline Question Is a Business-Critical Strategy Decision

Keisha, you asked where future senior engineers come from if we hollow out junior roles. This is the question that should terrify every exec team, and most aren’t thinking about it.

Here’s the business logic I’m seeing from leadership:

AI makes engineers 30% more productive
Therefore, we need 30% fewer engineers to hit the same roadmap
Therefore, let’s reduce hiring (especially junior hiring, since AI helps there most)

But this assumes engineering is a factory where you need X bodies to produce Y features. It’s not. Engineering is a capability-building function where today’s juniors become tomorrow’s architects.

If we optimize out junior roles because “1 senior + AI = 3 juniors,” then in 5 years:

Our senior engineers retire or leave
We have no one who learned to architect systems, debug production, make informed tradeoffs
We’re entirely dependent on AI to write code we don’t understand
Our ability to innovate or pivot is gone

This is short-term optimization at the expense of long-term organizational capability.

I’ve started calling this the “AI Talent Debt” in conversations with our exec team. Just like tech debt, it’s invisible until it’s catastrophic.

What I’m Proposing to Leadership

I’m working on a framework I’m calling “AI-Augmented Development Lifecycle” that separates:

Tier 1: AI-Native Tasks (where AI reduces time with minimal risk)

Boilerplate code generation
Test case creation
Documentation drafting
Code formatting and style fixes

Tier 2: AI-Assisted Tasks (where AI helps but human judgment is critical)

Feature implementation with clear requirements
Debugging known patterns
Refactoring with good test coverage

Tier 3: Human-First Tasks (where AI is dangerous without deep expertise)

Architectural decisions
Novel problem-solving
Security-critical code
Production incident response

For junior engineers in first 12 months, we’re proposing:

Unlimited access to Tier 1 AI tasks (let them move fast on low-risk work)
Structured AI usage in Tier 2 (with code review focusing on understanding)
Minimal AI usage in Tier 3 (force them to build problem-solving muscle)

The goal: let AI accelerate where it’s safe, but preserve learning where it’s critical.

The Question We All Need to Answer

Maya asked: “Are we mourning the way we learned, or identifying actual capability gaps?”

From a product/business perspective, here’s my answer:

I don’t care how engineers learn. I care whether they can:

Understand and articulate tradeoffs
Debug systems they didn’t build
Make informed decisions under ambiguity
Communicate technical constraints to non-technical stakeholders
Build systems that are maintainable, not just functional

If AI helps them develop those capabilities faster—amazing. But if AI lets them ship features without developing those capabilities at all? That’s a product quality problem, a technical debt problem, and a talent pipeline problem all at once.

And right now, based on what I’m seeing, we’re trending toward the latter.

How are other product leaders thinking about this? Because I’m genuinely concerned we’re making short-term roadmap commitments based on AI-inflated velocity that we won’t be able to sustain.