We Gave Our Junior Devs AI Tools and Saw 45% Speed Gains—But Stack Trace Analysis Was the Real Winner, Not Code Gen

So here’s something I wasn’t expecting to write about this quarter.

We rolled out AI coding assistants to our design systems team about six months ago. Our team is small—three senior engineers, two junior devs who work closely with me on component implementation, and a handful of contractors. The juniors were the most excited, which makes sense. Fresh out of bootcamp, eager to move faster, ready to prove themselves.

The headline numbers looked great :bar_chart:

Within three months, our junior devs were completing tasks about 45% faster. PRs were flying in. Velocity metrics went up. Leadership was thrilled. I was thrilled! We even presented it at a company all-hands as a success story.

But then I started digging into where those productivity gains were actually coming from.

The surprise: It wasn’t code generation :thinking:

I assumed the big win would be code generation—autocomplete on steroids, basically writing half the component implementation. And yeah, that helped. But when I actually tracked time spent on different activities (shoutout to the devs who humor my spreadsheet obsession), here’s what I found:

Where juniors spent AI-assisted time:

  • Code generation: ~20% of time saved
  • Documentation lookup: ~15% of time saved
  • Stack trace analysis and debugging: ~65% of time saved :high_voltage:

That last one blew my mind.

What’s actually happening

Our junior devs were using AI tools to paste in error messages and stack traces, and the AI would explain:

  • What the error actually means (in human language)
  • Which line is the likely culprit
  • Common causes for this specific error pattern
  • Suggestions for fixes based on similar issues

One of our juniors told me: “Before AI, I’d spend 30-45 minutes Googling an error, reading Stack Overflow threads from 2017, trying to figure out if it’s relevant to our codebase. Now I paste the error, get context in 30 seconds, and I’m unblocked.”

The cognitive load reduction is massive. It’s not about writing more code faster—it’s about being less frustrated and stuck when something breaks.

But here’s the catch nobody’s talking about :police_car_light:

All that speed created a new bottleneck: code review.

Our senior engineers (myself included when I review code) are now drowning in PRs. The juniors can generate and debug code way faster than we can thoughtfully review it. And AI-generated code needs more careful review, not less, because sometimes it looks right but makes architectural decisions we wouldn’t have made.

We’re essentially trading junior dev wait time for senior dev burnout.

The question I can’t stop thinking about

Everyone’s measuring velocity—lines of code, PRs merged, tasks completed. But are we measuring the right thing?

What if the real value isn’t “juniors ship faster” but “juniors spend less time feeling dumb and stuck”? That’s a cognitive load reduction that doesn’t show up in sprint metrics but absolutely matters for retention, learning, and long-term growth.

And honestly? I’m starting to worry that juniors are learning what works without understanding why it works. They’re getting unblocked by AI without building the mental model of “how to debug systematically.”

What I’m curious about from this community :thought_balloon:

  1. Are others seeing similar patterns where debugging/stack trace analysis is the real productivity gain, not code generation?

  2. How are you handling the code review bottleneck when juniors (or anyone) can generate code 10x faster but review capacity is still human-speed?

  3. What are you actually measuring? Just velocity, or are you trying to quantify cognitive load, learning, or other softer metrics?

  4. Any concerns about juniors learning to be productive without learning to be skilled? Or am I overthinking this?

I love that AI tools are making our juniors feel capable and move faster. But I want to make sure we’re setting them up for long-term success, not just short-term output.

Would love to hear if anyone else is wrestling with this stuff. :artist_palette::sparkles: