18% Increase in Engineering Executive Demand But Supply Can't Keep Up—Are We Promoting the Wrong People or Training Them Wrong?

[system]

Last month, I tried to hire a VP of Engineering to lead our infrastructure org. After 89 days, three offers declined, and countless “you’re a great company, but…” conversations, I’m convinced we’re facing something bigger than a tight hiring market.

The numbers back this up: there’s an 18% projected increase in demand for engineering executive roles by 2026, but supply isn’t keeping pace. Globally, we’re looking at $78 billion in lost revenue from unfulfilled critical engineering leadership positions. There are literally three engineering jobs for every one qualified candidate.

But here’s what keeps me up at night: Are we actually short on leadership talent, or are we promoting the wrong people and then failing to develop them?

The IC-to-Manager Pipeline Problem

I’ve watched brilliant engineers get promoted to manager because they shipped the most code, crushed the most tickets, or designed the most elegant architecture. Then six months later, their team is underwater, morale is tanking, and the new manager is either micromanaging every PR or completely absent because they’re still trying to be the top individual contributor.

The transition from IC to engineering manager requires a fundamental mindset shift—from optimizing your personal output to maximizing team performance. But we promote people based on technical excellence and then… hope they figure out the rest?

When I made the jump from Google to Slack as Director of Engineering, I had five years of management experience behind me. Even then, I struggled with the invisible divide that comes with a leadership title. Your words carry more weight. Your presence in a meeting changes the dynamic. Your decision to jump in and “help” with a technical problem might signal to your team that you don’t trust them.

Nobody taught me that. I learned by making mistakes and having patient mentors point them out.

What Actually Makes a Good Engineering Executive?

When I interview VP and Director candidates now, technical chops are table stakes. What I’m really assessing is:

• Can they multiply others’ output? Not add their own work to the pile, but remove blockers, create clarity, and help the team level up.
• Do they lead with empathy and coaching? Can they have difficult conversations while maintaining trust?
• Can they think strategically about technology decisions AND business impact AND organizational design?
• Do they build inclusive teams where diverse perspectives are heard and valued?

These aren’t skills you naturally develop by being the best engineer in the room. They require different muscles—and deliberate practice.

The Succession Planning Gap Nobody Talks About

Here’s the uncomfortable truth: over 60% of US engineering-led small and medium businesses lack formal succession planning for critical leadership roles. Only 35% have documented emergency succession plans for C-suite roles.

That means most companies are relying on ad-hoc promotions or external hires when a leader leaves. No wonder we’re scrambling.

Meanwhile, over 25% of working engineers plan to retire within five years. We’re about to lose a generation of leadership experience, and we haven’t built the pipeline to replace them.

Companies that handle leadership transitions smoothly do two things simultaneously: succession planning AND leadership development. They use stretch assignments, executive coaching, cross-functional rotations, board exposure, and formal leadership programs. 83% of organizations use mentoring and coaching to develop succession candidates.

But how many of us actually have those programs in place?

So… Are We Promoting the Wrong People or Training Them Wrong?

I think it’s both.

We’re promoting based on individual technical output when we should be identifying people who show potential for multiplying others’ effectiveness. Those are different skills, and we need different assessment criteria.

AND we’re throwing new leaders into the deep end without structured support, expecting them to figure out skills that take years to develop—while still delivering on aggressive roadmaps.

The pipeline problem compounds because we’re not just short on VPs and CTOs today. We’re short on Directors and Senior Managers who will be ready for those roles in 3-5 years. And we’re giving first-time managers almost no support, so many of them burn out or go back to IC work before they have a chance to grow into leadership.

A Question for This Community

For those of you who’ve made the IC-to-manager or manager-to-executive transition:

What actually prepared you for leadership? Was it the people you worked for, specific training, learning by failing, or something else?

And for those hiring or promoting into leadership roles:

What are you optimizing for? Technical excellence, leadership potential, or some combination—and how do you actually assess that?

Because right now, it feels like we’re flying blind—and the industry can’t afford an $78 billion talent gap while we figure this out.

[cto_michelle]

This resonates deeply. I’ve been on both sides of this problem—struggling as a new leader 20 years ago, and now watching the same patterns repeat as CTO.

The Two-Year Test and Reality

You mentioned the “try management for two years” recommendation, and I think that’s actually critical guidance we ignore. Most companies promote someone to manager and then… that’s it. They’re locked into that track because we don’t build off-ramps.

At Microsoft early in my career, I watched brilliant engineers get promoted to manager, struggle for 6-9 months, and then feel trapped because going back to IC felt like “failure.” Meanwhile, their teams suffered, they burned out, and we lost both a great engineer AND failed to develop a great leader.

The real question isn’t “should they try management” but “do we create psychologically safe environments where they can return to IC if it’s not the right fit?”

Where I’ve Seen This Work

When I joined my current company as CTO three years ago, we had zero formal leadership development. Now we have:

1. Leadership Rotation Program: Senior ICs do 6-month “acting” roles (tech lead, team lead) before any permanent promotion. They get coaching, they lead real initiatives, and they can decide if it’s for them.

2. Executive Coaching Budget: Every director+ gets $5K/year for external coaching. Not HR-mandated training, but real 1:1 coaching with someone who understands technical leadership.

3. Cross-Functional Exposure: VP candidates spend time with Product, Sales, Finance—not just Engineering. You can’t be an effective executive if you only understand your silo.

4. Board Observer Program: We rotate senior leaders through board meetings as observers. Seeing how executives communicate up teaches you what the CTO role actually requires.

But here’s what’s uncomfortable: even with all this, I still have gaps. I tried to hire a VP Engineering for 78 days last year. Three offers declined. The feedback? Two wanted more AI/ML experience on their resume, one wanted a company that already had their leadership development figured out (ironic, right?).

The Assessment Problem

You asked what we optimize for in promotion decisions. Honestly, I’m still figuring this out. Here’s my current framework:

Technical Excellence: Table stakes. But I look for “systems thinking” more than “best coder”—can they see patterns across the org?

Multiplier Mindset: Do they mentor? Do they write docs? Do they make others more effective even when it’s not in their job description?

Emotional Regulation Under Pressure: Engineering leadership involves a LOT of uncomfortable conversations. Can they stay grounded when things are on fire?

Curiosity About the Business: Do they ask about customer outcomes, revenue, go-to-market? Or do they only care about technical elegance?

The problem is these are qualitative and subjective. And if I’m being honest with myself, the people I recognize as having “leadership potential” probably look a lot like me and the leaders I learned from. That’s a problem for building diverse leadership pipelines.

The Question That Haunts Me

You ended with “what actually prepared you for leadership?” and I’ve been thinking about that all morning.

The brutal truth: Nothing prepared me. I learned by screwing up, having patient mentors call me on it, and doing better next time. That approach is inefficient, painful, and leaves casualties (both leaders who burn out and teams who suffer under ineffective leadership).

But I don’t know if we can really “train” people into great leaders. Leadership is relational and contextual—what works at Google might not work at a 50-person startup.

Maybe the real answer is: we need to identify leadership potential earlier, create many more opportunities to try leadership in low-stakes ways, and make it psychologically safe to say “this isn’t for me” without killing someone’s career.

How do we do that at scale? I genuinely don’t know. Open to ideas.

[eng_director_luis]

Reading this thread feels like looking in a mirror—except 18 months ago when I first got promoted to Director.

My Crash Course in “What Not to Do”

I was the senior-most engineer on my team, had designed most of our core systems, and when my manager left, leadership asked if I wanted the role. I said yes because… honestly, I thought it would be like my current job plus some 1:1s and roadmap planning.

I was so wrong.

Within three months, I had:

• Jumped into every technical problem because “I could solve it faster” (destroying my team’s confidence)
• Micromanaged every code review (creating a bottleneck where I became the blocker)
• Continued coding 30+ hours/week (while also doing all the manager stuff, leading to burnout)
• Made unilateral technical decisions (because I “knew best” as the former senior engineer)

My team’s velocity dropped. Two engineers asked to transfer. And I started having Sunday night anxiety attacks.

The Invisible Divide Is Real

@vp_eng_keisha, you mentioned the “invisible divide” when you get a leadership title, and that hit hard. Your presence changes the dynamic of every conversation.

I didn’t understand that when I walked into a design discussion and said “I think we should use Kafka here,” everyone heard “Luis wants us to use Kafka” and the discussion was over. I thought I was collaborating. They thought I was dictating.

Or when I’d jump in to help debug a production issue, my team heard “Luis doesn’t think I can handle this” instead of “Luis is being supportive.”

The mindset shift from IC to manager is harder than any technical problem I’ve solved. Because code is logical and predictable. People aren’t.

What Actually Helped (Eventually)

I got lucky. My VP sat me down after month 4 and said “Luis, you’re optimizing for the wrong thing. Your job isn’t to be the best engineer on the team. It’s to make your team the best it can be.”

That conversation changed everything. But I needed someone to explicitly say it.

After that, I:

1. Joined the SHPE (Society of Hispanic Professional Engineers) engineering leadership cohort. Having peer mentors who got the cultural dynamics plus the leadership challenges was huge. Seeing other first-gen professionals navigate this gave me permission to not have all the answers.

2. Worked with an executive coach for 6 months. My company didn’t pay for it (wish they had), but it was worth every dollar. Someone who could call me on my BS and help me see my blind spots.

3. Started measuring different things. Instead of “how many stories did I ship,” I tracked “how many engineers leveled up,” “how many blockers did I remove,” “how many decisions did the team make without me.”

4. Created explicit “Luis-free zones.” I stopped attending certain design meetings. I stopped reviewing every PR. It was terrifying to let go, but my team needed space to grow.

But Here’s the Thing…

I agree with @cto_michelle that we’re promoting based on technical output when we should assess for leadership potential. But I also think the problem is deeper:

We promote people and give them zero support structure.

I had no onboarding as a new manager. No training. No peer cohort. No explicit expectations beyond “you’re responsible for the team now.” Just… figure it out.

That’s not a skills gap. That’s organizational negligence.

And I recognize my privilege here: I had a patient VP, I could afford a coach, I had SHPE as a support network. What about first-time managers who don’t have those resources?

The Question I’m Wrestling With

You asked: “Are we promoting the wrong people or training them wrong?”

From where I sit: Both. But the training gap is more fixable than the promotion criteria.

Changing promotion criteria requires rethinking what we value, how we assess “potential,” and confronting biases about what leadership looks like. That’s hard organizational change.

But we could start leadership development programs tomorrow. Cohorts for first-time managers. Coaching budgets. Explicit off-ramps back to IC. Structured feedback loops.

Why aren’t we doing that?

Is it cost? Is it that leadership doesn’t see it as their problem? Is it that we genuinely don’t know what would work?

I’d love to hear from others: For those who struggled in early leadership roles and eventually figured it out—what support would have shortened that learning curve?

Because right now, we’re letting people (and their teams) suffer through a 12-18 month trial-by-fire when we could be doing so much better.