The AI Skills Gap Isn't Just Technical—It's a Hiring Process Problem

General
, , ,
1

I’ve been hiring software engineers in financial services for 8 years. Last quarter, I reviewed 243 applications for 4 AI/ML engineering positions. We made 3 offers. This quarter, I have 6 open roles and 180 applications pending review.

Here’s what keeps me up at night: I’m rejecting exceptional engineers with 10-15 years of experience because they don’t have 5 years working with tools that have existed for 3 years.

The Perfect Match Problem

Our current job posting for “Senior AI Engineer” requires:

  • 5+ years production ML experience
  • Expert-level Python, PyTorch, TensorFlow
  • Experience with LangChain, vector databases, RAG architectures
  • Knowledge of prompt engineering, fine-tuning, RLHF
  • AWS SageMaker or equivalent MLOps platforms
  • Financial services domain expertise
  • Plus the usual: system design, algorithms, CI/CD, security

I’m not exaggerating. That’s the actual JD approved by our talent acquisition team.

The problem? LangChain launched in late 2022. RAG architectures went mainstream in 2023. We’re asking for 5 years of experience in technologies that are 2-3 years old, combined with financial domain expertise that takes 5+ years to develop.

We’ve created an impossible candidate that doesn’t exist.

The Rejection Paradox

Last month I rejected a brilliant engineer with 12 years of backend experience, 6 years in fintech, deep knowledge of distributed systems, proven track record scaling services to millions of users. Smart, curious, great communicator.

Why did I reject him? He’d only been working with ML for 8 months through online courses and side projects.

The kicker: My team is currently stuck because our “AI-native” engineer doesn’t understand our legacy payment systems and keeps introducing bugs that someone with financial domain expertise would catch immediately.

I rejected the person we actually need because he didn’t match the requirements for the person we think we need.

The 10 Tools Phenomenon

I reviewed job postings from 50 companies in our industry. The average “AI Engineer” role lists 12-15 specific tools and frameworks. Some examples:

  • Python, PyTorch, TensorFlow, JAX, Hugging Face Transformers
  • LangChain, LlamaIndex, vector databases (Pinecone, Weaviate, ChromaDB)
  • OpenAI API, Anthropic API, open-source LLMs
  • MLflow, Weights & Biases, SageMaker
  • Kubernetes, Docker, Terraform
  • Spark, Airflow, dbt

No one has deep experience with all of these. The people who come closest are either currently employed at FAANG companies making $500K+, or they’re consultants charging $300/hour who won’t take full-time roles.

We’re competing for maybe 5,000 qualified people nationwide while posting 50,000+ AI engineering jobs.

Are We Creating Artificial Scarcity?

Here’s my uncomfortable realization: We might be manufacturing this talent shortage.

What if instead of posting “Must have 5 years ML production experience,” we posted:

  • “Strong software engineering fundamentals”
  • “Demonstrated ability to learn new technologies quickly”
  • “Excited to work with AI/ML in financial services”
  • “We’ll provide 3-month onboarding with ML training”

I ran this experiment informally. I met with 5 rejected candidates and asked: “If we hired you with a 6-month ramp period and paired you with our ML team, could you become productive?”

All 5 said yes. Based on their backgrounds, I believe 4 of them would succeed.

But our hiring process doesn’t allow for this. We need “ready on day one” because we’re under pressure to ship AI features now to justify the layoffs we made last quarter to fund these AI hires.

The Upskilling Investment Nobody Wants to Make

I proposed a hybrid approach to our VP of Engineering:

  1. Hire 3 strong engineers with adjacent skills (backend, distributed systems, data)
  2. Partner with a bootcamp or university for intensive ML training
  3. Pair them with our senior ML engineer for 6 months
  4. Total investment: ~$50K per person in training + reduced productivity

Response: “That’s 6 months. We need production AI features in Q2 to show board progress.”

So we continue searching for unicorns while our product roadmap slips because we have 4 people doing the work of 10.

The Question I Can’t Answer

Is perfect-match hiring sustainable when the technology changes every 6 months?

By the time someone builds 5 years of experience with today’s AI stack, we’ll be on to different architectures, different frameworks, different approaches. The GPT-4 prompt engineering expert might be obsolete when GPT-6 changes how we interact with models.

Meanwhile, the fundamentals—algorithms, system design, data structures, software craftsmanship—those don’t change. The engineer I rejected with 12 years of backend experience? He could learn ML. But I can’t teach the ML bootcamp grad 12 years of production system wisdom.

What I’m Proposing

I’m going to push back on our next hiring round. My proposal:

Hire for potential, train for skills:

  • Focus on software engineering fundamentals and learning ability
  • Assess problem-solving, not tool knowledge
  • Provide structured ML onboarding (3-6 months)
  • Pair new hires with experienced ML engineers
  • Measure success at 12 months, not 3 months

The alternative is continuing to compete for people who don’t exist while rejecting people who could excel with investment.

For other hiring managers: How are you solving this? Are you finding the perfect-match candidates? Or are you also stuck between unrealistic requirements and business pressure to hire fast?

I don’t have the answer, but I know our current approach isn’t working.

2

Luis, this resonates deeply. I’ve been pushing similar upskilling proposals and hitting the same wall: timeline pressure from investors who don’t understand the actual maturity of AI technology.

The 6-Month Problem

Your 6-month training proposal is completely reasonable from an engineering perspective. But here’s what I’m hearing from boards and VCs:

  • “Your competitor announced AI features last quarter”
  • “We told investors we’d have AI revenue by Q3”
  • “The layoffs were justified by AI efficiency—where are the results?”

The business is operating on a 3-month cycle. Engineering talent development operates on a 12-month cycle. This mismatch is killing rational decision-making.

When Upskilling Fails

I actually tried this at my previous company (Series C SaaS, 200 employees). We hired 4 experienced backend engineers and partnered with a university for ML training:

What worked:

  • 3 of 4 completed the program
  • All showed strong ML fundamentals after 6 months
  • Cost was reasonable (~$60K total)

What failed:

  • Business needed results in month 3, not month 7
  • VPs kept pulling them into “urgent” non-ML work
  • We didn’t have enough senior ML mentorship capacity
  • By month 8, two had left for roles at companies with mature ML teams

The program wasn’t bad—the organizational support wasn’t there. We were trying to build ML capability while simultaneously under pressure to ship ML products.

The University Partnership Approach

Based on that failure, here’s what I’d propose differently:

Structured partnership model:

  1. Partner with 2-3 universities with strong ML programs
  2. Create “AI Engineering Residency” (similar to medical residency)
  3. Companies commit to 12-month hiring pipeline
  4. Universities provide curriculum + initial training (months 1-3)
  5. Companies provide real-world projects + mentorship (months 4-12)
  6. Graduates join as full-time engineers with ML capability

The key difference: This requires CEO/board buy-in upfront that this is a 12-month investment, not a 3-month sprint.

What I’m Doing Now

At my current company, I’ve taken a different approach because I couldn’t get buy-in for long training programs:

Hire hybrid profiles:

  • Data engineers who want to move into ML
  • Backend engineers with PhD backgrounds (they know how to learn)
  • Researchers from academia who need production engineering skills

These people have 70% of what we need. The remaining 30% they can learn on the job in 3-4 months because they already have the foundational mindset.

The trade-off: We don’t get “AI-native” engineers, but we get smart people who can learn anything. For most business use cases (RAG, basic fine-tuning, prompt engineering), this is sufficient.

The Uncomfortable Truth

Companies want AI experts but aren’t willing to invest in creating them. We expect universities and bootcamps to produce job-ready ML engineers, but the technology evolves faster than curriculum can adapt.

Someone has to pay for training. Either:

  • Companies invest in upskilling (6-12 months)
  • Employees pay for bootcamps ($15K+) and career risk
  • Government funds workforce transition programs

Right now we’re in limbo where nobody wants to pay, so we have a persistent talent shortage despite massive layoffs.

Luis, I hope your proposal succeeds. If it does, please share the playbook—I’ll use it to push similar programs here.

3

This is hitting close to home because I lived through the exact same pattern in design automation 5 years ago.

The “Figma Will Replace Designers” Era

In 2019-2020, every company was talking about design automation tools:

  • “AI can generate UI mockups”
  • “Figma plugins automate repetitive work”
  • “We need design engineers, not traditional designers”

I watched design teams get cut in half with the justification that tools would fill the gap. Job postings started asking for:

  • Expert in Figma, Sketch, Adobe XD
  • Proficient in 5+ prototyping tools
  • Coding skills (HTML/CSS/JS)
  • Motion design experience
  • AI-generated design workflows

Sound familiar?

What Actually Happened

The tools didn’t replace designers. They changed what design work looked like:

  • More time on strategy and user research
  • Less time on pixel-pushing
  • Higher expectations for polish and iteration speed

The designers who survived were the ones who learned on the job. Not because they went to “Figma bootcamp,” but because they were curious and willing to experiment.

The tools changed every 6 months anyway. The specific Figma plugins from 2020 are mostly obsolete now. What mattered was the mindset: “I can figure this out.”

Is AI Actually That Different?

I’m genuinely asking: Is learning ML fundamentally harder than learning React was in 2015? Or Kubernetes in 2018?

When I talk to engineers who’ve made the transition, they say the concepts are learnable:

  • Vector databases are just… databases with different indexing
  • RAG is search + context injection + LLM calls
  • Fine-tuning is training, which has existed for years
  • Prompt engineering is partly art, partly structured experimentation

The hard part isn’t the technology—it’s getting the opportunity to learn while employed.

Most engineers can’t afford to quit their jobs for 6 months to do a bootcamp. They need to learn while working, which means companies need to provide:

  • Real projects they can learn from
  • Mentorship from people who’ve done it
  • Permission to be “not perfect” for the first 3-6 months

The “AI Apprenticeship” Model

What if we stopped hiring for “AI Engineer” and started hiring for “Engineer Learning AI”?

Job posting rewrite:

  • Strong software engineering fundamentals (required)
  • Excited to learn ML/AI (required)
  • Have built something with AI tools, even if just side projects (nice to have)
  • We’ll provide mentorship, project time, and training budget (we promise)

I bet you’d get 500 qualified applicants instead of 10.

Then treat the first 6 months like an apprenticeship:

  • Pair with senior ML engineer
  • Start with smaller, bounded projects
  • Gradually increase complexity
  • Judge success at 12 months, not 3

This is how every previous tech shift worked. Web development in the 90s, mobile in 2010s, cloud infrastructure in 2015+. We didn’t require 5 years of experience before the technology existed—we hired smart people and let them learn.

Why I’m Optimistic

I’ve pivoted my career 3 times:

  1. Graphic designer → UX designer (2014)
  2. UX designer → Product designer with code skills (2018)
  3. Product designer → Design engineer (2022)

Each time, I learned on the job. Each time, someone gave me a chance despite “not having the exact experience.”

The engineers being rejected today will be the AI leaders in 5 years—if companies give them the opportunity to learn.

Luis, I hope your proposal works. The industry needs more leaders willing to invest in people instead of chasing perfect-match unicorns.