We're all using AI coding assistants, but do we actually trust them?

General
1

Our team adopted GitHub Copilot eight months ago, and the numbers look great on paper—85% weekly usage, significant velocity gains in our sprints. But here’s what keeps me up at night: in every single code review, we’re catching issues in AI-generated code. Constantly.

And we’re not alone. The data is wild:

92.6% of developers use AI coding assistants monthly. 75% use them weekly. That’s essentially universal adoption.

Yet only 33% of developers actually trust the results. 46% explicitly don’t trust them. And here’s the kicker—only 48% always check AI-generated code before committing it.

Let me say that again: we have near-universal adoption of tools that half of us don’t trust, and half of us don’t consistently verify.

The Productivity Paradox I’m Seeing

My team codes 40% faster with AI assistance. That’s real. But our delivery velocity? Barely moved. Why? Because we’re spending that saved time in extended code reviews, fixing subtle bugs, and dealing with quality issues that slip through.

Projects using heavy AI-generated code saw a 41% increase in bugs and a 7.2% drop in system stability. We’re trading speed for quality, often without realizing it.

The frustrating part? Sometimes engineers spend more time reviewing and fixing AI output than they would have spent just writing the code themselves. The AI generates verbose solutions that are harder to debug, and spotting the errors requires understanding code you didn’t write.

The Verification Problem

Here’s what I’m observing on my teams: Engineers treat AI like a really fast junior developer who needs constant supervision. That’s fine when we’re conscious of it. But 96% of developers say they have difficulty trusting that AI-generated code is functionally correct. Yet we’re using it everywhere.

The trust gap creates this weird dynamic:

  • We use AI because we need the speed
  • We don’t trust it, so we verify everything
  • But verification is mentally exhausting
  • So sometimes we don’t verify thoroughly
  • And that’s when issues slip into production

Security is even scarier. We’re seeing hallucinated dependencies—AI references packages that don’t exist, which creates opportunities for supply chain attacks. Some teams report that AI-generated code has different failure modes than human-written code.

Is This a Maturity Curve or Fundamental Problem?

I genuinely don’t know if this is just early-adopter pain that will resolve as tools improve, or if we’re looking at a fundamental mismatch between what AI can do and what we need it to do.

The optimistic view: Tools will get better, we’ll develop better verification practices, and eventually the trust will catch up to the adoption. We’re just on the steep part of the learning curve.

The pessimistic view: We’re automating the easy parts of coding while making the hard parts (understanding, debugging, maintaining) even harder. And we’ve created dependency on tools we don’t trust because our velocity expectations now assume AI assistance.

What Are Other Leaders Doing?

I’m curious how other engineering leaders are handling this:

  • What guardrails have you implemented? Are you restricting AI use in certain contexts (security, critical paths)? Requiring different review standards for AI-generated code?

  • How are you measuring the actual ROI? Not just “developers code faster” but end-to-end impact on delivery and quality?

  • Training and enablement: Are you doing structured onboarding for AI tools, or is it just “here’s your Copilot license, good luck”?

  • The culture question: How do you build a culture where engineers feel safe saying “I don’t understand this AI-generated code” instead of pretending they do?

The adoption-trust gap feels important. We’re making architectural decisions based on tools we don’t fully trust. That seems… worth talking about honestly.

What’s your experience been?

2

Keisha, this is exactly what keeps CTOs up at night. You’ve articulated the problem perfectly—near-universal adoption with a massive trust deficit.

At our company, we experienced similar pain points until we implemented a governance framework specifically for AI-generated code. Here’s what we did:

Our AI Code Governance Framework

1. Security-focused prompting: We require engineers to use security-conscious prompts with AI tools. Not just “write me an auth function,” but “write a secure auth function following OWASP guidelines with input validation and rate limiting.” The quality difference is significant.

2. Tiered review requirements:

  • Low-risk code (tests, documentation, boilerplate): Standard review process
  • Medium-risk (business logic, data processing): Mandatory senior engineer review with explicit “AI-generated” flag in PR
  • High-risk (authentication, payments, PII handling): AI heavily restricted, and when used, requires security team review

3. Integration with static analysis: AI output gets scanned by our SAST tools before it enters code review. We’ve caught hallucinated dependencies, insecure patterns, and license violations this way.

4. Regular production audits: Quarterly reviews of AI-generated code that made it to production, looking for patterns in what works and what doesn’t.

The Real Challenge: Organizational Maturity Gap

Here’s what I’ve learned: the problem isn’t the tools themselves—it’s that teams adopted AI coding assistants faster than organizations could build appropriate governance.

Most companies are still treating AI as a “developer productivity tool” when they should be treating it like a “code generation pipeline” that requires the same rigor as any external dependency.

Think about it: if an engineer wanted to pull in a new open-source library, we’d review it for security, licensing, maintenance status, and community health. But AI-generated code? We’re letting it flow directly into our codebase with minimal oversight.

The Liability Question Is Real

Our legal and compliance teams are now asking hard questions about AI-generated code:

  • What’s the provenance? Can we trace what code came from where?
  • What training data was used? Are there IP concerns?
  • If there’s a security breach, who’s accountable?
  • How do we prove to auditors that we exercised due diligence?

These aren’t hypothetical concerns anymore. We’re seeing insurance underwriters ask about AI code governance during cyber insurance renewals.

What We Need: Industry Standards

Individual company policies aren’t enough. We need industry-level standards for AI-assisted development, similar to what we have for open-source supply chain security (SBOM, dependency scanning, etc.).

I’d love to see:

  • Standard metadata for AI-generated code (which tool, which model version, what prompt)
  • Shared vulnerability databases for AI coding assistant output patterns
  • Industry benchmarks for AI code quality by use case
  • Training certifications for secure AI-assisted development

This is infrastructure-level stuff, not just personal productivity enhancement. We need to treat it accordingly.

Keisha, your question about whether this is a maturity curve or fundamental problem resonates. I think it’s both—the tools will improve, but we’ll always need verification and governance because AI fundamentally works differently than human reasoning. It learns patterns, not principles.

3

This conversation is hitting hard. I need to confess something: I’m probably part of the problem.

The Convincing Nature of AI Code

I use Copilot daily for our design system work, and the code it generates looks so professional. It follows patterns, includes error handling, adds helpful comments. From a visual inspection, it’s often better formatted than what I’d write by hand.

Last month, I used it to generate a new React component for our accessibility toolkit. The code looked perfect. TypeScript was happy. The component rendered beautifully.

Then our accessibility consultant tested it with a screen reader and found it was a complete nightmare. The AI had generated semantically incorrect ARIA attributes, created a keyboard trap, and used div soup where semantic HTML belonged. It looked accessible but wasn’t.

Why We Don’t Always Check

Here’s the uncomfortable truth about why I don’t always verify AI code thoroughly:

Time pressure: When I’m under deadline pressure to ship features, and AI just gave me 80% of a component in 30 seconds, it feels wasteful to spend an hour rewriting it—even when something feels off.

Psychological bias: If AI wrote 80% of a function, there’s this weird mental trap where you convince yourself “the AI probably knows better than me” especially when you’re tired or working outside your comfort zone.

The convincing factor: AI code doesn’t look like junior dev code. It’s well-structured, uses modern patterns, includes edge case handling. It looks like it was written by someone who knows what they’re doing.

The Cognitive Load Problem Nobody Talks About

Reviewing AI-generated code is actually harder than writing code from scratch, and I don’t think we acknowledge this enough.

When you write code yourself, you understand your own assumptions and thought process. When you review AI code, you’re debugging someone else’s logic without understanding their (its?) reasoning. You have to reverse-engineer intent from implementation.

And the AI-generated code is often more verbose and clever than necessary, which makes errors harder to spot. It’s like the difference between debugging minified JavaScript versus code you wrote—except you can’t unminify the AI’s thought process.

What Actually Works for Me

After my accessibility disaster, I changed my approach:

1. AI for boilerplate only: I use it heavily for repetitive structure, but write the critical logic myself.

2. Treat it like external library code: I don’t trust third-party packages blindly, why would I trust AI output? Same scrutiny applies.

3. Pair with someone when using AI for complex work: Having another person review AI-generated code in real-time catches issues I’d miss solo.

4. Check the things AI typically gets wrong in my domain:

  • Accessibility (AI doesn’t deeply understand context)
  • Design system consistency (AI doesn’t know our specific patterns)
  • Edge cases specific to our product

Question for the Group

For those of you who DO consistently verify AI code—what does that look like?

Do you:

  • Review it like it’s an external dependency?
  • Run it through security scanners every time?
  • Have a checklist of common AI failure modes?
  • Only use AI for specific, low-risk tasks?

I want to do better, but I also need practical workflows that don’t make AI assistance a net negative on my time.

Also, Keisha’s point about engineers being afraid to admit “I don’t understand this AI code” is real. We need to normalize saying that out loud.

4

Keisha, Maya, Michelle—all of this resonates deeply. I’m leading a 40-person engineering team at a financial services company, and we’ve been wrestling with exactly these issues.

The Pressure from Above

Here’s the uncomfortable reality: When leadership sees the “40% faster coding” statistics, they start allocating work accordingly. Our velocity expectations now assume AI assistance. Teams feel like they NEED to use AI to meet sprint commitments.

We’ve created a dependency on tools we don’t fully trust, and that’s a dangerous place to be.

Our Journey: From Chaos to Structure

Phase 1 (Chaos): Initially, everyone used whatever AI tools they wanted however they wanted. We saw the velocity bump, celebrated it, allocated more work.

Phase 2 (Wake-up call): Two production bugs directly traced to AI-generated code. One was a subtle race condition in payment processing. The other was a SQL injection vulnerability that our normal code review missed because the AI-generated code was so verbose it was hard to spot.

Phase 3 (Structure): We implemented a tiered approach based on risk:

Tier 1 - AI freely encouraged (light review):

  • Unit tests and test fixtures
  • Documentation and code comments
  • Boilerplate and scaffolding
  • Simple CRUD operations

Tier 2 - AI with mandatory senior review:

  • Business logic and data transformations
  • API endpoints and integrations
  • Database queries and migrations
  • Frontend components

Tier 3 - AI heavily restricted:

  • Authentication and authorization
  • Payment processing
  • PII handling
  • Security-critical paths

What Actually Worked

1. Explicit guidelines gave teams permission to slow down. Before, engineers felt pressure to use AI everywhere. Now they have clear boundaries and don’t feel guilty about writing code manually when appropriate.

2. Pairing junior engineers with seniors when using AI for complex logic. This accomplishes two things: seniors catch issues in real-time, and juniors learn what to watch for.

3. Tracking “AI-generated bugs” separately in retrospectives. This gave us actual data on the cost. Turns out our “40% productivity gain” was closer to 15% when you factored in rework, extra review time, and production fixes.

What’s Still Broken

Attribution problem: After 2-3 rounds of editing, it’s hard to tell what was AI-generated versus human-written. Engineers can game the system by claiming they wrote AI code themselves.

Near misses: Our metrics capture bugs that made it to production, but not the “would have been a disaster” issues caught in code review. How do you quantify avoided disasters?

Cultural challenges: Some engineers view AI restrictions as lack of trust in their judgment. Others feel like they’re falling behind if they’re not using AI for everything.

The Governance Question

Michelle’s point about organizational governance is spot-on. In financial services, we have regulatory requirements around code quality, security, and auditability. Individual team policies aren’t enough.

We’re working with our compliance team on:

  • Provenance tracking for AI-generated code
  • Documentation of which AI tools/models were used
  • Audit trails for code review of AI-assisted work
  • Risk assessments specific to AI coding tools

This isn’t optional in regulated industries. Eventually, an auditor is going to ask “how do you ensure AI-generated code meets regulatory standards?” and we need a real answer.

To Maya’s Question About Practical Workflows

For verification, we’ve standardized on:

Mandatory security scanning: All PRs with AI-generated code get flagged for extra SAST/DAST analysis
Checklist in PR template: “If this PR contains AI-generated code, confirm you’ve verified: [list of common issues]”
Async code review culture: Reviewers explicitly take extra time with AI code—we don’t rush these reviews
Blameless post-mortems: When AI code causes issues, we focus on process improvements, not individual blame

The Data and Research Gap

Here’s what frustrates me: The research says teams that invest in review, governance, and training see 35% higher quality improvements. But there’s no playbook for what that actually looks like.

We’re all figuring this out independently, which seems inefficient. The industry needs to share what works and what doesn’t—not just the success stories, but the failures too.

Keisha’s original question—maturity curve vs fundamental problem—I think it’s both. AI will get better at generating correct code, but it will always require different verification approaches because it fails differently than humans do.

The key is building organizational muscle around AI-assisted development now so we’re ready as the tools evolve.

5

Coming at this from the product side, and I think there’s a critical dimension we’re not discussing enough: the trust gap isn’t just internal—it has business and customer implications.

The Business Reality: CFOs Are Asking Questions

Here’s what I’m seeing at the leadership level: CFOs are deferring 25% of AI spending to 2027 until ROI is proven. The honeymoon period is over. Executives want data, not promises.

When we present our product roadmap now, finance asks:

  • “How much of your velocity estimate assumes AI assistance?”
  • “What’s the rework rate on AI-generated features?”
  • “How are you accounting for quality risk in timeline projections?”

These aren’t easy questions to answer when we’re still figuring out the trust dynamics ourselves.

The Capacity Planning Problem

Luis hit on something crucial: when development is AI-dependent, how do you actually assess team capability for genuinely novel challenges?

We ran into this hard last quarter. Our velocity metrics looked great—lots of story points closed, features shipped quickly. Then we hit a complex integration problem that required deep system thinking.

Suddenly, our “high-velocity” team struggled. Why? Because we’d been optimizing for tasks where AI could help (CRUD, standard patterns, incremental features) and hadn’t built the muscle for hard problems where AI provides limited value.

AI tools can mask real skill gaps. Your team looks productive until they encounter something the AI hasn’t been trained on.

The Customer Trust Question

Here’s the uncomfortable mirror: If we don’t trust AI-generated code in our own codebase, why would customers trust AI-generated features we ship?

We’re building AI-powered features in our product while simultaneously not trusting the AI tools we use to build them. That’s… a weird cognitive dissonance.

And customers are starting to ask these questions during enterprise deals:

  • “What AI tools do you use in development?”
  • “How do you ensure AI-generated code meets security standards?”
  • “What’s your liability position if AI-generated code causes a data breach?”

These questions are coming from procurement and legal teams, not just technical buyers.

Are We Building Capability or Dependency?

Michelle’s framework and Luis’s tiered approach are exactly right from a governance perspective. But from a strategic lens, I keep asking: Are we building AI capability or AI dependency?

Capability means: Engineers understand when to use AI, how to verify output, and can work effectively without it when needed.

Dependency means: Teams can’t hit velocity targets without AI assistance, and skill development atrophies in areas where AI does the work.

The data suggests we might be trending toward dependency:

  • Velocity expectations now assume AI use
  • Engineers feel pressure to use AI to meet commitments
  • Novel problem-solving skills may be declining

That’s a strategic risk, not just an operational one.

Where AI Actually Delivers ROI

The market research is telling: AI coding tools are hitting 0.1B by 2032 with 27.1% CAGR. But enterprise adoption is cautious, and for good reason.

The companies seeing real ROI aren’t using AI to replace developer judgment—they’re using it for:

High-value, low-risk tasks:

  • Debugging and stack trace analysis (built-in verification)
  • Test generation and coverage improvement
  • Documentation and technical writing
  • Code modernization and refactoring
  • Boilerplate and repetitive patterns

Not for:

  • Novel algorithm development
  • Security-critical implementations
  • Complex system design
  • Anything requiring deep business context

The Investment Reframe

Maybe the ROI conversation is wrong. We’re measuring “coding speed” when we should be measuring “democratization of technical tasks.”

AI’s real value might not be making senior engineers 40% faster at writing complex code. It might be:

  • Making documentation actually happen (instead of being perpetually delayed)
  • Enabling product managers to prototype ideas without engineering bottleneck
  • Helping designers understand implementation constraints
  • Letting junior engineers contribute to test coverage meaningfully

That’s a different value proposition than “everyone codes faster.”

Back to Keisha’s Question: Trust and Transparency

You asked how to build culture where engineers can say “I don’t understand this AI code.”

From a product perspective, I’d flip it: Can we trace AI decisions like we trace bugs?

What if every piece of AI-generated code had metadata:

  • Which tool generated it (GitHub Copilot, Claude, etc.)
  • What model version
  • What prompt/context was used
  • Who reviewed it and when
  • What verification was performed

This isn’t about blame—it’s about traceability. If we can trace a bug back through git history, code review, and requirements, shouldn’t we be able to trace it back through AI generation?

That transparency might build trust more than any process policy.

The Strategic Question

Long-term, I think the adoption-trust gap is healthy. It means we’re being thoughtful about a powerful tool rather than blindly accepting marketing claims.

But we need to move from individual skepticism to organizational standards. Michelle’s call for industry benchmarks is right. We’re all reinventing the wheel, and that’s inefficient.

Also, we should probably stop calling these “productivity tools” and start calling them what they are: code generation pipelines that require governance, verification, and continuous evaluation.

That reframing might get executive teams to take the risk seriously.