I’ve been following the excellent discussions in the community about AI coding tools - the trust gap, security vulnerabilities, productivity paradoxes, and skill development challenges. These conversations mirror exactly what we’ve been working through at our EdTech startup.
After several false starts and learning experiences, we’ve landed on an AI governance framework that’s actually working. I want to share what we learned - both the successes and the failures - in case it’s helpful for other engineering leaders navigating this.
Context: Why We Needed Governance
When I joined as VP Engineering 18 months ago, we were 25 engineers. We’re now scaling to 80+. AI tools started appearing bottom-up about 8 months ago:
- Engineers using GitHub Copilot
- ChatGPT and Claude for coding questions
- Various AI debugging and code generation tools
Initial state: No policies, no guidelines, everyone doing their own thing.
For a while, this seemed fine. Productivity metrics looked good. Engineers were happy. Leadership loved the innovation narrative.
The Wake-Up Call: Data Leakage Incident
Six months ago, we had a serious incident that changed everything.
A mid-level engineer was debugging a complex authentication issue. Copy-pasted our proprietary OAuth implementation code into ChatGPT to get help troubleshooting.
The problem:
- Code included business logic specific to our platform
- Had comments referencing our internal architecture
- Contained patterns unique to our implementation
- Was sent to a public AI service with no data retention guarantees
Potential impact:
- Proprietary code potentially in training data
- Could be surfaced to competitors through AI queries
- Intellectual property exposure
- Violation of our security policies (though we hadn’t been clear about this)
This wasn’t malicious. The engineer was trying to solve a problem and used the tools available. But it highlighted a systemic gap: We had no governance around AI tool usage.
Failed First Attempt: Top-Down Ban
My initial reaction (and I cringe at this now): “No AI tools allowed.”
We sent out a company-wide policy:
- Prohibited use of public AI services for code generation
- Restricted to approved tools only (we hadn’t defined what those were)
- Emphasized security and IP protection
- Threatened consequences for violations
What happened:
- Surface compliance (“yes, we’ll follow the policy”)
- Shadow AI usage continued (people just didn’t talk about it)
- Engineer frustration and morale hit
- Trust between engineering and leadership damaged
- No actual improvement in security posture
Lesson learned: You can’t put the genie back in the bottle. Prohibition without alternatives doesn’t work.
What’s Actually Working: Tiered Framework
After the failed ban, we took a step back and designed something more nuanced. Inspired by discussions with other engineering leaders and security teams, we developed a three-tier framework.
The Red-Yellow-Green Model
RED ZONE (No AI Code)
AI-generated code prohibited:
- Authentication and authorization systems
- Payment processing and financial transactions
- Student data handling and PII processing
- Compliance-required audit logging
- Security controls and incident response
- Cryptographic implementations
Why red zone?
- Security critical
- Regulatory compliance requirements (FERPA, COPPA for EdTech)
- High cost of failure
- Requires deep domain expertise
Who can work in red zone?
Senior engineers only, with security review.
YELLOW ZONE (AI with Enhanced Review)
AI-assisted code allowed with mandatory review:
- Business logic involving student data
- API endpoints and integrations
- Database schemas and queries
- Background job processing
- Third-party service integration
- Complex feature implementations
Requirements:
- Security-focused code review
- Enhanced testing coverage
- Documentation of AI usage in PR
- Static analysis with security rules
- Senior engineer approval
GREEN ZONE (AI Encouraged)
AI usage encouraged for productivity:
- UI components and frontend code
- Test code generation
- Documentation writing
- Boilerplate and scaffolding
- Internal tools and utilities
- Refactoring for readability
Standard review process applies.
Making the Framework Practical
Decision Tree Tool
We built a simple internal tool where engineers answer questions to determine their zone:
- “Does this code handle payments or financial data?” → RED
- “Does this code process student PII?” → RED
- “Does this code authenticate users?” → RED
- “Does this code involve business logic?” → YELLOW
- “Is this UI, tests, or docs?” → GREEN
Code Path Mapping
We tagged our codebase by tier. Engineers can check the tier before starting work.
When in Doubt Channel
Slack channel: #ai-governance-questions. Security team responds quickly. No judgment, just clarity.
Approved Tool List with Security Vetting
Part of our framework is restricting which AI tools can be used.
Approved Tools:
- GitHub Copilot for Business (enterprise contract, no training on our code)
- Claude with organization account (data retention policies)
- Internal AI tools we’ve vetted
Prohibited Tools:
- Public ChatGPT (free tier)
- Unofficial AI coding tools
- Any tool without enterprise data protection
Why this matters:
- Enterprise contracts have data retention guarantees
- We control what data leaves our systems
- Legal protection and compliance
AI Usage Tracking (Learning, Not Punishment)
We added fields to our PR template:
- “AI usage: None / Light / Heavy”
- “If AI-generated, what percentage?”
- “Tier: Red / Yellow / Green”
Purpose: Not to police, but to learn.
What we track:
- Correlation between AI usage and bug rates
- Which features benefit from AI vs which don’t
- Training needs based on AI usage patterns
- Effectiveness of governance framework
Critical: We made it clear this is for organizational learning, not individual punishment. Trust is essential.
Security-Focused Prompting Guidelines
We developed internal guidelines for AI prompting when allowed:
Every prompt should include:
- “Implement secure coding practices”
- “Validate all inputs”
- “Follow OWASP security guidelines”
- “Consider security implications”
Research shows this works: Security-aware prompts improve secure code generation from 56% to 66%.
We provide templates:
- Prompt templates for common tasks
- Examples of good vs bad prompts
- Security checklist to include in prompts
Monthly AI Office Hours
This has been surprisingly effective.
Format:
- Open forum every month
- Engineers share AI usage patterns
- Questions about when to use AI
- Security team provides guidance
- Share lessons learned
Benefits:
- Continuous learning and adaptation
- Community-driven best practices
- Reducing fear/confusion around policies
- Building AI literacy across team
Training: AI-Specific Security Review
We trained senior engineers on AI-specific code review:
What to look for:
- Common AI vulnerability patterns (SQL injection, XSS, etc.)
- Missing edge case handling
- Overly optimistic assumptions
- Copy-paste patterns without integration thinking
- Security-critical code that shouldn’t be AI-generated
Certification required to review yellow zone code.
Metrics Showing Improvement
Six months into this framework, we’re seeing positive results:
Security Metrics:
- Data leakage incidents: 0 (down from 1)
- Security vulnerabilities in production: Down 40%
- Audit findings related to code quality: Significantly reduced
Process Metrics:
- Developer clarity about AI usage: 85% report clear understanding
- Senior engineer review time: Stabilizing (peaked, now decreasing)
- Incident response time: Improved (better code quality)
Cultural Metrics:
- Developer satisfaction with governance: 72% positive
- Trust in leadership around AI policy: Recovered from ban damage
- Willingness to ask questions: High engagement in office hours
What’s Still Challenging
1. Tool Proliferation
New AI coding tools release constantly. We can’t evaluate them all. Have to balance innovation with security.
Current approach: Default to no unless there’s a strong case, then security review.
2. Varying Skill Levels
Engineers at different skill levels use AI differently. Junior engineers need more guidance and oversight.
Current approach: Tiered onboarding (covered in other threads about skill development).
3. Balancing Innovation with Risk
We don’t want to stifle innovation, but we can’t ignore security risks.
Current approach: Green zone encourages experimentation. Red zone maintains strict controls. Yellow zone is the pragmatic middle ground.
4. Competitive Pressure
Other EdTech companies using AI very aggressively, shipping faster (at least in appearance).
Challenge: Explaining to leadership why we’re more careful.
Current approach: Data. Show incident rates, security posture, long-term sustainability metrics.
Framework for Other Leaders
If you’re starting your AI governance journey, here’s what I’d recommend based on our experience:
1. Accept AI Is Here to Stay
Don’t fight adoption. Channel it productively.
2. Define Risk Zones, Not Blanket Policies
Not all code is equally sensitive. Differentiate your approach.
3. Make Security Training AI-Specific
Traditional security training isn’t enough. AI-generated code has specific vulnerability patterns.
4. Measure Verification Quality, Not AI Usage
Track: Are we catching issues? Not: Are people using AI?
5. Iterate Based on What You Learn
Start with a framework, but expect to evolve it. We’ve adjusted ours three times based on learnings.
6. Build Trust Through Transparency
Explain why policies exist. Make it about learning, not punishment.
7. Provide Clear Tools and Guidance
Decision trees, templates, office hours, Slack channels. Make it easy to do the right thing.
The Leadership Conversation
I mentioned competitive pressure. Here’s how I’ve approached the conversation with our CEO and board:
Their question: “Why aren’t we moving as fast as [competitor] who’s using AI heavily?”
My answer:
"We’re optimizing for different metrics:
- They’re optimizing for feature velocity
- We’re optimizing for sustainable growth and security
Their approach:
- Ship features 20% faster
- 23% more production incidents
- Higher customer churn due to reliability issues
- Security vulnerabilities that could become costly
Our approach:
- Ship features thoughtfully with AI assistance
- Maintain security and quality standards
- Build customer trust through reliability
- Develop engineering capability for long-term success
The bet: Reliability and security win long-term in EdTech. Parents and schools care about student data protection. One security breach could end our company.
**Speed matters. But not at the cost of security and trust."
Result: Leadership aligned with our approach. They’ve stopped comparing our velocity to competitors’ and started focusing on our quality metrics.
The Cultural Shift Required
Implementing governance isn’t just about policies. It’s about culture change.
From: “Move fast and break things”
To: “Move safely and verify everything”
From: “AI makes us more productive”
To: “AI is a tool we use thoughtfully”
From: “Maximize velocity”
To: “Maximize sustainable value delivery”
This shift requires:
- Leadership modeling the behavior
- Performance reviews reflecting the values
- Celebrating security catches, not just features shipped
- Transparency about tradeoffs
- Continuous learning and adaptation
What Success Looks Like
After six months, here’s what “working” looks like for us:
Engineers:
- Clear understanding of when/how to use AI
- Confidence in their decisions
- Feel supported, not restricted
- Developing both AI proficiency and fundamental skills
Security:
- Reduced vulnerability rates
- No data leakage incidents
- Improved security posture
- Faster incident response
Leadership:
- Confidence in our governance approach
- Alignment on quality over speed
- Data supporting our strategy
- Trust in engineering judgment
Customers:
- Reliable product experience
- Strong data protection
- Fewer incidents affecting them
- Trust in our platform
The Ongoing Journey
We’re not done. This is an ongoing journey, not a destination.
What’s next:
- Expanding our approved tool list carefully
- Refining our training programs based on feedback
- Building better automation for AI code detection and routing
- Continuing to learn and adapt
The goal: Enable our engineers to use AI as a powerful productivity tool while maintaining the security, quality, and reliability our customers depend on.
Questions for the Community
I’d love to hear from other engineering leaders:
1. What governance approaches are you trying?
What’s working? What’s not?
2. How are you handling the leadership conversation?
Especially around competitive pressure and velocity expectations.
3. What metrics are you tracking?
How do you measure success of your AI governance?
4. What challenges are you facing?
What hasn’t been solved yet?
We’re all figuring this out together. The more we share, the faster we collectively learn.
Thank you to this community for the thoughtful discussions that helped shape our approach.