Three months ago our CFO dropped a challenge on my team: Show me that our AI investments are working or I am cutting the budget.
The problem? Our existing metrics were not cutting it. We could show adoption rates and developer satisfaction but we could not connect AI usage to business outcomes in a way that satisfied our finance team.
So we went looking for better frameworks and we found GAINS: Generative AI Impact Net Score.
What Is GAINS
GAINS is an emerging framework designed to measure AI maturity across organizations. Unlike simple productivity tracking it attempts to:
- Measure AI adoption (are people using the tools?)
- Identify organizational friction (what is blocking value creation?)
- Connect usage to outcomes (what business impact are we seeing?)
The framework assigns a score from 0-100 that represents organizational AI maturity not just tool adoption. Think of it like NPS (Net Promoter Score) but for AI readiness.
The 90-Day Pilot
We ran a pilot with our 40-person core engineering team. Here is what we tracked:
Week 0-2: Baseline (current AI tool usage patterns, existing productivity metrics, team satisfaction and friction points)
Week 2-8: Active Measurement (daily adoption tracking, weekly friction surveys, biweekly outcome measurement)
Week 8-12: Analysis and Adjustment (identified bottlenecks, made process changes, re-measured)
The Surprising Findings
Finding 1: High Usage Does Not Equal High Impact
We had several developers using GitHub Copilot heavily (90 percent plus of their code sessions) but their delivery velocity was not improving. Why?
Turns out: they were generating code faster but our code review process was the bottleneck. AI-generated code was sitting in PR queues for 3-5 days waiting for review.
The fix: We expanded our code review capacity by training more engineers to review effectively and implemented async code review rotations. Suddenly the AI productivity gains actually flowed through to shipping features.
Finding 2: Organizational Friction Is Invisible Without Framework
GAINS includes a friction index that asks developers what prevents you from getting full value from AI tools.
The top answers surprised us: Unclear when to use AI versus when to write code manually (38 percent), AI suggestions conflict with our coding standards (31 percent), Review process does not account for AI-assisted code (28 percent), Lack of training on effective AI usage (24 percent).
These were not technical problems. They were process and culture problems. We could have bought more AI tools but it would not have helped. We needed organizational changes.
Finding 3: The Value Emerges in Unexpected Places
We expected AI to accelerate feature development (it did modestly). But the biggest gains were: Onboarding (new engineers productive 40 percent faster because AI helped them understand unfamiliar codebases), Maintenance (bug fixes and tech debt work 55 percent faster), Documentation (engineers actually writing docs because AI makes it less painful).
We were not measuring these initially. GAINS framework prompted us to look at broader organizational impact not just features shipped.
The Business Case That Worked
After 90 days here is what I presented to our CFO:
GAINS Score: 64/100 (baseline was 38/100). Adoption improved from 45 percent to 78 percent. Friction decreased by 60 percent. Organizational readiness rating: Emerging Maturity.
Business Impact: Cycle time reduced 22 percent, onboarding time reduced 40 percent, maintenance velocity up 55 percent, developer satisfaction up 18 points.
Translation to CFO Language: Faster time-to-market equals earlier revenue capture (about $400K ARR impact), faster onboarding equals hiring efficiency gain (about $120K per new engineer), more maintenance capacity equals technical debt reduction (estimated $200K incident prevention), better retention equals avoided replacement costs (about $300K saved on reduced attrition).
Total estimated annual impact: about $1.8M value created from $380K AI tool investment.
Our CFO approved continued investment and asked me to expand the measurement framework to other teams.
The Framework Is Not Perfect
Challenges we are still working through: Measurement overhead (the initial 90-day pilot required significant time investment), attribution complexity (hard to prove causation versus correlation), survey fatigue (weekly pulse checks annoyed some engineers), scoring calibration (the 0-100 score feels somewhat arbitrary).
But even with these limitations GAINS gave us a structured way to have ROI conversations with finance that our previous ad-hoc metrics could not provide.
The Key Insight
AI productivity is an organizational capability not just a tool adoption metric.
You can give every engineer the best AI tools in the world but if your processes culture and workflows do not support AI-assisted work you will not see the value.
GAINS forced us to look at the whole system: tools plus processes plus culture plus skills. That is where the insights came from.
Question for the Community
Is anyone else experimenting with structured AI measurement frameworks?
I would especially love to hear from folks who have tried GAINS or similar maturity models, custom frameworks you have built, what worked and what did not.
We are still early in this journey. The frameworks are immature the best practices are still emerging and frankly we are all figuring this out together.
But the pressure from CFOs is not going away. Having a structured approach to measurement—even an imperfect one—is better than having no approach at all.