The AI Feature OKR Mismatch: Why Quarterly Cadence Breaks AI Roadmaps

The team commits to "ship the AI summarizer this quarter," gets it past the technical bar by week ten, takes a victory lap at the all-hands, and ships. Six weeks later the telemetry curve starts bending the wrong way — quietly, slowly, in a way nobody dashboards because nobody owns the shape. The OKR is already marked green. The next quarter's OKRs are already drafted around new launches. The summarizer is now somebody's second-priority maintenance job, and by quarter-end review the team is wondering why customer satisfaction on the feature dropped fifteen points without anything obvious changing.

This is not a bug in the team. It's a bug in the operating model. Quarterly OKRs were calibrated for software where a feature can be scoped, built, shipped, and then largely left alone until the next major rev. AI features don't have that shape. They have a launch curve and a sustain curve, and the sustain curve is where most of the value — and most of the risk — actually lives. The OKR template that treats them as deliverables with launch dates quietly produces a portfolio of demos that decay before the next planning cycle.

Why deterministic OKRs fit deterministic software

The standard quarterly OKR contract assumes three things. The feature is scopable in advance — you can write down "ship X" and know roughly what X means. The feature is measurable on day one — a checkout flow either completes purchases or it doesn't, and you can ship a regression-test suite that proves correctness. And the feature is stable after launch — barring a customer-found bug, the code that worked in week twelve still works in week twenty.

None of these hold for AI features in any meaningful way. The eval suite that tells you whether the feature is shippable doesn't exist when scoping starts; it has to be built as a deliverable in its own right, and writing a good eval is often harder than building the prototype it scores. The prompt that works on hand-picked test cases rarely survives the first distribution shift to real traffic, where users phrase things you didn't anticipate, paste in malformed input, and chain requests in ways your prototype never saw. And the system is unstable after launch by design — not because the code changed, but because the model changed, the tokenizer churned, the retrieval index drifted, and user behavior moved as the feature taught users how to use it.

Industry data backs this up bluntly. A pattern often cited as "the 90-day degradation curve" describes how production LLM features quietly lose quality across the first quarter of life — not from failure modes anyone codified, but from the accumulation of small drifts. Gartner's projection that 30% of generative AI projects will be abandoned after proof of concept by end of 2025 is, charitably, the same pattern viewed from one quarter further out. The features got built. The features didn't get sustained.

What "done" actually means for an AI feature

The reframing leadership has to drive — and has to drive repeatedly, because the deterministic-software muscle memory is strong — is that an AI feature isn't done when it ships. It's done when it has a sustained-quality plan and a named owner.

That sentence does a lot of work. "Sustained-quality plan" means there is a defined metric for what good means in production, a defined cadence for measuring it, a defined threshold for when to act, and a defined runbook for what acting looks like. "Named owner" means a single person whose name is on the dashboard, whose calendar has the review meeting on it, and whose performance review reflects whether that curve trended up or down. Without these, the feature ships into a ghost ownership state where quality regressions become collective-action problems that get triaged by whoever has the loudest customer the day the complaint lands.

The corollary is that the OKR shouldn't say "ship the AI summarizer." It should say something closer to "ship the AI summarizer at quality threshold X with eval coverage Y, then sustain quality threshold X across the following quarter with named owner Z." That's not a single key result. It's a paired commitment — a launch metric and a sustained-quality metric — tracked across a minimum of two quarters. The shape of the OKR has to match the shape of the work, and the work doesn't end at launch.

Eval coverage as a leading indicator, not a deliverable

In deterministic software, tests are a tax you pay for the privilege of refactoring later. They're a deliverable in the sense that "did you write tests" is a code-review question, but they're not a leading indicator of anything meaningful. The build either passes or it doesn't.

In AI, eval coverage is a leading indicator of whether the feature is shippable at all. The team that scopes an AI feature without a held-out eval set, a calibrated judge, a frozen baseline, and an attribution discipline that distinguishes "the prompt got better" from "the judge got more lenient" is scoping a feature it cannot measure. And a feature it cannot measure is a feature it cannot defend against drift, regress against, or improve against. The eval suite is the substrate the rest of the work runs on.

This implies a hard prerequisite that has to land before the planning cycle starts: eval coverage isn't an OKR; it's a gating criterion for whether a feature can be an OKR. The team that hasn't built the eval set for a domain doesn't get to scope a feature in that domain this quarter. They get to scope the eval. This will feel like a step backwards to leadership the first time it's enforced. It will feel like the only sane way to plan AI work the second time.

The corollary is that eval coverage on shipped features is a maintained property, not a one-time deliverable. Models get upgraded. Tokenizers churn. Tool schemas drift. The eval that was calibrated against last quarter's checkpoint may be measuring noise this quarter unless it's been re-baselined against the current production stack. Maintenance of the eval suite has to be a line item, not a slack-time project.

The maintenance burden that scales with feature count

Once you have one AI feature in production, the maintenance burden is a small fraction of one engineer's time. Once you have ten, it is no longer a small fraction. The math is mechanical: each shipped feature has a prompt that needs revalidation, an eval set that needs upkeep, a retrieval index that needs reindexing, a tool schema that may have drifted, a model alias that may have resolved to a new checkpoint, and a quality dashboard that needs a human to read it. None of this work shows up on the launch roadmap. All of it shows up on the maintenance docket.

The staffing math that scales is the inversion of the deterministic-software default. In deterministic software, the ratio of net-new feature engineers to maintenance engineers can stay heavily weighted toward new work for a long time, because the maintenance burden of a shipped feature decays toward near-zero. In AI, the maintenance burden of a shipped feature does not decay — it stays roughly constant per feature, and the total burden grows linearly with the number of shipped features. Once you cross a threshold (somewhere around five to ten production AI features for most teams), the ratio has to invert. Otherwise, you ship more than you can sustain, and the portfolio decays under its own weight.

This is the line item that doesn't get written into roadmaps and quietly produces the symptom that everything ships but nothing improves. The team is busy. The dashboard is full. The features are all marked "live." And the quality curve on each of them is bending the wrong way because nobody has the cycles to maintain them.

Designing OKRs that match the shape of the work

A few patterns help align the OKR template with the actual physics of AI features.

Replace launch metrics with paired commitments. Every shipped feature has two metrics: a launch metric (did we hit threshold X by date Y) and a sustained-quality metric (did we hold threshold X across the subsequent N weeks). The second metric is tracked across the planning boundary — quarter Q+1 inherits a quality KR from quarter Q, automatically, for every AI feature that shipped in Q. This forces the planning cycle to allocate sustain capacity rather than only launch capacity.

Make eval coverage a gating criterion. Features without sufficient eval coverage cannot be scoped for the quarter. The work to build the eval is its own deliverable. This stops the recurring failure mode where a feature gets committed to a roadmap that the team does not yet have the instruments to measure.

Allocate explicit sustain headcount. Maintenance of shipped AI features is a line item with a name on it, not slack time on the corner of someone's calendar. The fraction varies, but for most teams running more than a handful of AI features it's not a rounding error. If it isn't budgeted, it isn't happening.

Ship behind flags with explicit rollout plans. Replace "launch on date Y" with "behind a flag at N% rollout, expand to M% based on metric Z." This decouples the launch event from the quality bar and makes the rollout itself a measurable, reversible process rather than a fait accompli.

Track AI debt explicitly. Prompts older than their revalidation date, retrieval indexes past their refresh window, models pinned to deprecated checkpoints — each is a maintenance debt item that should appear on a tracked list, the way infra teams track deprecated dependency versions. If it's invisible, it's accruing without a cap.

The architectural realization

The deepest version of this argument is that AI features are products, not deliverables. A deliverable has a launch date and an end state. A product has an operational lifecycle, an owner, a quality baseline maintained against entropy, and a roadmap that's mostly about sustaining and incrementally improving rather than about shipping the next thing. The deterministic-software OKR template treats features as deliverables and is broadly right; the AI OKR template that copies it treats AI features as deliverables and is broadly wrong.

This isn't a small framing tweak. It changes who gets hired, how teams are structured, how performance is reviewed, how quarterly planning works, and how leadership measures whether the AI portfolio is healthy. A team that has internalized "AI features are products with lifecycles" will allocate sustain capacity, hire for maintenance temperament, and treat eval coverage as infrastructure. A team that hasn't will keep shipping demos that decay between planning cycles, blaming model providers when the curves bend the wrong way, and rediscovering the same lesson every two quarters.

The teams that get this right won't necessarily ship faster than the ones that don't. They'll ship at the same rate, but their portfolio will be healthy six months later. The teams that get it wrong will look productive for one cycle and then spend the cycle after that triaging the consequences of the cycle before. The OKR template is a small artifact, but the operating model behind it is the thing that compounds — and AI is unforgiving of operating models that pretend the work is simpler than it is.