Your AI Roadmap Needs a Decommission Column

Look at any AI product roadmap and you'll find columns for the things you're going to launch: a new agent, a better retrieval pipeline, a model upgrade, a fine-tune. Every item is a birth. Nothing on the board is a death. There's no column for the model you'll stop supporting, the prompt you'll retire, the eval set you'll archive, the feature you'll turn off. Roadmaps are optimistic by construction — they're a list of futures you're promising to create.

That omission is fine for deterministic software, where a feature you shipped two years ago mostly keeps working until you touch it. It is quietly catastrophic for AI systems, where the things you shipped are decaying whether you touch them or not. A prompt tuned to GPT-4's quirks degrades the day the provider nudges the model. An eval suite written against last year's failure modes stops measuring anything real. A model you depend on gets a 60-day deprecation notice you didn't plan for. A roadmap that only plans launches is accruing silent liability on every line it doesn't track.

The fix isn't more discipline applied to the same board. It's a structural change: add a decommission column, and treat sunset as a first-class deliverable with an owner, a date, and a definition of done — the same way you'd treat a launch. This post is about why AI specifically forces that change, what belongs in the column, and how to run it without it becoming a graveyard of good intentions.

Why AI Decays Differently Than Software

Traditional technical debt is something you take on by choice. You ship a shortcut, you write an IOU to your future self, and the debt sits there at a fixed interest rate until you pay it down. It doesn't get worse on its own. The code you wrote in 2022 does exactly what it did in 2022.

AI debt doesn't work that way, because three of its core ingredients rot from the outside:

• Models rot because the provider moves them. Frontier providers retire models on a roughly 12-to-18-month cadence, often with only weeks of warning before an endpoint goes dark. You don't choose the timing; you receive it. When a model retires, applications tightly coupled to it require re-architecture, and the "improved" replacement exhibits different biases and generates subtly different responses even when it benchmarks higher.
• Prompts rot because they were tuned to behavior, not spec. A production prompt is rarely a clean specification. It's an accretion of patches, each one fitted to the behavioral quirks of a specific model version. When that model shifts under you, the patches that compensated for its weaknesses become noise — or active liabilities. Studies have shown that formatting and phrasing changes alone can swing few-shot accuracy by dozens of points. A prompt is implicitly versioned against a model you no longer have.
• Eval sets rot because the world moves past them. An evaluation suite is a snapshot of the failure modes you knew about when you wrote it. The longer it runs unchanged, the more it measures a problem you've already solved and the less it catches the problems you actually have now. A green eval dashboard on a stale suite is worse than no dashboard, because it manufactures confidence.

Put those together and you get a system where doing nothing is not a neutral act. The launch you shipped six months ago is not holding its value. It's depreciating, and the depreciation is invisible until something breaks in production or a deprecation email lands in an inbox nobody's watching.

That's the core asymmetry the roadmap has to absorb. In software, the default state of a shipped thing is stable. In AI, the default state of a shipped thing is decaying. A planning artifact built for the first world will systematically under-budget the second.

What Belongs in the Decommission Column

The column is not a backlog of cleanup tickets. It's a forward-looking ledger of things you have explicitly decided to end, each with the same metadata you'd demand of a launch: an owner, a target date, a trigger condition, and a definition of done. Here's what actually goes on it.

Model versions you depend on, with their expected end-of-life. Every model in production gets a row the day you adopt it, not the day the provider warns you. The row carries the provider's stated support window (or your best estimate of it), the date you'll begin migration testing, and the fallback you'll cut to. The point is to convert an external surprise into an internal milestone. When the deprecation notice arrives, the work is already scheduled instead of being an emergency.

Prompts and prompt versions that are pinned to a retiring model. When a model goes on the decommission column, every prompt tuned to it inherits a dependency. Those prompts need their own line: rewrite, re-test against the successor, or retire entirely if the feature they serve is going away. This is where prompt debt gets paid down deliberately instead of discovered during an outage.

Eval sets and benchmarks that no longer reflect reality. An eval suite should have a review-by date the way food has an expiry date. Past that date, someone owns the decision to refresh it, re-weight it toward current failure modes, or archive it. Archiving a stale eval is a real deliverable — it stops the suite from lying to you.

Features and surfaces you're turning off. Not every AI feature earns its keep. Some were experiments that shipped, found a small audience, and now cost more in maintenance, monitoring, and migration drag than they return. The decommission column is where you name them and set a sunset date, using the same retirement triggers portfolio managers have used for legacy applications for years: low usage, high cost relative to value, security or compliance risk, and redundancy with a newer surface.

Fine-tunes and adapters built on a base model that's going away. A fine-tune is downstream of its base. When the base is deprecated, the fine-tune is on borrowed time, and re-training is not free. That dependency belongs on the board the moment the base model's clock starts.

Loading…