4 posts tagged with "roadmap"

There is a specific phrase that should make every engineering leader stop the meeting: "This will just work once the models get better." It is usually said with confidence, sometimes with a slide showing a capability curve bending upward, and it almost always marks the moment a roadmap quietly stopped being a plan and became a forecast.

The distinction matters more than it sounds. A plan is a sequence of things you control: code you will write, integrations you will build, tests you will run. A forecast is a bet on something you don't control — in this case, a model that doesn't exist yet, shipping on a schedule set by a vendor who has never promised you a date. When you scope a feature around "the context window will double" or "reasoning will be good enough by Q3," you have not de-risked the hard part of the problem. You have relocated it onto someone else's release calendar and called it done.

Look at any AI product roadmap and you'll find columns for the things you're going to launch: a new agent, a better retrieval pipeline, a model upgrade, a fine-tune. Every item is a birth. Nothing on the board is a death. There's no column for the model you'll stop supporting, the prompt you'll retire, the eval set you'll archive, the feature you'll turn off. Roadmaps are optimistic by construction — they're a list of futures you're promising to create.

That omission is fine for deterministic software, where a feature you shipped two years ago mostly keeps working until you touch it. It is quietly catastrophic for AI systems, where the things you shipped are decaying whether you touch them or not. A prompt tuned to GPT-4's quirks degrades the day the provider nudges the model. An eval suite written against last year's failure modes stops measuring anything real. A model you depend on gets a 60-day deprecation notice you didn't plan for. A roadmap that only plans launches is accruing silent liability on every line it doesn't track.

There are two clocks ticking on your AI product, and they are not synchronized. The model providers run on a roughly quarterly heartbeat — Claude Opus 4.6 in February 2026, GPT-5.4 in March, Claude Opus 4.7 in April, GPT-5.5 a week later. Your product roadmap was committed in January and does not look up again until July. Somewhere in between, a capability you spent eight engineer-weeks building gets shipped as a one-line API parameter, and nobody on the team has a process for noticing.

This is not a forecasting problem. The releases were widely telegraphed — anyone who reads the changelog could have seen each of them coming. It is a planning-artifact problem. Roadmaps were invented for a world where the platform underneath your product changed once a decade. The platform now changes once a quarter, and the artifact has not been updated to match.

The demo always works. That is the most expensive sentence in AI product development. The product manager sees the model handle the happy path, the engineer ships the obvious version of the feature, and six weeks later the support queue is full of complaints that the metric did not predict. Nothing in the model regressed. Nothing in the prompt got worse. The feature was simply not the shape the model could do well, and the team did not have a way to say so before the work began.

A meaningful fraction of shipped AI features fail this way — not because the model is bad, but because the task is wrong. The output the product needs is deterministic and the engine is stochastic. The user's tolerance for the tail is one bad answer per thousand and the model's failure distribution is heavier than that. The latency budget the unit economics require is half of what the model can deliver at any tier you can afford. The ground truth required to evaluate quality does not exist and cannot be cheaply created. None of these are model problems. They are task-shape problems, and they should have been screened before the first prompt was written.