The AI Roadmap Priced In a Model That Doesn't Exist Yet

There is a specific phrase that should make every engineering leader stop the meeting: "This will just work once the models get better." It is usually said with confidence, sometimes with a slide showing a capability curve bending upward, and it almost always marks the moment a roadmap quietly stopped being a plan and became a forecast.

The distinction matters more than it sounds. A plan is a sequence of things you control: code you will write, integrations you will build, tests you will run. A forecast is a bet on something you don't control — in this case, a model that doesn't exist yet, shipping on a schedule set by a vendor who has never promised you a date. When you scope a feature around "the context window will double" or "reasoning will be good enough by Q3," you have not de-risked the hard part of the problem. You have relocated it onto someone else's release calendar and called it done.

This is not an argument against ambition. Capability is improving fast, and pretending otherwise is its own kind of malpractice. METR's widely-cited measurements show the length of tasks that frontier agents can complete autonomously roughly doubling every seven months over the last six years — and closer to every four months in the 2024–2025 window. That is a real, steep curve, and building as if models will stay frozen is how you get lapped. The problem is not optimism. The problem is that "the model will improve" is being used to close design questions that improving models will not actually answer.

Forecasting Loss Is Not Forecasting Behavior

The scaling-laws literature is genuinely impressive at predicting one thing: training loss. Given compute, data, and parameter counts, you can forecast how well a model will fit its training distribution with striking accuracy. This is the empirical bedrock that lets labs plan multi-hundred-million-dollar training runs without flying blind.

Here is the catch that rarely makes it into the roadmap deck: loss is not the thing your feature depends on. Your feature depends on reasoning, robustness, instruction-following under adversarial input, calibration, and the absence of a specific failure mode that breaks your specific workflow. And the same researchers who trust scaling laws for loss are explicit that those laws are much weaker as a proxy for reasoning or general capability without task-grounded evaluation. Newer forecasting work — approaches that predict benchmark performance rather than loss — exists precisely because the loss-to-capability translation is unreliable.

So when someone extrapolates "GPT-N was 60% on our internal eval, GPT-N+1 will be 80%," they are drawing a straight line through a space where straight lines are known not to hold. Capability gains are lumpy, uneven across task types, and occasionally negative on the specific thing you care about even as aggregate benchmarks climb. GPT-5's 2025 launch was the object lesson: aggregate numbers went up, and yet a meaningful slice of users found it worse at their particular tasks than the model it replaced, to the point where the vendor had to walk back the forced migration. If you had priced your roadmap on "the next model will be strictly better at everything," you were wrong on arrival.

The Delivery-Date Fallacy

Even if you assume the capability will arrive — and eventually, for many things, it will — you have made a second, sneakier bet: that it arrives when you need it, and that you can access it when it does.

Both halves of that assumption have gotten shakier, not firmer. Frontier launches now slip, get staged behind safety review, and occasionally get gated behind approval processes that have nothing to do with your product timeline. A two-week delay sounds trivial until it is the two weeks between your launch date and a model that doesn't exist yet. And "we'll upgrade the moment it drops" assumes drop-in compatibility that rarely survives contact — new models bring new tokenizers, new refusal behaviors, new latency profiles, and new prompt sensitivities that quietly break the harness you built around the old one.

The uncomfortable reframing: a roadmap that depends on an unreleased model has a critical-path dependency on a vendor's internal schedule, priced at zero. You would never accept a hardware plan built on a chip with no ship date and no SLA. Somehow "the model will be better by then" gets a pass because the curve looks so convincing on the slide.

How To Tell a Capability Bet From Wishful Thinking

Not every forward-looking bet is reckless. The teams that navigate this well are not the ones who refuse to anticipate progress — they are the ones who can articulate exactly what they are betting on and what happens if they lose. A few questions separate a defensible bet from a wish:

• What specifically has to improve, and by how much? "Reasoning will get better" is a wish. "The model needs to hold a 12-step tool-calling chain without losing the thread past step 6, and today's best model manages 4" is a bet. The second one you can measure, track against each release, and reason about. The first one is a mood.
• Is the improvement on-trend or off-trend? Betting that agents will handle somewhat longer task horizons next year is betting with a well-documented curve. Betting that a model will suddenly stop hallucinating structured output, or become calibrated about its own confidence, is betting on a discontinuity that the trend lines do not support. Know which one you're doing.
• What is your evidence that this is hard for the model and not hard for you? A shocking number of "the model can't do this yet" problems are actually retrieval problems, context-construction problems, or prompt problems wearing a capability costume. Those don't get fixed by a new model. They get fixed by you.
• What is the cost of being early versus late? Building the harness ahead of the capability is often cheap and smart. Shipping a product that only works once the capability lands is expensive and fragile. The difference is whether your thing is dead today or merely better tomorrow.

That last question is the whole game, and it deserves its own section.

Design So a Better Model Is Upside, Not Oxygen

The single most important architectural decision in an AI product is not which model you pick. It is whether a better model is upside or oxygen — whether the next release makes your product incrementally better, or whether your product literally does not function until it arrives.

Oxygen-dependent designs are seductive because they demo beautifully on a cherry-picked example and let you skip the unglamorous engineering. "Once the model is smart enough, we won't need the retrieval layer / the validation step / the human review / the fallback path." Every one of those deletions is a bet that a model doesn't exist yet will cover for work you chose not to do. Upside-dependent designs do the opposite: they work today, on the model you can actually buy, at a quality you can actually ship — and they get quietly better every time the underlying model improves, with no code change and no launch dependency.

Concretely, designing for upside looks like a few disciplines:

• Keep a working version on today's model. If your feature can't ship on the current best model at some acceptable quality bar, you don't have a roadmap item — you have a research bet, and it should be labeled and funded as one.
• Build the harness now, decouple the model. The scaffolding around the model — evals, retrieval, tool definitions, guardrails, observability — is real engineering you control and can start today. When a better model lands, you drop it into a harness that's ready, rather than starting the hard part on the day the clock is already running.
• Make model swaps a config change, not a rewrite. Abstract the provider boundary enough that testing next quarter's model against your eval suite is an afternoon, not a project. This is also how you turn "the new model regressed our task" from a launch-day catastrophe into a quiet non-upgrade.
• Instrument the gap. Track your feature's quality against each new release on a fixed eval. This turns "the model will get better" from an article of faith into a chart with a slope you can actually plan around — and gives you an early warning when a release moves the wrong way.

Do this, and the improving-capability curve becomes exactly what it should be: a tailwind that makes a working product better, rather than a crutch holding up a product that can't stand on its own.

The Conversation With Leadership

Eventually someone above you asks why the roadmap can't be more aggressive, why you're not "assuming the models will keep improving like everyone else." The honest answer is not "AI won't get better." It is a cleaner separation than most roadmap conversations ever achieve: which parts of this plan are engineering, and which parts are speculation.

Engineering is the work you can start Monday and finish on a date you'll mostly hit. Speculation is the capability you're hoping shows up. A healthy roadmap contains both — but it labels them, funds them differently, and never lets a speculative dependency sit silently on the critical path pretending to be a committed deliverable. When you can point at a plan and say "these three features ship on today's model, and these two get materially better if the next release lands as expected, and none of them are dead if it slips," you have given leadership something they can actually make decisions with. That is worth more than an aggressive-looking timeline secretly underwritten by a model that doesn't exist yet.

The curve is real. Capability is coming. But a plan you control that gets better when the model improves will beat a forecast you don't control every single quarter — including the quarters when the forecast happens to come true.