Your Model Thinks Your Stack Is Two Years Old

There is a class of AI-generated bug that passes code review almost every time, and it isn't the hallucinated function or the fabricated package. It's the perfectly idiomatic code — idiomatic for the version of your stack that existed when the model's training data was frozen. The model writes a tailwind.config.js for a project running Tailwind v4, reaches for a class component lifecycle method in a hooks codebase, or calls an API that was deprecated three minor versions ago and removed in the one your lockfile actually pins. Nothing about the code looks wrong. It looks like code from a well-regarded tutorial. The tutorial is just from 2024.

Call it the training-cutoff bug class: defects that exist not because the model reasoned badly, but because the model's knowledge of your dependencies has a timestamp and your lockfile doesn't care. An ICSE 2025 study of seven code models across eight popular Python libraries found deprecated-API usage rates of 25–38% in plausible completions — and when the surrounding code already contained outdated patterns, that rate climbed to 70–90%. These aren't rare edge cases. They're the default failure mode of asking a frozen artifact to write for a moving target.

The uncomfortable part is that review doesn't catch these bugs, because review is calibrated for a different threat. Human reviewers scan for logic errors, missing edge cases, and unfamiliar patterns. Training-cutoff bugs are the opposite of unfamiliar — they are the most familiar possible code, the statistical center of everything ever written about that library. A reviewer who learned React in 2022 will nod right past a pattern that React 19 quietly broke. The code doesn't smell. It just doesn't run — or worse, it runs with silently changed semantics.

Why This Bug Class Is Invisible by Construction

Three properties make training-cutoff bugs uniquely hard to catch with the instincts and tooling most teams already have.

First, the code is locally correct. A hallucinated API fails fast: the import errors out, the function doesn't exist, CI goes red in seconds. A stale API often exists — it's deprecated, aliased, or subtly changed. Pandas kept DataFrame.append around for years after concat replaced it. The code completes, the tests you didn't write don't fail, and the deprecation warning scrolls past in a wall of install output. Research on API evolution found that the most confusing changes for models weren't dramatic removals but minor ones — package refactors, renamed keyword arguments, moved modules — precisely the changes that produce code which almost works.

Second, the error is distributed, not localized. A logic bug lives in a function. A staleness bug lives in the relationship between every generated line and a version number in a file the model never read. When an agent scaffolds a component, wires a config, and writes a migration in one pass, the 2024-isms are smeared across the diff. There's no single line to point at in review, which is exactly why nobody points at anything.

Third, the model is confident, and confidence is contagious. Studies of version-conditioned generation — asking a model to write code for a specific library version — found frontier models succeeding only about half the time, while sounding equally sure in both halves. Worse, the same research showed that even when correct, current documentation was placed directly in context, models frequently regressed to their memorized version mid-generation. Stale parametric knowledge isn't a gap to be filled; it's an active prior that fights your corrections.

The dependency-selection variant is nastier still. When models pin versions themselves — in a requirements.txt, a package.json, a Dockerfile — they reproduce the version distribution of their training data. Measurement studies comparing LLM-specified versions against vulnerability databases found models routinely pinning releases with known CVEs, because the model isn't choosing a version; it's remembering one. The code is correct. The dependency is a two-year-old snapshot of your supply chain.

Stop Treating It as a Model Problem

The instinctive framing — "the model is out of date, wait for the next release" — is wrong twice over. It's wrong empirically: model knowledge lags months to a year behind release, and your dependencies ship weekly; the window never closes. And it's wrong architecturally, because it puts the fix somewhere you don't control.

The productive framing is that knowledge staleness is a dependency-management problem, and dependency management is something engineering teams are already good at. You don't hope your CI "knows" the right version of a library — you pin it, you lock it, you audit it, and you get alerted when it drifts. The model's knowledge of your stack deserves exactly the same treatment: it's an implicit dependency with a version (the training cutoff) that is perpetually behind and can't be upgraded on your schedule.

Once you frame it that way, the mitigations stop being tips and start being infrastructure:

• Pinned model knowledge needs a diff against your lockfile. For each major dependency, you either know the model's knowledge predates your pinned version, or you're guessing.
• Staleness needs a feedback channel that fires at generation time, not at code review or production.
• Corrections need to live in context, because the weights won't change and, as the research shows, even in-context docs need to be aggressive to beat the memorized prior.

Docs Injection, Keyed to the Lockfile

The first working mitigation is injecting current documentation into the model's context — but the naive version of this fails in an instructive way. Pasting "the latest docs" helps only if the docs match the version you actually run. A team on Next.js 14 that injects Next.js 15 docs has replaced one staleness bug with another, and this time the model has an authoritative-looking source backing its wrong answer.

The unit of docs injection should be the lockfile entry, not the library name. This is what tools in the Context7 mold got right: resolve the library and version from the project, then fetch documentation for that exact release into context at query time. If you build this in-house, the pipeline is mechanical — parse the lockfile, map each critical dependency to a docs source (an llms.txt, a changelog, a migration guide), and inject the slice relevant to the current task. The hard part isn't retrieval; it's the discipline of keying retrieval to the resolved version rather than to "latest."

Migration guides deserve special priority in that context budget. A model producing pre-v4 Tailwind doesn't need the full v4 reference — it needs the delta: "the JavaScript config is gone, theme lives in CSS @theme blocks, the init command no longer exists." Changelogs and upgrade guides are the densest possible encoding of exactly the knowledge the model is missing, because they're written by maintainers specifically to correct people (and now models) who learned the old way. A few hundred tokens of "what changed since 2024" outperforms thousands of tokens of general documentation, because it targets the prior directly instead of hoping to outweigh it.

Lint Rules Are the Feedback Channel You Already Own

Documentation injection is preventive and probabilistic. The second mitigation is corrective and deterministic: make your toolchain reject stale patterns loudly, at the moment of generation.

This works because of an asymmetry worth internalizing: models are bad at knowing what's current, but agents are good at responding to structured errors. An agent that writes tailwind.config.js from memory will happily keep going; the same agent, shown a lint error saying "tailwind.config.js is not used in v4 — theme configuration belongs in CSS," fixes it immediately and correctly. The knowledge doesn't need to be in the weights or even in the prompt. It needs to be in the failure message.

Concretely, this means promoting things most teams treat as optional hygiene into the agent's inner loop:

• Turn deprecation warnings into errors in the environment where generated code first runs. A warning scrolls past an agent exactly the way it scrolls past a human; an error becomes a tool result the agent must handle.
• Adopt rulesets that encode API evolution — ESLint plugins that flag removed React patterns, Ruff and pyupgrade rules that catch superseded Python idioms, cargo clippy lints tracking Rust API churn. Every rule is a unit of post-cutoff knowledge delivered exactly when it's needed, with zero context-window cost until it fires.
• Write custom rules for your internal libraries. Your in-house framework changed its API last quarter; no model will ever have that in weights. A lint rule that says "use createClient instead of Client() — changed in v3.2" is you patching the model's knowledge through the one channel guaranteed to reach it.

Type checkers do this for free where signatures changed, which is one of the strongest practical arguments for typed codebases in the agent era. But types miss same-signature semantic changes, and they miss config files entirely — the lint layer is what covers the gap. The through-line: every correction you can move from "reviewer memory" into "machine-checkable rule" converts an invisible bug class into a self-healing loop.

Auditing Your Staleness Surface

Prevention and correction both assume you know where you're exposed. Most teams don't. The audit is simple enough to run this week:

1. List your load-bearing dependencies — the ten or twenty libraries where most generated code touches an API. Not everything in the lockfile; the ones agents actually write against daily.
2. For each, find the last breaking or idiom-shifting release date — not the version number, the date. Major versions, config format changes, "new recommended way" announcements.
3. Compare against your models' training cutoffs. Every dependency whose last idiom shift postdates the cutoff is a staleness hazard; every one where the shift is recent and the old idiom was popular for years is a severe one. Tailwind v4, released January 2025 against years of v3 training data, is the canonical severe case: maximal prior, maximal drift.
4. For each hazard, decide the mitigation tier: a line in the project's agent instructions ("we use Tailwind v4 — never create tailwind.config.js"), injected version-keyed docs for heavy-use libraries, or a lint rule where violations are frequent and mechanical.

Then treat the audit as a living artifact, because it decays on two clocks: rerun the delta whenever a major dependency upgrades, and whenever you switch or upgrade models — a newer cutoff retires some hazards and your next major upgrade creates new ones.

There's also a quiet strategic implication for anyone who maintains a library or platform: models are now a distribution channel for your old API. Every year your deprecated pattern lives in tutorials is a year it compounds in training data, and teams increasingly experience "hard to use with AI assistants" as plain "hard to use." Shipping an llms.txt, agent-readable migration guides, and lint rules alongside a breaking release is no longer documentation polish — it's how you patch a million models you don't control.

The training-cutoff bug class isn't going away; frozen weights and moving ecosystems guarantee it. But it's also not mysterious. It behaves like any other dependency-drift problem: invisible if you rely on memory and review, tractable the moment you pin it, lint it, and audit it. Your model thinks your stack is two years old. That's fine — as long as your tooling knows what year it actually is.