You Didn't Choose a Model, You Married One: The Prompt-Level Lock-In Nobody Budgets For

Ask any engineering leader whether they're locked into a model vendor and they'll point you at the abstraction. "We route everything through a gateway. Swapping providers is a config change." The endpoint is one line. The base URL is an environment variable. On paper, migration is a Tuesday afternoon.

Then they try it. They flip the config to a different model family, the integration tests stay green, and production quietly falls apart. The JSON that always parsed now arrives wrapped in a markdown fence. The classifier that hit 94% drops to the low eighties. A prompt that ran clean for a year starts refusing one request in twenty for reasons no one can reproduce. The endpoint swapped in seconds. The behavior did not come with it.

This is the lock-in nobody budgets for. It doesn't live in your contract or your SDK. It lives in your prompts — thousands of small accommodations your team made, one at a time, to the quirks of a single model family. You didn't choose a model. You married one, and the prenup is every prompt you've ever shipped.

The lock-in is in the prompt, not the API

When people picture vendor lock-in, they picture infrastructure: proprietary SDKs, custom tokenizers, embeddings you can't regenerate, monitoring wired to one dashboard. That layer is real, and a decent gateway like an OpenAI-compatible router papers over most of it. The request shape normalizes. The response schema normalizes. The bill consolidates.

But the gateway only standardizes the envelope. It cannot standardize what's inside the prompt, and that's where the real coupling accumulated. Every model family has a personality — a set of formatting habits, instruction-following idioms, refusal reflexes, and reasoning styles. Your prompts didn't get written in a vacuum. They got written against that personality, then tuned against it every time something broke.

Consider how a single prompt actually evolves. The first draft asks for JSON and the model occasionally adds a preamble, so someone appends "Return only valid JSON, no other text." The model sometimes hedges, so someone adds "Be decisive." It over-refuses a benign edge case, so someone adds a sentence reassuring it the request is legitimate. Six months later that prompt has eleven such patches, and every one of them is a fingerprint of the model it was tuned against. Move to a new family and the patches don't just stop helping — several of them actively hurt, because they were correcting a tendency the new model never had.

Multiply that by a few hundred prompts across your product and you have the migration cost nobody put on the roadmap. Industry estimates put real model migrations at 20–50% of the original development time, and the dominant line item isn't engineering the swap — it's re-deriving prompt behavior you thought you owned but were actually renting.

Three quirks you've been silently absorbing

The lock-in compounds because the accommodations are invisible. They feel like "how prompting works" rather than "how this model works." Three categories cause the most damage on a switch.

Formatting and output discipline. Models differ sharply in how reliably they honor output contracts. One family returns clean JSON when you ask; another wraps it in a code fence by default; a third tends to narrate before the payload unless you fight it. Your parsing layer encodes the assumption of whichever one you started with. Even native structured-output and function-calling features differ in strictness — what counts as a valid schema, how enums are handled, whether the model will invent fields under ambiguity. The day you switch, your downstream parsers meet outputs they were never written to tolerate.

Refusal boundaries. This is the most underappreciated form of lock-in, and the hardest to test. Every provider draws its safety line in a slightly different place, and that line is not a clean threshold — it's a noisy local decision boundary. Recent evaluation work found that even when two models from the same vendor refuse over 94% of clearly disallowed prompts, roughly a third of base prompts flip their refusal behavior under small, meaning-preserving rewordings. Across different vendors the variance is larger still. Your prompts have quietly been shaped around your current model's exact boundary — the phrasings that get through, the reassurances that unlock a benign-but-borderline request, the topics you learned to route around. None of that transfers. After a switch, you discover the new boundary the way users discover it: in production, one false refusal at a time.

Loading…