4 posts tagged with "vendor-lock-in"

Ask any engineering leader whether they're locked into a model vendor and they'll point you at the abstraction. "We route everything through a gateway. Swapping providers is a config change." The endpoint is one line. The base URL is an environment variable. On paper, migration is a Tuesday afternoon.

Then they try it. They flip the config to a different model family, the integration tests stay green, and production quietly falls apart. The JSON that always parsed now arrives wrapped in a markdown fence. The classifier that hit 94% drops to the low eighties. A prompt that ran clean for a year starts refusing one request in twenty for reasons no one can reproduce. The endpoint swapped in seconds. The behavior did not come with it.

This is the lock-in nobody budgets for. It doesn't live in your contract or your SDK. It lives in your prompts — thousands of small accommodations your team made, one at a time, to the quirks of a single model family. You didn't choose a model. You married one, and the prenup is every prompt you've ever shipped.

Your best AI engineer can recite Claude's XML formatting preferences from memory. They know that Claude Opus refuses to generalize implicit instructions, that few-shot examples actually hurt performance on o1-series models, and that Azure OpenAI imposes an extra 8–12 seconds of latency versus the direct API in some regions. This expertise took months to accumulate. It also represents one of the most underappreciated risks in AI engineering today.

When a provider deprecates a model or silently shifts behavior, that knowledge doesn't transfer. It vanishes. And teams that built their systems — and their institutional competence — around a single model family often discover this the hard way.

Everyone talks about avoiding LLM vendor lock-in. The advice usually boils down to "use an abstraction layer" — as if swapping openai.chat.completions.create for litellm.completion solves the problem. It doesn't. The API call is the easy part. The real lock-in is invisible: it lives in your prompts, your evaluation data, your tool-calling assumptions, and the behavioral quirks you've unconsciously designed around.

Provider portability isn't a boolean. It's a spectrum, and most teams are further from the portable end than they think. The good news is that the patterns for genuine portability are well understood — they just require more discipline than dropping in a wrapper library.

Every team that ships an LLM-powered feature eventually has the same conversation: "What if we need to switch providers?" The standard answer — "we'll just swap the API key" — reveals a dangerous misunderstanding of where coupling actually lives. In practice, teams that attempt a provider migration discover that the API endpoint is the least of their problems. The real lock-in hides in seven distinct coupling points, each capable of turning a "quick swap" into a quarter-long project.

Migration expenses routinely consume 20–50% of original development time. Enterprise teams who treat model switching as plug-and-play grapple with broken outputs, ballooning token costs, and shifts in reasoning quality that take weeks to diagnose. Understanding where these coupling points are — before you need to migrate — is the difference between a controlled transition and an emergency scramble.