The Overfitting Org: When Your AI Team's Model Expertise Becomes a Liability
· 9 min read
Your best AI engineer can recite Claude's XML formatting preferences from memory. They know that Claude Opus refuses to generalize implicit instructions, that few-shot examples actually hurt performance on o1-series models, and that Azure OpenAI imposes an extra 8–12 seconds of latency versus the direct API in some regions. This expertise took months to accumulate. It also represents one of the most underappreciated risks in AI engineering today.
When a provider deprecates a model or silently shifts behavior, that knowledge doesn't transfer. It vanishes. And teams that built their systems — and their institutional competence — around a single model family often discover this the hard way.
