The Build-vs-Buy Line Moved: Deciding AI Features When Provider Primitives Absorb Your Infra
Eighteen months ago, "we built our own retrieval pipeline" was a reasonable thing to say in an architecture review. You had a chunking strategy, an embedding model you'd benchmarked, a vector store you'd tuned, a re-ranker, and a context-packing heuristic that took three engineers a quarter to get right. That stack was a real piece of differentiated infrastructure. Today, the same capability is a single hosted tool call: upload files to a vector store, attach it to a request, and the provider parses, chunks, embeds, stores, retrieves, and re-ranks — all behind one API. The three-engineer-quarter is now a config object.
This is the uncomfortable pattern of building AI products right now. The line between what you build and what you buy is not fixed. It moves — and it moves in one direction. Every few months a provider ships a primitive that absorbs a layer you used to own: memory, retrieval, structured output, tool routing, even multi-step orchestration. The infra you were proud of last quarter becomes the thing a competitor gets for free this quarter, with better defaults and a lower latency floor because it runs inside the provider's own datacenter.
The instinct is to treat this as a threat to defend against. That's the wrong frame. The provider absorbing commodity infra is doing you a favor — it deletes maintenance you never wanted. The real question is whether the thing you chose to build sits above or below the rising waterline. Most teams never make that call explicitly. They build whatever the demo needed, and then discover a year later that 70% of their codebase is reimplementing a primitive the provider now offers, while the 30% that's actually defensible got starved of attention.
