The Build-vs-Buy Line Moved: Deciding AI Features When Provider Primitives Absorb Your Infra

Eighteen months ago, "we built our own retrieval pipeline" was a reasonable thing to say in an architecture review. You had a chunking strategy, an embedding model you'd benchmarked, a vector store you'd tuned, a re-ranker, and a context-packing heuristic that took three engineers a quarter to get right. That stack was a real piece of differentiated infrastructure. Today, the same capability is a single hosted tool call: upload files to a vector store, attach it to a request, and the provider parses, chunks, embeds, stores, retrieves, and re-ranks — all behind one API. The three-engineer-quarter is now a config object.

This is the uncomfortable pattern of building AI products right now. The line between what you build and what you buy is not fixed. It moves — and it moves in one direction. Every few months a provider ships a primitive that absorbs a layer you used to own: memory, retrieval, structured output, tool routing, even multi-step orchestration. The infra you were proud of last quarter becomes the thing a competitor gets for free this quarter, with better defaults and a lower latency floor because it runs inside the provider's own datacenter.

The instinct is to treat this as a threat to defend against. That's the wrong frame. The provider absorbing commodity infra is doing you a favor — it deletes maintenance you never wanted. The real question is whether the thing you chose to build sits above or below the rising waterline. Most teams never make that call explicitly. They build whatever the demo needed, and then discover a year later that 70% of their codebase is reimplementing a primitive the provider now offers, while the 30% that's actually defensible got starved of attention.

The waterline keeps rising, and it only goes up

It helps to picture the AI stack as layers stacked on water, where the water is "what the provider gives you for free." The raw model floats at the bottom. Above it sit retrieval, memory, structured output, function calling, agentic loops, evals, and vertical workflows. The waterline is the height the provider has absorbed. Every release raises it.

The trajectory is unambiguous if you watch the API surface. Structured output went from "parse the JSON and pray, then retry on malformed output" to a guaranteed-schema response mode. Function calling went from a clever prompting trick you hand-rolled to a first-class request parameter. Retrieval went from a pipeline you assembled out of five vendors to a hosted file-search tool. Memory — persistent, scoped, cross-session state — is the layer being absorbed right now, with providers and a crowd of frameworks racing to own conversation history and long-term user facts so you don't store them yourself.

Two things follow from "the waterline only rises." First, anything you build at the current waterline has a short shelf life — you're doing unpaid R&D for a feature the provider will ship and then maintain better than you. Second, the waterline rising is not symmetric across the stack. It rises fastest through the horizontal primitives — the capabilities every AI app needs identically — and slowest through vertical depth, the stuff specific to your domain, your data, and your customers' workflows. The provider has no incentive to build your industry's compliance logic. They have every incentive to build retrieval, because everyone needs it.

So the strategic move is not to build below the waterline and lose, nor to guess where it'll be next year. It's to build in the direction the water can't reach.

Sherlocking is the rule, not the risk

There's a name for an incumbent absorbing a feature a smaller company built on top of it: sherlocking, after Apple repeatedly shipping OS features that erased third-party tools. f.lux got sherlocked by Night Shift. Tile got sherlocked by AirTag. Pebble got sherlocked by the Apple Watch. The pattern is old; what's new is the speed. An OS update shipped yearly. A model provider ships primitives monthly, and the "OS update that kills your product" can now be a paragraph in a changelog.

The examples are no longer hypothetical. Providers have shipped in-chat app builders that compete directly with the prompt-to-app startups deploying their models. They've launched vertical offerings — legal, coding, customer support — that put the startups building in those exact verticals on the defensive, including companies whose entire product runs on the provider's model at its core. When your supplier is also willing to become your competitor, "build on the platform" carries a risk that has nothing to do with your code quality.

The lesson founders who survived Apple's sherlocking actually teach is not "don't build on platforms." Dropbox, Spotify, and 1Password all built on platforms that later competed with them, and all survived — by moving up into workflow, network effects, and enterprise integration that the platform couldn't replicate without becoming a different company. The defensible position was never the feature. It was the depth around the feature: the operational integration into approvals, compliance, procurement, and analytics pipelines; the high-touch implementation; the accumulated domain data and feedback loop. None of that is a primitive. All of it is build, not buy — but it's a different kind of build than reimplementing retrieval.

A decision framework: build below the waterline only to escape it

Here's the rule I'd apply when deciding whether a given capability is build or buy. For each capability, ask two questions and you get four quadrants.

Question one: Is this horizontal or vertical? Horizontal means every AI product needs it in roughly the same shape — retrieval, memory, structured output, tool calling, basic orchestration, observability plumbing. Vertical means it's specific to your domain, your proprietary data, your customers' workflow, or your trust and distribution.

Question two: Is it on the provider's roadmap, or against their incentives? A primitive that serves all their customers is on the roadmap whether or not they've announced it. A capability that only your niche needs is something they will never prioritize, because the addressable market inside their API is too small.

That gives you the playbook:

• Horizontal + on-roadmap → buy, and design for replacement. This is retrieval, structured output, function calling. Use the provider primitive. Don't build it. If a primitive doesn't exist yet but obviously should, build the thinnest possible version behind an interface you can swap, and treat your version as a temporary bridge, not an asset. The day the provider ships it, you delete your code and feel relieved.
• Horizontal + against-incentives → build carefully, expect competition. Cross-provider routing, vendor-neutral evals, and portability layers fall here. Providers won't build the thing that helps you leave them. These are defensible-ish, but commoditizing fast through open frameworks, so don't over-invest.
• Vertical + on-roadmap → this is the sherlocking zone; move up or out. If the provider is heading into your vertical, your feature parity is a losing race. Survive by going deeper than they will — operational integration, services, and proprietary data — or pick a narrower wedge they won't bother with.
• Vertical + against-incentives → build with everything you have. This is the only quadrant where building is unambiguously the right call. Your proprietary data, your domain logic, your workflow integration, your customer trust, your feedback loop. The model is a necessary input; this is the moat. Spend your scarce engineering here.

The framework is uncomfortable on purpose, because it tells most teams that the majority of what they're building belongs in the "buy and design for replacement" bucket — and that the small slice they've been neglecting is the only part worth their best engineers.

The migration tax is real, so build for the swap

The objection to "just buy the primitive" is migration cost, and it's legitimate. The day you replace your hand-rolled retrieval with the provider's hosted tool, you inherit their chunking defaults, their metadata-filter semantics, their latency profile, and their pricing model — and you lose the knobs you tuned. If you built directly against your own pipeline's internals everywhere, the swap is a multi-month rewrite, and you'll rationally postpone it until your custom code is a liability.

The way out is to decide in advance that horizontal capabilities live behind a narrow interface. Your application code should ask for "retrieve relevant context for this query," not call your vector store directly. Then "swap our pipeline for the provider's tool" is one adapter implementation, not a refactor that touches a hundred files. The teams who report writing thin wrappers over provider APIs after fighting heavy frameworks in 2024 learned exactly this: own the seam, rent the implementation behind it. The seam is cheap to keep. The implementation behind it is the thing the provider will eventually give you, and you want to accept that gift without a rewrite.

This also changes how you read a provider's release notes. A new primitive is not a competitive event to fear — it's a maintenance burden you get to retire. The teams that suffer when a provider ships memory or retrieval are the ones who bet their differentiation on owning that exact layer. The teams that benefit are the ones who treated it as rented all along and put their effort one layer up.

Where this leaves you

Stop asking "should we build or buy this?" as a one-time decision per capability. The honest version is "how fast is the waterline rising under this capability, and is what we're building above it or below it?" Below the waterline, buy, and own only the seam so you can swap implementations as primitives ship. Above it — in your domain depth, your data, your workflow integration, your customers' trust — build with conviction, because that's the only ground the provider's roadmap can't flood.

The mistake that sinks AI products in 2026 is not picking the wrong vector store. It's spending a year building a beautiful version of a capability the provider was always going to commoditize, then having nothing left in the budget for the part of the product no one can absorb. Audit your codebase against the four quadrants. The lines of code you'd be relieved to delete when the provider ships a primitive are exactly the lines that shouldn't have been your differentiation in the first place. Find them now, get them behind a seam, and spend the reclaimed quarter on the 30% that's actually yours.