Skip to main content

3 posts tagged with "vendor-risk"

View all tags

The Deprecation Date That Moved While It Sat in Your Backlog

· 9 min read
Tian Pan
Software Engineer

The deprecation notice arrived on a Tuesday with a sunset date six months out. Your platform team logged it in the dependency tracker with a "Q3 cutover" label and a yellow severity. It joined two other migrations already in the queue. Three weeks later, the provider amended the date inside the same URL, no diff, no inbox notification, just a quietly updated paragraph that pulled the sunset sixty days earlier into the middle of your code freeze.

The lifecycle page you treated as a planning document was always a contract clock. The only thing that changed is which team's calendar it controlled — and the team that owns it is not yours.

The Embedding Deprecation That Halved Your Retrieval Recall Without a Deploy

· 10 min read
Tian Pan
Software Engineer

The most expensive embedding bug a RAG system can ship is the one where nothing in your repository changes. Your retrieval code is the same. Your index is the same. Your query path is the same. And one Tuesday in week six, somebody notices that the answers used to be better.

The provider posted a sunset notice for the embedding family your index was built against twelve months ago. The platform team filed it in a deprecations dashboard with a year of runway and moved on. The sunset path wasn't a hard cutoff — it was a quiet quality regression where the deprecated endpoint started routing to a "compatibility" successor that returned vectors in the same dimensionality and a subtly different semantic geometry. Query embeddings began drifting against the corpus you embedded a year ago. Recall@10 on your standing eval slid by 47% over six weeks. The team only traced it back when an unrelated quality dashboard crossed a threshold, dragging a senior engineer into a root-cause exercise that ended at an embedding endpoint no one on the call had touched in a year.

Fourth-Party Risk: When Your Vendor's Vendor Owns Your Customer's Incident

· 11 min read
Tian Pan
Software Engineer

Your contract is with the model provider. Your runbook handles the case where that provider is degraded. Your status page subscription pages you when their dashboard turns yellow. You feel covered. Then one Wednesday afternoon the underlying cloud region your provider runs in starts brownouts, your provider's failover region is also affected because they consolidated capacity to control unit economics, and your product is half-down for ninety minutes because of a vendor decision two layers upstream from any contract you signed.

The customer postmortem request lands in your inbox the next morning. They want a root cause. The root cause lives in a layer your status page cannot see and your contract does not let you compel. That layer is what fourth-party risk actually is — not a procurement checkbox, but a silent dependency tier that propagates failures upward with attenuation but not absorption.