Your Provider's Safety Filter Is Product Policy You Can't Edit
A clinical decision support tool has exactly one job: help a doctor reason about symptoms, medications, and treatment plans in the same vocabulary the doctor would use in their own notes. That means the agent has to talk about overdose thresholds, drug interactions, contraindications, and dosing by name. None of that is optional. It is the product.
So it is a special kind of failure when the model interrupts a consult to say it cannot provide medical advice without professional oversight — to a professional, mid-consult, about a patient in front of them. The doctor didn't change. The patient didn't change. The prompt didn't change. What changed was a safety filter the team doesn't own, tuned against a population the team isn't part of, updated on a schedule the team doesn't control.
This is the vendor moderation seam, and it is one of the least-discussed boundaries in applied AI. Most teams treat the provider's safety layer as a fixed property of the model, like its context window. It isn't. It's a policy — a living one — and right now it's your product's policy whether you wrote it or not.
