34 posts tagged with "api-design"

When you deprecate an API for human developers, you have a whole ceremony for it. You bump the version, add deprecated: true to the OpenAPI spec, ship a Sunset HTTP header, send an email to the developer mailing list, post to the changelog, and give people six months to migrate. The signal reaches a human who reads it, files a ticket, and updates their client before the old path disappears.

Now point that same deprecation at an agent. The model calling your tool does not read your changelog. It does not subscribe to your mailing list. It never sees the Sunset header unless you deliberately put it somewhere the model looks, and even then it has no reliable habit of acting on it. The deprecation notice you so carefully authored lands in a mailbox with no reader. The agent keeps calling the old shape of the tool until the shape is gone, and then it fails — often silently, often in production, often at 2 a.m.

This is the quiet asymmetry of building tools for agents instead of people. Every discipline we built over two decades of API evolution assumes a human sits between the deprecation and the migration. Take the human out, and the entire mechanism goes dark.

"Be conservative in what you send, be liberal in what you accept." Postel's law is arguably the most successful design principle in the history of networking — it's how TCP implementations from different vendors managed to interoperate in the 1980s, and it shaped four decades of protocol and API design. It is also, in the judgment of the IETF itself, a principle that curdles over time: RFC 9413 grew out of a draft bluntly titled "The Harmful Consequences of the Robustness Principle," arguing that liberal acceptance helps interoperability in the short term while quietly rotting the ecosystem in the long term.

Agent systems compress that long-term rot into weeks. When the "sender" is a language model emitting tool calls, every act of liberal acceptance — casting "5" to 5, dropping an unknown field, fuzzy-matching an enum typo — destroys the exact signal you need to keep the system healthy. The tool boundary is the one place in an agent architecture where failing loudly is the reliability feature, and most teams get it backwards.

You ran the deprecation playbook flawlessly. Announcement email six months out. A migration guide with code samples in four languages. Sunset headers on every v1 response. A banner in the developer dashboard. Two reminder emails. Then you turned off v1 — and instead of the quiet cutover the playbook promised, your error rate went vertical and stayed there. Traffic didn't migrate. It just kept arriving, failing, and retrying, at higher volume than before, because every failure triggered another attempt.

The playbook didn't fail because you executed it badly. It failed because it assumes a human is on the other end — someone who reads email, skims changelogs, and files a Jira ticket to migrate before the deadline. A growing share of your traffic has no such person behind it. It comes from agents whose knowledge of your API is frozen in system prompts, tool schemas, and model training data scattered across thousands of repositories you can't see and will never reach. Your sunset notice was written for a reader. Your biggest client is a prompt.

GraphQL's core bet was that clients should compose their own data requirements. For a decade, that bet mostly lost — because the clients were human teams who didn't want to compose anything. Frontend engineers wanted a stable endpoint they could call and forget. The flexibility GraphQL sold was a tax they paid in resolver complexity, caching workarounds, and security review, in exchange for a benefit — per-request field selection — that a known, slow-changing web app barely needed. By 2024, the practitioner consensus had visibly cooled: most internal APIs serve two or three known clients, and a well-shaped REST endpoint or a BFF layer covers them fine.

Then a new kind of client showed up. An AI agent doesn't have a fixed set of screens. It decides, per task, what data it needs, and it pays for every byte of the response — literally, in tokens, and cognitively, in degraded reasoning as the context window fills with fields nobody asked for. The client that actually composes its own data requirements finally exists. It just isn't human.

In 1980, Jon Postel wrote a sentence into the TCP specification that became a founding principle of the internet: "be conservative in what you do, be liberal in what you accept from others." For forty years, engineers have applied it everywhere — parsers that tolerate trailing commas, APIs that coerce "10" into 10, HTML renderers that quietly repair broken markup. The web arguably exists because browsers forgave everyone's mistakes.

Then the caller stopped being a human, and the advice inverted. When an AI agent calls your tool with a stringly-typed number, a mis-nested JSON object, or an enum value that's almost right, the tolerant parser that saves a human developer twenty minutes of debugging does something much worse to the agent: it confirms that the sloppy call was correct. The agent's only training signal inside a loop is the feedback your tool returns. Accept garbage, and you are teaching the model — right now, in this episode — that garbage works.

Count the readers of your stack traces. For most internal tools, the answer used to be "one tired engineer, occasionally." Today the highest-volume reader of your error output is almost certainly a language model inside a retry loop. Coding agents read your linter warnings, your CLI usage strings, your API error bodies, and your test failures thousands of times a day — far more often than any human ever will. And unlike the human, the agent takes every word literally.

That changes what an error message is. It is no longer documentation of a failure. It is an instruction injected into the context window of the next attempt — a prompt you wrote months ago, now steering fleets of agents you've never met. A precise error converges the loop in one retry. A vague or misleading one sends the agent spiraling: wrong fixes, --no-verify workarounds, hallucinated flags, burned tokens. If you maintain a tool, a service, or a build system, you are already doing prompt engineering. You're just doing it in your error strings, and probably by accident.

Your platform team measures success by developer adoption. Monthly active engineers on the internal API. Time-to-first-call for a new service. Net promoter score from the quarterly DX survey. Every one of those metrics assumes a human is on the other end of the request — someone who reads a getting-started guide, copies a curl example, and grumbles in Slack when the error message is unhelpful.

That assumption is quietly becoming false. The fastest-growing consumer of your internal APIs is not a person. It's an agent: a coding assistant resolving a ticket, a workflow that reconciles invoices overnight, a support bot that calls six services to answer one question. These callers don't skim your docs — they ingest your tool schemas into a context window. They don't file a bug when an error is cryptic — they silently retry, burn tokens, and give up. And they are about to outnumber the humans.

Your incident timeline shows no deploys. Your code did not change. Your traffic mix did not change. Your prompts did not change. And yet your p99 output length doubled inside a week, your downstream rendering layer started clipping responses, and your output-token bill rose 38% on traffic that wasn't asking for longer answers. The change was real, the regression was measurable, and nothing in your version control system records it — because the value that moved was one your code never sent.

The provider raised an implicit default. The release notes filed it under "improved long-form behavior." The parameter in question was max_tokens, which your application has been omitting since day one because the documented default was generous and your outputs rarely came close. The default moved from 4096 to 8192 to accommodate longer reasoning in the provider's newer models. Your application got the new default whether you wanted it or not, because the absence of a parameter is itself a configuration choice — and the provider owns the right to change the value behind it.

This is the failure mode where a "no-op" release on the provider's side propagates through your system as a behavior change, a cost change, and a UX change all at once, and your team's only diagnostic signal is the bill arriving at the end of the month.

The deprecation notice went out on a Tuesday. The downstream team rotated the response shape on their search tool — results[].snippet became results[].excerpt, a clean rename, six-week window, banner in the docs, three reminder emails to the engineering list. Every human consumer migrated. The agent did not, because the agent does not read email. For fourteen days the retry loop quietly parsed the new payload, found the field it was looking for missing, raised a KeyError, and counted that as a retryable failure. The retry hit the same endpoint, got the same new shape, raised the same error, gave up after three attempts, and returned an apologetic message to the user. The retry budget dashboard stayed green the entire time — retries were never exhausted, they were just permanently failing within budget. Success rate, measured at the tool layer, sat at zero on that path. Nobody looked because there was no page.

This is the shape of the failure that gets the most engineers in 2026: not the dramatic outage, but the silent contract drift where a human-facing migration runs to completion and the agent-facing one never starts because nobody knew there was one to run. The deprecation worked exactly as designed for the consumers it was designed for. The agent was a consumer nobody listed.

Your handler did everything right. The HTTP status was 200. The body parsed. The text field had characters in it. You incremented responses_succeeded, appended the message to the conversation, returned the JSON down to the client, and moved on. The user got a sentence that ended mid-clause, a redacted answer dressed up as a normal one, or a polite refusal phrased as a completion. Your dashboard does not know any of that happened. The provider told you. You did not read the field.

Every major inference API returns a stop signal alongside the text: OpenAI calls it finish_reason, Anthropic calls it stop_reason, Gemini calls it finishReason. The field is small. It is one enum value per response. It is also the only out-of-band channel the model has for telling you whether the response you just shipped is the answer or a fragment of one. Treating it as cosmetic is the same shape of bug as ignoring HTTP status codes — except your monitoring caught the HTTP one a decade ago and has no opinion about this one.

The rate limit was never a fairness primitive. It was a sales-engineering quote that grew up — a number a solutions engineer typed into a docs page during onboarding three years ago, copied into a tier definition, and never revisited because no one ever hit it. The limit said "100 requests per minute" and it meant "more than any sane integration will ever need," because every integration on the platform was a backend service driven by a human at a keyboard, and humans do not type a hundred times a minute.

Then a paying tenant pointed an agent at the endpoint. The agent did not type. It did not pause to read responses. It did not have a UI to render between requests. It executed a planning loop that called the API once per reasoning step, and one reasoning step took the model about thirty milliseconds of wall time to formulate. The agent hit the per-minute ceiling in three seconds, the per-hour ceiling in three minutes, and the daily quota before the on-call engineer's coffee had cooled. The support escalation landed before the throttle dashboard had updated.

A search tool returns ten document IDs. An asset tool returns an S3 presigned URL. A database tool returns a row handle. A file tool returns a path. Each of those returns is, formally, a pointer — a small string that names a value the agent does not yet possess. The model's downstream behavior depends entirely on whether it knows that and dereferences before reasoning, or whether it treats the pointer as if it were already the thing.

The failure mode is invisible from the trace. The tool call succeeded. The return is well-formed. The model emitted plausible-looking output. Nothing in the log says "the agent reasoned about a filename and called it a document." The pointer-vs-value confusion sits underneath the visible behavior, in a layer your tool schema never named.