27 posts tagged with "api-design"

Your AI agent is brilliant at reasoning, planning, and generating natural language. Then you point it at your enterprise SAP endpoint and it spends 4,000 tokens trying to understand a SOAP envelope. Welcome to the impedance mismatch — the quiet tax that turns every enterprise AI integration into a token bonfire.

The mismatch isn't just about XML versus JSON. It's a fundamental collision between how LLMs think — natural language, flat key-value structures, concise context — and how enterprise systems communicate: deeply nested schemas, implementation-specific naming, pagination cursors, and decades of accumulated protocol conventions. Unlike a human developer who reads WSDL documentation once and moves on, your agent re-parses that complexity on every single invocation.

In 2023, a team at Stanford and UC Berkeley ran a controlled experiment: they submitted the same prompt to GPT-4 in March and again in June. The task was elementary — identify whether a number is prime. In March, GPT-4 was right 84% of the time. By June, using the exact same API endpoint and the exact same model alias, accuracy had fallen to 51%. No changelog. No notice. No breaking change in the traditional sense.

That experiment crystallized a problem every team deploying LLMs in multi-service architectures eventually hits: model aliases are not stable contracts. When your downstream payment processor, recommendation engine, or compliance system depends on structured JSON from an LLM, you've created an implicit API contract — and implicit contracts break silently.

In 2024, automated bot traffic surpassed human traffic on the internet for the first time. Gartner projects that more than 30% of new API demand by 2026 will come from AI agents and LLM tools. And yet only 24% of organizations explicitly design APIs with AI clients in mind.

That gap is where production systems break. Not because the LLMs are bad, but because APIs built for human developers have assumptions baked in that silently fail when an autonomous agent is the caller. The agent can't ask for clarification, can't read a doc site, and can't decide on its own whether a 422 means "fix your request" or "try again in a few seconds."

This post is for the backend engineer who just found out their service is being called by an AI agent — or who is about to build one that will be.