10 posts tagged with "documentation"

Your team rejected the microservices split in 2024. There was a two-hour meeting, a heated Slack thread, and a spreadsheet comparing operational costs. The conclusion was firm: the monolith stays until the platform team ships multi-region deploys. Everyone who was in the room still remembers.

Your coding agent was not in the room. Last Tuesday it proposed the split again — confidently, with a clean migration plan and a well-argued design doc. The Tuesday before that, a different agent session proposed extracting the billing service. Next week, a third one will suggest replacing your bespoke job queue with the message broker you evaluated and rejected twice. None of them are wrong on the merits as they can see them. They just can't see the merits, because the reasoning that settled these questions lives in a Slack thread that expired, a meeting that wasn't recorded, and the heads of two engineers, one of whom left.

This is the quiet failure mode of agent-assisted engineering: settled questions get re-litigated at machine speed. And the fix is a practice most teams filed under bureaucratic nice-to-haves a decade ago — the architecture decision record.

Somewhere, right now, an AI assistant is explaining your product to a prospect. It is quoting a price you changed eighteen months ago, recommending an integration you sunset last quarter, and suggesting an API endpoint that returns 410 Gone. You will never see this conversation. No analytics event fires. No session recording exists. The prospect either believes the wrong answer and files a confused support ticket, or believes the wrong answer and quietly buys from the competitor the model mentioned in the same breath.

This is not a hypothetical future problem. AI referrals already account for meaningful traffic — up to 5–8% for some technology and e-commerce sites — and the answers behind those referrals are generated from whatever the model absorbed about you, whenever it absorbed it. Your marketing team spent a decade learning to monitor brand search, review sites, and social mentions. Almost nobody is monitoring the surface that is growing fastest: what the models say when someone asks about you.

For two decades, documentation was where good intentions went to die. You wrote the README during the first sprint, when the architecture was clean and your enthusiasm was high. Nobody read it. By the third sprint it was lying about the build command, and by the sixth it described a service that had been deleted. Documentation was a tax everyone agreed to pay and nobody actually paid — a moral imperative with no feedback loop. Write bad docs and nothing happened. Write no docs and nothing happened either, because the senior engineers carried the architecture in their heads.

Then we pointed coding agents at our repositories, and the feedback loop arrived overnight. The README is now the single highest-leverage file you own — not because anyone gave a motivational talk about documentation hygiene, but because the quality of that file now visibly determines whether your agent ships correct code or confidently hallucinates an architecture that no longer exists.

This is the documentation renaissance, and it has almost nothing to do with the documentation we used to write.

A product manager pings you in Slack asking what happens when a customer asks the assistant to cancel their subscription. You start typing the answer from memory, then second-guess yourself, then open the system prompt and read it for thirty seconds. You paste back a summary. They thank you and move on. Three hours later, support asks the same question. By Thursday, the head of partnerships pastes a screenshot of the prompt into a deal review.

This is the prompt-as-documentation anti-pattern, and the first time you notice it happening, it feels great. The artifact you spent six weeks tuning is now the canonical source of truth for what the product does. PMs are reading it. Support is reading it. Sales is reading it. Somewhere a designer is reading it. Your work is load-bearing in a way the old service-layer code never was, and you can prove it by counting the number of unrelated people who can pull the file from memory.

Here is an uncomfortable finding from Google Research: when a RAG system retrieves insufficient or outdated context, the hallucination rate doesn't stay flat — it jumps from 10.2% to 66.1%. Adding a stale knowledge base doesn't make your AI help center neutral. It makes it sixfold more likely to give a confident wrong answer than if you had shipped nothing at all.

Most teams building AI-powered search and help centers focus on retrieval quality, embedding models, and chunk size. Almost none of them have a process for tracking whether the documents in the corpus are still accurate. That gap — documentation debt — is now showing up as a production reliability problem, not just a content problem.

Your AI feature shipped cleanly. The documentation looked good: input schema, expected outputs, a worked example. Three months later, a model update arrives silently. The outputs shift. Your docs are wrong but nobody knows it yet — because they still look right.

This is the core of AI documentation debt, and it compounds faster than any other kind of technical debt because the failure is invisible until a user finds it.

A team I worked with last quarter shipped an AI assistant with a tidy stack of supporting documents: an in-product tooltip warning that the AI may produce inaccurate results, a help-center article titled "How does the assistant work," an internal support runbook for handling escalations, and a public model card listing the underlying model, the tools the assistant could call, and the data domains it covered. The launch went well. Six months later the prompt had been edited fourteen times, the model had been swapped from one tier to another with subtly different refusal behavior, two new tools had been added, one tool had been deprecated but not removed from the prompt, and the language settings had been opened from English-only to nine locales.

Every single one of those documents was wrong. Not catastrophically wrong — the kind of wrong where a sentence is half-true, a capability is described in language the model no longer matches, a refusal pattern is documented that the new model never triggers, a tool name appears in the help article that the assistant won't actually call. The kind of wrong that produces a slow drip of confused support tickets, a few customer trust regressions when the AI does something the docs say it won't, and — because the company sells into a regulated vertical — a small but real compliance gap that nobody on the AI team had thought to track.

A senior engineer at a mid-sized SaaS company spent two days last quarter chasing a deployment bug that turned out to be the agent's fault. The agent had read a runbook last updated in 2023, faithfully followed step three, and ran a command that no longer existed in the deploy tooling. The runbook still rendered fine in the wiki — the screenshots were even still legible — but it had silently become hostile to a reader who couldn't tell that the surrounding context was stale. The human authors had no idea the doc was now a load-bearing input for every new hire's AI assistant.

This is the quiet shift that has happened in most engineering orgs over the past eighteen months: the internal wiki has accumulated a second audience. The same Confluence pages, the same architecture diagrams, the same "how we deploy" gists are now being read by two distinct consumers — the engineers themselves and the AI assistants their engineers use. The two readers consume the same words under entirely different constraints and produce systematically different failure modes when the docs were written with only the first one in mind.

There's a specific kind of dread that hits when you inherit an AI system from an engineer who just left. The system prompts are hundreds of lines long. There's a folder called evals/ with 340 test cases and no README. A comment in the code says # DO NOT CHANGE THIS — ask Chen and Chen is no longer reachable.

You don't know why the customer support bot is forbidden from discussing pricing on Tuesdays. You don't know which eval cases were written to catch a regression from six months ago versus which ones are just random examples. You don't know if the guardrail blocking certain product categories was a legal requirement, a compliance experiment, or something someone added because a VP saw one bad output.

The system still works. For now. But you can't safely change anything.

Your documentation says the /summarize endpoint returns a concise summary. That is true. It returns a different concise summary every time, sometimes misses a key point, occasionally returns structured JSON when you forgot to specify format in the prompt, and degrades silently after a model update you didn't know happened. None of this appears in the docs.

Traditional API documentation captures contracts: given input X, expect output Y. AI-powered features break that model at its foundation. There is no stable contract to document. The same prompt, same model, same parameters — different output. And yet teams ship these features with the same style of documentation they'd write for a database query: a function signature, a return type, maybe a sentence about error codes.

The gap between what your docs say and what your feature actually does is where developer trust goes to die.