Page 20
12 articlesYour Fallback Path Is the Only Untested Code in Production
The degraded path you built to survive an outage runs almost never, so it rots quietly and makes its debut during the exact incident it was designed to survive.
The Happy Path Is the Only Path Your Agent Eval Ever Tested
A 94% pass rate measures how good you are at imagining success, not whether the agent works. How golden-path bias creeps into agent eval suites and how to fix it with failure-mode coverage, harvested production cases, and fault injection.
The Idempotency Key Your Agent Never Sent
When an agent's tool call times out, it retries — and without an idempotency key, that retry charges the card again. How to make agent retries harmless instead of dangerous.
The Incident Ticket With No Repro Steps: Reproducibility as Something You Engineer
An agent deleted the wrong record and the postmortem has a hole where the cause should be. Reproducibility for AI agents is not inherited from your stack — it is something you capture, version, and replay on purpose.
Your Internal API Became a Public API the Day an Agent Called It
An internal API stays internal only as long as you can name every caller. Wire an LLM agent to it and the contract you never wrote down becomes a liability — here's the public-API discipline you suddenly owe it.
The LLM Judge Is a Versioned Dependency, Not Neutral Infrastructure
Grade model outputs with an LLM judge and the judge is a model with its own behavior. The day it changes, every historical score becomes a foreign currency — and most teams never notice.
When LLMs Review LLMs, Errors Get Laundered Not Caught
A closed loop where one model reviews another and feeds the next eval has no ground truth anywhere — errors get laundered into high scores. Here is where to put the human back.
The Model Reached End of Life and Took Your Prompt With It
A model deprecation notice reads like a one-line config change, but the prompt you tuned for six months was fitted to one model's quirks and does not survive the swap. Treat model end-of-life as a recurring migration project with a re-runnable eval set.
Onboarding an Agent Like a Junior Engineer Is a Category Error
A junior engineer accumulates context every week; an agent accumulates nothing. Why the new-hire metaphor misallocates your attention, and where to put the learning instead.
The Permission Prompt Is a UX Bug: When Human-in-the-Loop Becomes Human-as-Rubber-Stamp
Agent products gate dangerous actions behind approval dialogs and call it oversight, but by the fortieth prompt the human clicks approve on reflex. Why prompt volume is the real safety bug, and how to fix it.
Prompt Caching's Hidden Tax: When a Cache Hit Serves the Wrong User's Context
A prompt cache key is a correctness boundary, not a billing knob. Draw it for hit rate and you invite cross-tenant context bleed and stale personalization.
Prompt Injection Is a Confused Deputy, Not a Content-Filtering Problem
Treating prompt injection as a content-filtering problem is a losing arms race. The real vulnerability is a confused deputy: an agent acting on untrusted instructions with borrowed authority. Scope the capability instead.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
