Page 23
12 articlesThe Agent That Wouldn't Stop: Scope Creep as a Runtime Failure Mode
Agents fix the bug, then keep going — refactoring nearby code, expanding scope, racking up tokens. A guide to designing stopping criteria, step budgets, and 'done' signals into agent tasks before scope creep ships as a silent failure mode.
The Confidence Score Your Users Learned to Ignore
A confidence percentage next to an LLM answer is a trust contract you usually cannot back. After a few miscalibrated 90%s, users learn to ignore the gauge — and three honest design options remain.
The Demo Worked Because You Were Watching: Session Length Is the Eval Dimension Your Suite Forgot
Five-turn demos hide the compounding error, attention drift, and sticky commitment that show up at turn twenty-eight. Treat session length as a first-class eval dimension or ship a reliability number that the user has already met the other version of.
The Embedding That Aged Out of Meaning
An embedded knowledge base silently rots as the world's vocabulary moves on. Recall dashboards miss it because they grade by yesterday's notion of similar.
The Filler Tool Call: When Agents Perform Diligence Instead of Doing Work
Production agents emit tool calls that change nothing — burning tokens, latency, and accuracy. Here is how filler emerges from training, what it actually costs, and how to engineer it out with counterfactual instrumentation and call budgets.
The First-Time User Cliff Your Aggregate Metrics Are Hiding
AI features often look healthy in aggregate while new users churn after one or two tries. Here is how the first-time user cliff hides in your metrics and the patterns that close it.
The Kill Switch Nobody Wired Because the Feature Never Failed
Launch flags get cleaned up; kill switches do not. Why every AI feature needs a durable runtime disable, a pre-decided fallback chain, and a runbook that names the lever.
The OOO Auto-Reply Your Agent Did Not Read
Most agent stacks page humans by name, not by role — so when somebody takes PTO, the agent loops with the auto-reply until on-call notices.
The Tool-Call Authorization Layer Nobody Wrote
Your gateway checks who. Your endpoint checks what. Nothing in your agent stack checks whether the model was allowed to make the call in the first place.
The Tool You Added For One Agent Is Now In Every Agent's Hand
A shared tool registry quietly hands every agent every capability. Per-agent allow-lists, not the catalog, are the unit of authorization that actually constrains what an agent can do.
What You Deleted Is Invisible to Your Coding Agent
Coding agents reintroduce code you deleted yesterday because absence leaves no trace in the repo. A field guide to recording the negative decisions agents need to respect.
The Nightly Batch Job That Quietly Became a Latency-Critical Service
A nightly batch job becomes a latency-critical service one reasonable request at a time. Why batch and online inference optimize for opposite goals, how the drift produces quiet failures, and how to re-architect on purpose.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
