Page 37
12 articlesThe Tail-Tolerant Retry Policy Your LLM Gateway Doesn't Have
Microservice retry defaults applied to an 8-second LLM call inflate P99, burn tokens during provider incidents, and hide a customer-visible latency cliff that the gateway dashboard never shows.
Why Token Forecasts Drift After Launch — and How to Catch the Spike Before Finance Does
Pre-launch cost models assume a synthetic traffic mix. Post-launch reality shifts the moment the feature actually works. The bill is the worst possible detector — here's how to catch the drift in real time.
Tool Schema Design Is Your Blast Radius: When Function Definitions Become Security Boundaries
The tool registry stopped being documentation the moment your agent could call it. Why every parameter type is a security control, and how to design schemas that survive prompt injection.
The Two Clocks Problem: When Your Model Provider's Cadence Breaks Your Roadmap
AI providers ship capability jumps every quarter while product roadmaps run on six- to twelve-month horizons. The mismatch turns roadmaps into museum pieces — here's a planning structure that keeps up.
Why The Weekly Transcript Review Beats Your AI Dashboard
A weekly hour of reading production transcripts catches prompt drift, untaxonomized intent, and lazy phrasing your dashboard averages into invisibility. Here is how to run the meeting, who attends, how to sample, and the privacy discipline that makes it sustainable.
The Abandon Primitive: Why Your Agent Loop Needs a First-Class Way to Quit a Plan
Most agent harnesses ship continue, return, and retry — but no first-class way to throw out a doomed plan. The missing primitive turns wasted budget into pivots.
Agent IAM Is Not Service IAM: Why OAuth Breaks When Intent Is Constructed at Runtime
OAuth and IAM were designed for callers with stable intent. An agent's intent is composed at runtime from prompts, retrieved documents, and tool outputs — the IAM layer never sees most of the inputs that decide what gets called.
Why Your AI Roadmap Shouldn't Have a 12-Month Plan
Frontier model capability turns over in 90 days. A 12-month feature roadmap commits you to obsolete bets. Replace it with a capability portfolio with explicit kill criteria.
The Air-Gapped LLM Blueprint: What Egress-Free Deployments Actually Need
Cloud AI stacks treat outbound HTTPS as a free primitive. Pulling the cable forces every layer — model provenance, evals, fleet, telemetry — to grow primitives the cloud version quietly hides.
The Brownout Pattern: When Your LLM Provider Is Slow but Not Down
Provider availability is continuous, not binary. Your fallback chain handles the easy outage and misses the brownout that quietly drains user trust for hours.
Clarification Budgets: When Your Agent Should Ask Instead of Guess
Most agents either over-ask and exhaust users or over-guess and lose their trust. The fix is a per-task clarification budget plus a policy layer the model is structurally unqualified to own.
Your Embedding Model Choice Sets the Ceiling Your LLM Can't Raise
The embedding model sets the upper bound on RAG quality, and swapping LLMs can't move it. A practical framework for choosing one: domain match, dimensionality, multilingual behavior, and instruction tuning.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
