Page 24
12 articlesThe Retry That Changed the Answer: Idempotency Keys for Nondeterministic LLM Calls
Retrying a timed-out LLM call does not re-fetch the same answer — it samples a new one. Here is why retry-on-timeout breaks against a nondeterministic backend, and how idempotency keys make it safe again.
The Streaming Response That Returns 200 Then Fails: How Mid-Stream Errors Break Your SLOs
A streaming endpoint commits to a 200 the moment the first token flushes, so every failure after that hides from your load balancer, retry middleware, and SLO dashboard. Here is how to make the body carry the verdict the header no longer can.
The AI Feature With Two Latencies: You Measure One, Your Users Feel the Other
Streaming AI features have two latencies that diverge — time-to-first-token and time-to-completion — and most teams instrument only the one users feel least. Here is how to split the metric and the SLO.
When 'Can the Agent Do X?' Becomes a Ship Commitment
AI capability probes quietly become roadmap commitments as 'it works once' travels through standup, roadmap, and a sales call. Here is the capability-test artifact and promotion gate that stop a demo from turning into a contract.
The Agent Debugger Has No Breakpoints: Why Trace-First Workflows Replace Step-Through
Step-through debugging breaks when inputs are stochastic. The replacement is trace-first and replay-based, with four affordances — timeline scrubbing, branch comparison, replay-with-perturbation, and per-step intent recovery — that look nothing like the IDE's debug toolbar.
The Agent That Refuses to Fail Loud: How Over-Eager Fallbacks Hide Production Regressions
Production agents accumulate retries, fallbacks, and repair passes that quietly mask quality regressions until eval-on-traffic drifts and the team can't trace why.
The AI Accessibility Audit Nobody Runs
Streaming AI interfaces routinely fail screen reader and keyboard users. Here is what the accessibility audit looks like, why it matters in 2026, and the fixes that take a half-day to ship.
The AI Feature Sunset Playbook Nobody Writes
Most AI feature sunsets stop at the endpoint and leave the prompt, judge, regression set, and incident memory behind. Here's the asset-by-asset playbook for retiring an AI feature without the orphaned configs, ghost eval runs, and lost institutional knowledge that show up two quarters later.
Build vs Buy for the AI Gateway: The Decision That Locks in Your Next 18 Months
Most teams pick build or buy for the AI gateway on instinct in week one, then regret it in month nine. A framework for the decision that actually matters at 18 months.
The AI Gateway Is the SPOF Nobody Named
The thin abstraction in front of your LLM providers became the load-bearing control plane for every AI feature you ship. A look at why its blast radius now exceeds any provider outage, and the SRE discipline that should follow.
The AI Indemnification Gap: When the Model Was Wrong and Nobody's Contract Covers You
Enterprise AI products sit in a three-link liability chain where each layer assumes someone else read the fine print. Here is how the indemnification gap forms, why the copyright shield doesn't cover hallucinations, and what discipline closes it before the first claim.
The AI Told Me So Defense: When Code Review Quietly Stops Pushing Back
Agent-authored PRs land with a 1.7x higher defect rate while reviewers fold to confident model prose. The structural fixes that keep senior engineers in the merge path before the incident graph bends.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
