Page 57
12 articles"Done!" Is Not a Return Code: Why Agent Completion Needs a Structured Signal
Agents signal completion in prose; orchestrators need structured events. A done-tool with a status enum, reason code, and resumable handle turns silent agent failures into loud schema violations your pipeline can actually route on.
Durable Agents: Why Async Queues Break for Long-Running AI Workflows
Multi-step AI agents fail in production because queues deliver at-least-once and LLMs plan non-deterministically. The fix is durable execution — sagas, idempotent checkpoints, and a stateful substrate around the stateless planner.
The Embedding API Hidden Tax: Why Vector Spend Quietly Eclipses Generation
Embedding API spend grows silently and crosses generation cost at scale. A breakdown of the workloads that dominate the bill, the architectural levers that bend the curve, and the breakeven math for self-hosting.
Embedding Model Rotation Is a Database Migration, Not a Deploy
Swapping an embedding model is not a config change — the new vectors live in a different manifold from the old ones, so it is a full re-embed plus a cutover disguised as a deploy. A migration playbook with shadow indexes, dual-read agreement metrics, staged traffic, and the operational tax teams forget to budget.
Your Eval Harness Runs Single-User. Your Agents Don't.
A sequential eval harness can't catch the bugs that detonate when many agents share infrastructure. Four failure modes, and the design moves that fix them.
Eval-Prod Drift: The Agent Under Test Isn't the Agent in Production
Why the agent your eval harness measures silently diverges from the agent your users actually talk to — and the fingerprint, canary suite, and ownership discipline that closes the gap.
Your Gold Labels Learned From Your Model: Eval-Set Contamination via Production Leakage
Eval sets silently memorize your model's biases when labels come from production feedback, human annotators who see drafts, and RLHF traces. A look at the provenance discipline that keeps the mirror from winning.
The 'We'll Add Evals Later' Trap: How Measurement Debt Compounds
Skipping evals ships faster for one quarter and slower for four. A look at how measurement debt compounds, the early warning signs, and the org-level forcing functions that prevent the drift.
Your Fine-Tuning Corpus Is a GDPR Data Artifact, Not Just an ML Asset
Fine-tuned weights encode customer PII that survives database deletion. A practical guide to treating training corpora as data artifacts under GDPR — lineage documentation, adapter isolation, and the compliance conversation to have before the first fine-tune ships.
First-Touch Tool Burn: Why Your Agent Reads Twelve Files Before Doing What You Asked
AI agents burn 60–80% of their token budget on reads before the first edit. Task-class routing, exploration budget caps, and plan-then-act gating cut the waste.
Free Tier Abuse Economics: When Your AI Generosity Gets Ratio'd by Bots
Free tier strategies built for SaaS quietly bankrupt AI products. Here's how bots monetize your generosity, and the rate limits, proof-of-work, and fingerprinting patterns that stop the bleeding.
GPU Starvation: How One Tenant's Reasoning Prompt Stalls Your Shared Inference Endpoint
One reasoning prompt can drag p99 latency for every other request on a shared inference endpoint. Here is why continuous batching and KV-cache pinning cause head-of-line blocking, the diagnostic signal nobody watches, and four mitigations — chunked prefill, priority scheduling, per-tenant token caps, and request-class isolation — ordered by how invasive they are.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
