Page 80
12 articlesConversation Branching as a First-Class Primitive: Why Linear Threads Force Users to Kill and Restart
Linear chat threads force users to kill-and-restart to explore alternatives. The copy-on-branch state model, DAG storage, and UI patterns that make divergence native instead of bolted-on.
Conversation History Is a Liability Your Prompt Never Admits
Chat history is not free context. Every turn adds noise, poisons attention, and bends the per-turn accuracy curve downward — here is how to detect, compact, and curate it.
Cost Per Feature, Not Cost Per Token: The Allocation Gap in AI Budgets
Token spend per endpoint hides which AI features make money. A tagging discipline that joins inference traces to product telemetry turns pricing, gating, and deprecation calls into decisions with numbers instead of vibes.
The Demo Loop Bias: How Your Dev Process Quietly Optimizes for Impressive Failures
Demos select for fluent, confident output — not correct output. Here is how the LLM dev loop quietly drifts toward charismatic failure, and the eval workflow that fixes it.
"Done!" Is Not a Return Code: Why Agent Completion Needs a Structured Signal
Agents signal completion in prose; orchestrators need structured events. A done-tool with a status enum, reason code, and resumable handle turns silent agent failures into loud schema violations your pipeline can actually route on.
Durable Agents: Why Async Queues Break for Long-Running AI Workflows
Multi-step AI agents fail in production because queues deliver at-least-once and LLMs plan non-deterministically. The fix is durable execution — sagas, idempotent checkpoints, and a stateful substrate around the stateless planner.
The Embedding API Hidden Tax: Why Vector Spend Quietly Eclipses Generation
Embedding API spend grows silently and crosses generation cost at scale. A breakdown of the workloads that dominate the bill, the architectural levers that bend the curve, and the breakeven math for self-hosting.
Embedding Model Rotation Is a Database Migration, Not a Deploy
Swapping an embedding model is not a config change — the new vectors live in a different manifold from the old ones, so it is a full re-embed plus a cutover disguised as a deploy. A migration playbook with shadow indexes, dual-read agreement metrics, staged traffic, and the operational tax teams forget to budget.
Your Eval Harness Runs Single-User. Your Agents Don't.
A sequential eval harness can't catch the bugs that detonate when many agents share infrastructure. Four failure modes, and the design moves that fix them.
Eval-Prod Drift: The Agent Under Test Isn't the Agent in Production
Why the agent your eval harness measures silently diverges from the agent your users actually talk to — and the fingerprint, canary suite, and ownership discipline that closes the gap.
Your Gold Labels Learned From Your Model: Eval-Set Contamination via Production Leakage
Eval sets silently memorize your model's biases when labels come from production feedback, human annotators who see drafts, and RLHF traces. A look at the provenance discipline that keeps the mirror from winning.
The 'We'll Add Evals Later' Trap: How Measurement Debt Compounds
Skipping evals ships faster for one quarter and slower for four. A look at how measurement debt compounds, the early warning signs, and the org-level forcing functions that prevent the drift.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
