Page 7
12 articlesToken Accounting Drift: When Your Trace Logs Don't Match the Provider Invoice
Every team building on a hosted LLM eventually finds the token counts in their traces don't match the monthly invoice. The gap is rarely fraud — it's a structural measurement problem with six compounding causes.
Two-Hop Tool Chains: Why 95% Tools Compose Into 80% Pipelines
Per-tool dashboards stay green while end-to-end agent reliability collapses. The failure lives at the seams between tools, where contract drift, pagination, and unit mismatches turn 95% primitives into 80% pipelines.
User Trust Half-Life: Why One Bad Session Erases Weeks of Calibration
AI users spend weeks building trust calibration and lose it in one bad session. Build telemetry for verification, undo, and engagement-without-action to catch trust erosion before churn shows up.
The Vendor SLA Gap: Why Your LLM Provider's Uptime Misses the Failure Mode That Breaks Your Product
Your LLM vendor's 99.95% uptime number does not cover refusal spikes, silent model bumps, or quota-driven degradation. Here is the functional-availability instrumentation that does.
Your Fine-Tuning Corpus Is a Codebase. Stop Shipping It Through a Bucket.
Application code has PR review, signed commits, and named authors. Fine-tuning corpora have an S3 bucket and a Mechanical Turk batch from 2024. The threat models are inverted, and 250 documents can backdoor a 13B model.
Escalation Rate Is the Eval Signal Your Offline Tests Missed
Escalation rate is one of the few honest signals of agent capability, but in most companies it lives on the ops team's staffing dashboard — not the AI team's eval review. Here's how to close that gap.
On Intelligence, Chapter by Chapter: A 2004 Book That Predicted Half of Modern AI
A chapter-by-chapter walk through Jeff Hawkins' On Intelligence — what it got right about prediction-as-cognition, what it missed about scaling, and why it still frames how I read transformer behavior in 2026.
The Platform-Readiness Gap: When AI Features Ship Before the Infra to Operate Them
AI features ship before the platform exists to operate them, and the debts compound. A launch gate, named owners, and a deliberate platform ramp are the only way out.
Agent Branch Coverage: Your Eval Hits the Happy Path, Not the Planner's If-Else
Agent prompts hide if-else branches your eval suite never executes. Borrow MC/DC discipline, instrument planner decisions with branch IDs, and gate prompt diffs on coverage so silent re-routings stop slipping into production.
Agent Memory Eviction: Why LRU Survives a Model Upgrade and Salience Doesn't
Salience-weighted memory eviction looks like a quality win on day one and turns into a migration project at every model upgrade — here is why LRU is the boring choice that survives.
The Agent Scratch Directory: The Unowned Filesystem PII Surface Nobody Inventoried
Agent workers accumulate ephemeral filesystem state — extracted PDFs, transcribed audio, cached attachments — on disks no one classified. The fix is naming the tier, not architectural sophistication.
The AI Engineering Perf Packet: Making Stochastic Work Legible at Promotion Review
AI engineering work doesn't fit deterministic packet templates. How to frame ramp-shipped, eval-driven, stochastic work so calibration committees can actually credit it.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
