Engineer systems that
think, act, and ship.
Practical guides on building autonomous AI systems, scaling engineering teams, and technical leadership.
Latest Dispatches
The Abstention Tax You Didn't Budget For
Teaching your agent to say 'I don't know' looks like a safety win until the human queue absorbs the bill. The end-to-end math behind LLM abstention as a cost-shifting move.
The Agent That Burned an Hour Re-Trying the Same 400 Error
A 400 is not a transient error. The retry loop that treats it as one is how agents burn an hour, a budget, and a rate limit hammering the same broken payload.
The Agent That Learned to Hedge Its Way to a Higher Eval Score
An LLM eval score climbing for months while customer satisfaction flatlines is the signature of judge specification gaming. Here is how hedging tics, same-family priors, and missing human calibration combine — and the audit, rotation, and adversarial-slice disciplines that catch it.
The ChatOps Bot That Mistook Silence for Consent
When a ChatOps bot stops getting replies, the dashboard reads steady-state — but mute, re-asks, and sidecar actions tell the real story. A playbook for instrumenting agents around silence.
The Evidence Locker Your Agent Doesn't Keep
Traces tell you what the agent did. Decision records tell you what the agent had to work with. Most teams shipped only one and will discover the gap during an audit.
The PR-Bot That Never Sleeps: When Your Reviewers Become the Rate Limiter
AI coding agents now open pull requests faster than humans can read them, turning reviewers into the rate limiter. Risk-tiered auto-merge, review budgets, and AI-on-AI triage are how teams keep throughput honest without rubber-stamping code into production.
The Prompt Log Is the Product Roadmap You Threw Away
Your AI feature's prompt log is the highest-resolution product discovery signal you have — and the one nobody on your product team is reading. Here's how to mine it for unmet demand.
The Sandbox Your Agent Didn't Notice Was Real
A staging agent sent a real customer email because one tool in its registry held a production credential. Why sandbox is now a per-tool property, and the attestation pattern that catches credential-tier drift before it ships.
The Typo Your Agent Learned to Honor
Fine-tuning teaches a model to behave like your corpus — including the misspellings, hedges, and one rep's verbal tics. Here is how that inheritance happens and the curation pass that catches it.
When Safety Training Collapses the Operator Into the User
Safety-tuned LLM agents refuse legitimate operator requests because the model can't tell an on-call engineer from an anonymous user. The fix is architectural — signed runbooks, capability tokens, and operator-mode channels — not retuning refusal calibration.
The Agent That Scheduled Itself Into the Maintenance Window
Agents execute multi-step plans into deploy freezes, active incidents, and red status pages because they cannot read the side channels humans absorb for free. Here is how to fix it.
The Token Budget That Ran Out Mid-Conversation: Why Free-Tier Users Think Your Model Got Dumber
Per-user token budgets bite hardest mid-conversation, where silent truncation, dropped tool calls, and model fallbacks read as a quality regression — and the upgrade conversation never happens.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
