Engineer systems that
think, act, and ship.
Practical guides on building autonomous AI systems, scaling engineering teams, and technical leadership.
Latest Dispatches
The A/B Testing Trap: Why Standard Experiment Design Fails for AI Features
Standard A/B testing breaks for LLM-powered features — non-deterministic outputs, heteroskedastic variance, and engagement metrics that miss semantic quality all conspire to produce false confidence. Here's what to do instead.
When Accuracy Becomes a Liability: How Users Build Workflows Around Your AI's Failure Modes
Improving your AI model's accuracy can break your most engaged users — because they've built load-bearing workarounds around your old failure modes. Here's the backwards-compatibility thinking AI teams need before shipping model updates.
Agent Blast Radius: Bounding Worst-Case Impact Before Your Agent Misfires in Production
A production AI agent that misfires doesn't just fail — it acts, at scope. The pre-deployment exercise most teams skip: modeling worst-case impact per tool, classifying actions by reversibility, and enforcing permission ceilings before the first incident teaches you where the limits should have been.
Agent Memory Contamination: How One Bad Tool Response Poisons a Whole Session
A single factually wrong or adversarially crafted tool response can corrupt an LLM agent's reasoning for an entire session. Here's the failure anatomy and the defenses that actually work.
Agentic Systems Are Distributed Systems: Apply Microservices Lessons Before You Learn Them the Hard Way
The failure modes plaguing multi-agent AI systems today are distributed systems problems from 2015 in disguise. Teams that internalized microservices lessons before building agents are shipping more reliable systems.
Why AI Engineering Training Programs Are Perpetually Behind the Models
AI engineering training programs are structurally doomed to lag 12–18 months behind current tools. The first-principles curriculum that survives model generations — and what seniority really means when tools expire faster than they're mastered.
AI Feature Payback: The ROI Model Your Finance Team Won't Fight You On
Traditional ROI spreadsheets break when applied to AI features. Here's a cost decomposition and payback model that engineering and finance teams can both use.
The Compliance Attestation Gap Nobody Talks About in AI-Assisted Development
SOC 2, HIPAA, and PCI-DSS all assume the person who approved your code understood it. AI-generated code breaks that assumption — and auditors are starting to notice.
AI Model APIs Are Software Dependencies You Can't See, Pin, or Track
Foundation model APIs change behavior without semver, never appear in your lockfiles, and aren't tracked by SBOM tools — here's the discipline that prevents the resulting production failures.
AI-Native API Design: Building Backends That Agents Can Actually Use
REST APIs were designed for human-authored clients. AI agents break them in entirely predictable ways — hallucinating endpoint names, retrying without idempotency, ignoring sparse error messages. Here's how to build backends that agents can call reliably.
AI-Native Logging: Capture Decisions, Not Just I/O
Conventional logs tell you what your LLM system did. AI-native logging tells you why — capturing the decision logic, rejected alternatives, and confidence signals that explain production failures.
The AI Onboarding Gap: Why Engineers Can't Learn What They Can't Test
New engineers can't bisect LLM regressions, can't read the implicit constraints baked into prompts, and can't test their way to confidence. Here's the scaffolding that makes AI systems legible to people who didn't build them.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
