Page 111
12 articlesThe On-Call Runbook for AI Systems That Nobody Writes
Traditional SRE runbooks break down when the failure mode is probabilistic model behavior, not a crashed service. Here's what incident response actually looks like for LLM-powered systems, and the signals worth alarming on.
On-Device LLM Inference: When to Move AI Off the Cloud
A practical decision framework for when on-device LLM inference beats cloud APIs — covering privacy requirements, cost math, quality tradeoffs, and the deployment problems nobody warns you about.
Onboarding Engineers into AI-Generated Codebases Without Breaking How They Learn
AI coding tools ship features faster but silently erode the code-reading that builds system intuition in new engineers. Here's how to restore learning without slowing delivery.
The Pilot Graveyard: Why Enterprise AI Rollouts Fail After the Demo
88% of enterprise AI pilots never reach production. The problem isn't the model — it's everything that happens after the demo. A practitioner's breakdown of why compelling POCs die at 12% WAU and how to fix it.
Post-Training Alignment for Product Engineers: What RLHF, DPO, and RLAIF Actually Mean for You
RLHF, DPO, and RLAIF aren't just research acronyms — they determine whether the user feedback you're logging today becomes a training asset or stays noise. Here's what product engineers need to know.
The Pretraining Shadow: The Hidden Constraint Your Fine-Tuning Plan Ignores
Fine-tuning changes how a model talks, not what it fundamentally knows or believes. Here's what the research says about the ceiling practitioners keep hitting — and how to build around it.
Pricing AI Features: The Unit Economics Framework Engineering Teams Always Skip
Variable inference costs break fixed-price SaaS assumptions. A practical framework for per-workflow cost modeling, heavy-user subsidy math, and consumption cap design that preserves margin as usage scales.
Prompt Cache Break-Even: The Exact Math on When Provider-Side Prefix Caching Actually Pays Off
Prompt caching advertises a 90% discount on cache hits, but the write premium means low hit rates cost you more than no caching at all. Here's the exact math and the session architecture decisions that determine whether you capture the discount.
Prompt Canaries: The Deployment Primitive Your AI Team Is Missing
Code canary deployments catch crashes and latency regressions — but they're blind to the behavioral failures that actually hurt LLM systems. Here's the metric stack, deployment manifest pattern, and auto-rollback design that closes the gap.
Prompt Injection Detection at 100,000 Requests Per Day: Why Simple Defenses Break and What Actually Works
Static filters and LLM-as-judge approaches both fail at high throughput. Here's the layered classifier architecture that actually catches prompt injections under a 200ms latency budget.
The Prompt-Model Coupling Trap: Why Your Prompts Only Speak One Model's Dialect
Carefully tuned prompts silently accumulate dependencies on specific model behaviors — JSON formatting quirks, instruction hierarchy, refusal thresholds — that break on migration day. How to build a portability test harness and write lower-coupling prompts.
Property-Based Testing for LLM Outputs: Finding the Bugs Your Eval Set Never Imagined
Curated eval sets encode only the failure modes you imagined. Property-based testing generates thousands of adversarial input variants to find the bugs at domain boundaries your test suite structurally cannot reach.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
