Page 15
12 articlesThe AI Code Review Inversion: What to Focus on When the Author Is a Machine
When AI agents write the majority of your commits, line-by-line correctness review misses the bugs that matter. Here's the review discipline that actually works for machine-authored code.
AI Co-Pilot vs. AI Pilot: The Evidence-Based Product Decision Framework
The specific deployed-system signals — task completion rate, error recovery time, user override frequency, edge-case exposure — that determine whether AI should be advisory or autonomous, and why the wrong default costs you user trust that's hard to recover.
Why AI Quality Monitors Conflate Model Drift, Data Drift, and Prompt Drift — and What to Do About Each
When AI quality degrades in production, the root cause is one of three distinct problems — but conventional monitoring treats them all the same way and wastes weeks pointing at the wrong fix.
The AI Efficiency Paradox: When Your Best Feature Kills Your Revenue
AI features that make users more productive can compress per-seat revenue — a structural pricing problem that catches teams after the renewal cycle, not before. Here's how to think about it before you ship.
Story Points Don't Survive First Contact With an LLM
Why the assumptions behind velocity-based sprint planning collapse for AI features — and the milestone-based, eval-driven approach that keeps LLM engineering teams predictable.
AI Feature Dependency Graphs: Resilience Engineering When Your Services Share a Model
When fifteen product features share the same embedding model and LLM endpoint, one provider incident becomes a distributed systems outage with no stack trace. How to map AI feature dependencies, apply circuit breakers at each layer, and design degradation chains that fail features cleanly instead of corrupting outputs.
AI Feature PMF Signals: Why Your Metrics Are Lying to You
Conventional signals like NPS, thumbs-up ratings, and activation rates systematically mislead for AI features. Here's what genuine product-market fit actually looks like — and how to measure it.
Why Rolling Back an AI Feature Is Harder Than Rolling Back Code
A technical code rollback fixes the system, but it doesn't fix the users. Here's why AI behavior changes are sticky in ways code changes aren't, and the patterns that let you reclaim design space without breaking trust.
The AI Incident Postmortem Nobody Writes: A Four-Layer Diagnosis Framework
When an AI feature causes a production incident, standard postmortems fail. Here's a four-layer diagnosis framework — model, data, integration, infrastructure — that lets teams assign accountability without blame diffusion.
AI Output Volatility Is a Business Risk You're Probably Underpricing
Building pricing tiers, SLAs, and customer commitments on top of a probabilistic system is carrying undisclosed risk. Here's how to quantify it and hedge against it.
Your System Prompts Are Still in English: The Silent Cost of Incomplete AI Localization
Translating UI strings while keeping system prompts in English silently degrades non-English users. How the failure compounds across formality, structured outputs, tokenization, and invisible eval gaps — and what to do about it.
Your AI Feature's Quiet Quitters: How to Detect Silent User Distrust
Most AI feature failures are invisible in aggregate metrics. Users don't file tickets or disable features — they quietly route around them. Here's how to detect the behavioral signals that reveal silent trust abandonment before it shows up in your retention curve.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
