Page 44
12 articlesWhen to Reach for an LLM vs. a Simple Heuristic: A Four-Factor Framework
A four-factor framework — signal quality, human performance ceiling, data availability, and reversibility — that helps engineering teams decide when AI genuinely creates leverage and when a simple rule-based system is the right tool.
Agent as User: Why Your Product Analytics Break When Bots Become Your Power Users
When AI agents become your heaviest product consumers, session funnels lie, engagement metrics invert, and NPS surveys measure nothing. Here's how to instrument for agent consumers and why your existing analytics dashboard is actively misleading you.
The Agent Portfolio Audit: How to Consolidate 15 Independent Agents Into a Platform Without Killing Team Autonomy
When independently-built AI agents outnumber your ability to govern them, you don't need more agents — you need an audit. Here's the consolidation playbook.
AI Writes Code in Seconds. Your Team Reviews It for Hours. The Math Isn't Working.
AI coding tools generate code 55% faster, but PR review time has climbed 91% in high-adoption teams. The real ROI calculation for AI coding tools depends on how you handle the verification overhead — and most teams aren't counting it.
The Ethics Review Gate Your AI Shipping Process Is Missing
Most engineering teams run security reviews before every AI feature ships — but no equivalent gate exists for fairness, bias, or accessibility risk. Here's the checklist, trigger conditions, and sprint integration that change that.
Why AI-Generated Terraform and Kubernetes Configs Are Silently Wrong
LLM-generated Terraform, Kubernetes manifests, and CDK pass syntax checks but carry hallucinated dependencies, outdated provider patterns, and security holes that only show up in production. Here's the failure taxonomy and what actually catches them.
Adding AI to Trusted Features: How Variance Destroys the Trust You Spent Years Building
Retrofitting AI into your most-used features isn't building on top of trust — it's borrowing against it. The failure modes, the asymmetric recovery curve, and a staged introduction framework for engineers who want to add AI without destroying what they've already earned.
The Automation Cliff Edge: When Partial AI Automation Is Worse Than None
Partial AI automation can produce worse outcomes than fully manual handling. Here's the engineering framework for identifying when you shouldn't automate unless you can automate the whole thing.
Consent Decay in Agentic Systems: When Your Authorization Becomes Ambient
When users authorize AI agents at setup time, those permissions become ambient authority exercised in contexts no one anticipated. Here's why static OAuth scopes fail long-lived agents — and what to do instead.
The Dual Newspaper Test for AI Features: Catching the Failure Modes Your Post-Mortems Miss
Most engineering teams audit AI features for technical failures while missing the non-technical failure modes that end up in ethics reporting. The dual newspaper test is a pre-ship framework that closes that gap.
Choosing Eval Metrics Is a Product Decision, Not a Technical One
Metric choice encodes which failure modes your team is willing to tolerate. Here's why engineering-driven metric selection systematically optimizes for the wrong thing — and how to fix it.
Golden Paths for AI Agents: How Platform Teams Can Enable Adoption Without Becoming a Bottleneck
Platform teams that centralize AI approval workflows become bottlenecks. The fix is golden paths — opinionated defaults that let product teams ship AI features autonomously while keeping governance in the infrastructure, not the approval queue.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
