Page 21
12 articlesThe Ethics Review Gate Your AI Shipping Process Is Missing
Most engineering teams run security reviews before every AI feature ships — but no equivalent gate exists for fairness, bias, or accessibility risk. Here's the checklist, trigger conditions, and sprint integration that change that.
Why AI-Generated Terraform and Kubernetes Configs Are Silently Wrong
LLM-generated Terraform, Kubernetes manifests, and CDK pass syntax checks but carry hallucinated dependencies, outdated provider patterns, and security holes that only show up in production. Here's the failure taxonomy and what actually catches them.
Adding AI to Trusted Features: How Variance Destroys the Trust You Spent Years Building
Retrofitting AI into your most-used features isn't building on top of trust — it's borrowing against it. The failure modes, the asymmetric recovery curve, and a staged introduction framework for engineers who want to add AI without destroying what they've already earned.
The Automation Cliff Edge: When Partial AI Automation Is Worse Than None
Partial AI automation can produce worse outcomes than fully manual handling. Here's the engineering framework for identifying when you shouldn't automate unless you can automate the whole thing.
Consent Decay in Agentic Systems: When Your Authorization Becomes Ambient
When users authorize AI agents at setup time, those permissions become ambient authority exercised in contexts no one anticipated. Here's why static OAuth scopes fail long-lived agents — and what to do instead.
The Dual Newspaper Test for AI Features: Catching the Failure Modes Your Post-Mortems Miss
Most engineering teams audit AI features for technical failures while missing the non-technical failure modes that end up in ethics reporting. The dual newspaper test is a pre-ship framework that closes that gap.
Choosing Eval Metrics Is a Product Decision, Not a Technical One
Metric choice encodes which failure modes your team is willing to tolerate. Here's why engineering-driven metric selection systematically optimizes for the wrong thing — and how to fix it.
Golden Paths for AI Agents: How Platform Teams Can Enable Adoption Without Becoming a Bottleneck
Platform teams that centralize AI approval workflows become bottlenecks. The fix is golden paths — opinionated defaults that let product teams ship AI features autonomously while keeping governance in the infrastructure, not the approval queue.
When AI Sounds Right but Isn't: LLM Confabulation in Technical and Scientific Domains
LLMs confabulate with extraordinary plausibility in physics, chemistry, and engineering — domains where 'sounds right' and 'is right' diverge most dangerously. Here's how to build grounding architectures that catch confident-but-wrong outputs before they cause real damage.
The Knowledge Half-Life Problem: Why Your RAG System Is Already Wrong
Semantic similarity has no concept of time — and that's why production RAG systems silently degrade. A practical guide to freshness classification, tiered reindex schedules, staleness detection, and treating your knowledge base like infrastructure.
Training Data Self-Poisoning: When Your AI Feature Corrupts Its Own Ground Truth
Deployed AI recommendation features shift user behavior in ways that corrupt the very data used to retrain them. Learn how to detect feedback loop contamination, maintain uncontaminated ground truth, and apply counterfactual evaluation before silent model collapse destroys your metrics.
The A/B Testing Trap: Why Standard Experiment Design Fails for AI Features
Standard A/B testing breaks for LLM-powered features — non-deterministic outputs, heteroskedastic variance, and engagement metrics that miss semantic quality all conspire to produce false confidence. Here's what to do instead.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
