Page 114
12 articlesAI-Assisted Incident Response: How LLMs Change the SRE Playbook Without Replacing It
AI copilots in on-call workflows can surface correlated signals and draft runbook actions—but they introduce failure modes traditional SREs aren't trained to catch. A practical guide to integrating LLMs into incident response without making outages worse.
The AI Capability Ratchet: How One Smart Feature Breaks Your Entire Product
Shipping one impressive AI feature permanently raises user expectations for every other feature in your product — including ones you haven't touched. Here's the mechanism, real examples, and how to manage the expectation debt before it hits your support queue.
The AI Dependency Footprint: When Every Feature Adds a New Infrastructure Owner
Every AI feature you ship introduces new infrastructure dependencies — vector databases, embedding models, eval frameworks, GPU serving layers. The problem isn't the dependencies themselves. It's that nobody owns them.
AI Feature Decommissioning Forensics: What Dead Features Teach That Successful Ones Cannot
The AI features your company quietly killed contain the failure patterns your next launch will hit. A forensic template, a leading-indicator catalog, and how to read the evidence dead features leave behind.
The AI Incident Severity Taxonomy: When Is a Hallucination a Sev-0?
Traditional severity classification breaks for probabilistic AI systems. A multidimensional framework for classifying AI incidents — beyond binary broken/working to capture scope, reversibility, and compounding damage.
AI On-Call Psychology: Rebuilding Operator Intuition for Non-Deterministic Alerts
On-call for AI systems breaks standard SRE intuition. A practical taxonomy, rotation design, and training curriculum for operating stochastic production systems without burning out the team or missing real regressions.
AI Product Metrics That Don't Lie: Behavioral Signals Over Thumbs-Up Scores
Aggregate satisfaction scores and thumbs-up rates hide the cases where AI is confidently wrong. Here's the behavioral signal stack that actually tells you whether your model improvement moved the needle.
The AI Reliability Floor: Why 80% Accurate Is Worse Than No AI at All
There is a reliability floor below which an AI feature actively destroys user trust faster than it can build value. Here is how to find it before shipping.
The AI Procurement Gap: Why Your Vendor Evaluation Process Can't Handle Probabilistic Systems
Traditional RFPs score features and uptime SLAs that mean nothing for stochastic outputs. The eval-driven assessment, contract clauses, and vendor transparency signals that procurement teams are missing for AI.
Stop Writing Prompts by Hand: Automated Optimization with DSPy and MIPRO
DSPy and its MIPRO optimizer replace manual prompt engineering with declarative signatures and Bayesian search — producing prompts that outperform hand-written ones by 20–40% on complex tasks. Here's how the system works and when it's worth the overhead.
Backpressure for LLM Pipelines: Queue Theory Applied to Token-Based Services
How to apply Little's Law, admission control, bulkheads, and token-bucket backpressure to LLM call graphs — and why naive retry logic turns transient provider blips into outages.
The Bias Audit You Keep Skipping: Engineering Demographic Fairness into Your LLM Pipeline
Safety filters and fairness checks are different problems requiring different engineering responses. Output quality disparities across gender, race, and language group won't surface in your guardrails — here's the methodology that catches them before they ship.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
