Page 31
12 articlesLLM-as-Classifier in Production: Why Accuracy Is the Wrong Metric
Accuracy alone doesn't predict whether an LLM-based classifier will survive in production. The real constraints are calibration, per-class metrics, latency SLOs, and the testing patterns that reveal production readiness.
The 20% Problem in Model Routing: When Cost Optimization Creates Second-Class Users
Cost-optimized LLM routing silently creates quality gaps for specific user groups. Learn why the escalated 20% of queries isn't randomly distributed, how to audit routing tiers by cohort, and how to design policies with fairness constraints.
The Org Chart Problem: Why AI Features Die Between Teams
42% of AI initiatives fail not because the model doesn't work, but because no single team owns the feature end-to-end. A breakdown of the accountability gaps that kill AI features and the ownership models that actually close them.
The Output Coupling Trap: Why Multi-Agent Systems Fail Silently at Interface Boundaries
Downstream agents silently depend on the exact output format of upstream agents. When that format drifts, the failure looks like model error. Here's how to build agent interfaces that make format dependencies explicit before they kill your pipeline.
The Consistency Gap: Why Parallel LLM Calls Contradict Each Other and How to Fix It
When agents fire multiple concurrent LLM calls about the same entity, they often reach incompatible conclusions. Here are the architectural patterns—entity canonicalization, cache warming, evidence-based reconciliation, schema enforcement—that prevent it.
Permission-Aware Retrieval: Why Access Control in Enterprise RAG Must Live in the Vector Layer
Most enterprise RAG systems enforce access control in the application layer — and most of them leak confidential documents to the wrong users as a result. Here's why security must live in the retrieval layer itself.
The Persona Lock Problem: How Long-Lived AI Sessions Trap Users in Their Own Patterns
AI systems that adapt to user behavior over time create self-reinforcing loops where early preferences calcify into defaults users can't escape. Here's what persona lock looks like in practice and how to design around it.
The Words You Choose in Your System Prompt Change What Your Agent Will Risk
Gain-frame vs. loss-frame prompts produce measurably different agent behavior at decision boundaries. Here's what that means for how you write system prompts.
The Prompt Versioning Problem: Why Your Prompt Changes Are Untracked Production Risks
Prompt changes break production as reliably as API contract changes — but most teams ship them with zero versioning, no evaluation, and no rollback. Here's the engineering discipline that fixes this.
The Provider Behavioral Fingerprint: What Doesn't Survive a Model Switch
Switching LLM providers breaks production in ways capability benchmarks never catch — refusal tone, JSON serialization quirks, whitespace conventions, and context degradation curves that your codebase silently depends on. Here's how to surface these invisible contracts before migration.
The Rollout Sequencing Problem: Why Co-Deploying Model and Infrastructure Changes Destroys Observability
Co-deploying a context window increase, a model version bump, and a batch size change in the same release turns a debugging problem into a debugging impossibility. Here's the sequencing discipline that keeps AI systems legible.
The Shadow Compute Tax: Why Your AI Inference Bill Is Bigger Than Your Users Deserve
Proactive generation, background summarization, and eager context preparation consume inference budget for outputs users never see. Here's how to measure the waste and stop paying for it.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
