Page 73
12 articlesModel Routing Is a System Design Problem, Not a Config Option
Treating LLM selection as a runtime dispatch decision — not a deployment constant — unlocks real cost savings. Here's how to think about routing signals, fallback failure modes, shadow routing, and the cost accounting that most teams skip.
Multi-Model Consistency: When Your Pipeline's Sequential LLM Calls Contradict Each Other
Three LLM calls in a single workflow can produce conflicting facts, entity references, and state claims. Here's how to design pipelines that stay coherent.
Multi-Session Eval Design: Catching the AI Feature That Gets Worse Over Time
Single-turn evals miss the class of AI failure that emerges only after state accumulates. How to design a multi-session eval harness, decay curves, and regression methodology that catch quality rot before users churn.
Multi-User Shared Agent State: The Concurrency Primitives You Actually Need
Most agent designs assume one user per session. Shared workspaces need distributed systems primitives to prevent silent data corruption when concurrent users give contradictory instructions.
Multimodal Pipelines in Production: What Breaks When You Go Beyond Text
Going multimodal in production means confronting a new class of failures: silent image rejections, PDF table misalignment, audio latency budgets, and cross-modal hallucination that text evals never surface.
The Noisy Neighbor Problem in Shared LLM Infrastructure: Tenancy Models for AI Features
When one feature's batch job eats the shared API quota, paying users see 429s. Detection signals and isolation patterns for shared LLM infrastructure.
PII in the Prompt Layer: The Privacy Engineering Gap Most Teams Ignore
How personally identifiable information flows uncontrolled into LLM inference calls, and the masking, tokenization, and logging architectures that close the compliance gap.
Pricing Your AI Product: Escaping the Compute Cost Trap
Traditional SaaS pricing assumes near-zero marginal cost per user. LLM features break that assumption — tokens can consume 20–40% of gross margin. Here's how to build a pricing architecture that survives.
Proactive Agents: Event-Driven and Scheduled Automation for Background AI
Most agent design literature assumes a human triggers execution. Production AI increasingly runs in the background — on schedules, change events, and system state transitions. Here's what that changes architecturally.
Prompt Canary Deployments: Ship Prompt Changes Like a Senior SRE
Prompt edits are as dangerous as code deploys — but almost nobody treats them that way. Here's the traffic-splitting, quality-monitoring, and rollback discipline that separates teams that catch regressions before users do from teams that find out on Twitter.
Prompt Diff Review as a Discipline: What Reviewers Actually Need to Ask
Traditional code review instincts don't map to prompt edits. Here's the checklist, the tooling, and the reviewer-author dialog that turn a prompt PR into a behavioral contract.
The Prompt Entropy Budget: Measuring Output Variance as a First-Class Production Metric
Most production LLM systems track accuracy but ignore variance. Measuring the distribution of outputs over identical inputs — your prompt entropy budget — is the missing metric that determines UX consistency at scale.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
