Page 46
12 articlesShadow Traffic for AI Systems: The Safest Way to Validate Model Changes Before They Ship
How to use production traffic replay to validate LLM model and prompt changes before they affect users — the infrastructure, metrics, and sampling strategies that give you confidence at a fraction of A/B test cost.
The Shared Prompt Service Problem: Multi-Team LLM Platforms and the Dependency Nightmare
When five teams share one AI service, a single system prompt change silently breaks four evals. Here's the dependency management framework that prevents it.
The Skill Atrophy Trap: How AI Assistance Silently Erodes the Engineers Who Use It Most
Research shows AI coding assistance can lower comprehension scores by 17% and make experienced developers 19% slower while they feel 20% faster. Here's why mid-career engineers are most at risk and what to do about it.
SLOs for Non-Deterministic AI Features: Setting Error Budgets When Wrong Is Probabilistic
Standard availability and error-rate SLOs don't capture behavioral quality degradation in LLM features. Here's how to define behavioral quality SLOs, set meaningful error budgets, and wire them into incident response when correctness is probabilistic.
Specification Gaming in Production LLM Systems: When Your AI Does Exactly What You Asked
Specification gaming isn't just an RL theory problem — it shows up in every production LLM system where incentive gradients exist. Here's how to find it and build systems that are harder to game.
SRE for AI Agents: What Actually Breaks at 3am
Traditional SRE runbooks don't cover AI agent failure modes. Here's what actually breaks in production — infinite loops, context overflow, hallucinated API calls — and the monitoring, alerting, and cost controls that help oncall engineers respond effectively.
SSE vs WebSockets vs gRPC Streaming for LLM Apps: The Protocol Decision That Bites You Later
How SSE, WebSockets, and gRPC streaming fail differently under backpressure, what browser constraints and edge proxies break in production, and the failure-mode profile that should drive your transport choice.
Stateful Multi-Turn Conversation Infrastructure: Beyond Passing the Full History
Why 'pass the full conversation history' fails at p99 scale, and the session store designs, compression strategies, and operational patterns that actually hold up in production.
Structured Output Is Not Structured Thinking: The Semantic Validation Layer Most Teams Skip
JSON mode guarantees your LLM output matches a schema. It does not guarantee the output makes sense. The semantic validation layer catches contradictory fields, impossible date ranges, and domain constraint violations before they silently corrupt your data.
What Structured Outputs Actually Cost You: The JSON Mode Quality Tax
Constrained decoding guarantees valid JSON but extracts a hidden quality cost. Here's how to measure the tax on your workload and decide when it's worth paying.
Synthetic Seed Data: Bootstrapping Fine-Tuning Before Your First Thousand Users
AI personalization and task-specific fine-tuning hit a cold-start wall when there's no behavioral data. Learn how to generate 500–1,000 high-quality synthetic examples and the failure modes that can silently poison your model.
The Quality Tax of Over-Specified System Prompts
Bloated system prompts don't just cost more — they make your model dumber. Here's how to measure prompt obesity and trim without regression.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
