Page 81
12 articlesWhat 99.9% Uptime Means When Your Model Is Occasionally Wrong
Traditional SLAs are meaningless for AI features where success is probabilistic. Here's the contract language and internal SLO design that lets engineering teams ship AI without open-ended liability.
Structured Output Reliability in Production: Why JSON Mode Is Not a Contract
JSON mode guarantees valid syntax — not correct answers. A breakdown of the three failure modes that kill production AI pipelines and the three-layer validation architecture that actually catches them.
Subgroup Fairness Testing in Production AI: Why Aggregate Accuracy Lies
Aggregate accuracy hides systematic failures for specific demographic and linguistic subgroups. The subgroup eval methodology, disparity SLOs, and production monitoring patterns that catch bias before it reaches users at scale.
The Sycophancy Trap: Why AI Validation Tools Agree When They Should Push Back
RLHF-trained models have a systematic agreement bias that makes them dangerous for code review, fact-checking, and decision support. How to measure it and restore appropriate pushback.
Synthetic Eval Bootstrapping: How to Build Ground-Truth Datasets When You Have No Labeled Data
How to build a working LLM evaluation pipeline from zero labeled data using synthetic test generation, human-validated anchors, cross-model disagreement, and behavioral invariants — plus the failure modes that synthetic evals share with the models they test.
System Prompt Sprawl: When Your AI Instructions Become a Source of Bugs
As system prompts grow from hundreds to thousands of tokens, internal contradictions accumulate and model behavior becomes unpredictable. Here's how to detect, contain, and restructure before it costs you.
Temperature Governance in Multi-Agent Systems: Why Variance Is a First-Class Budget
Running all your agent components at the same temperature is as wrong as giving them all the same timeout. A guide to per-role sampling policy design that matches output variance to what each pipeline stage actually needs.
Temporal Context Injection: Making LLMs Actually Know What Day It Is
LLMs have no clock. Every date-sensitive feature you ship is broken by default — unless you engineer temporal context in explicitly. Here's how to do it without destroying your prompt cache.
Text-to-SQL in Production: Why Natural Language Queries Fail at the Schema Boundary
Why vendor demos of text-to-SQL work perfectly and production deployments fall apart — and the engineering techniques that actually close the gap.
The Token Economy of Multi-Turn Tool Use: Why Your Agent Costs 5x More Than You Think
Agent cost estimates built on single-call math are wrong by design. Here's how multi-turn tool use compounds token costs non-linearly — and the specific design levers that keep long-horizon agents economically viable.
Tokenizer Blindspots That Break Production LLM Systems
Why the '1000 tokens ≈ 750 words' assumption breaks in the cases that matter most: multilingual text, structured outputs, and code-heavy workloads — and the production bugs that follow.
Tool Output Compression: The Injection Decision That Shapes Context Quality
Tool results in AI agent pipelines vary 100× in token density. The strategy you choose for injecting them into context — raw, compressed, or extracted — sets a hard ceiling on your agent's accuracy, cost, and latency at scale.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
