Page 109
12 articlesFleet Health for AI Agents: What Single-Agent Observability Gets Wrong at Scale
Scaling from one agent to a thousand exposes fleet-level failure modes that single-agent observability tools miss entirely: version heterogeneity, correlated provider cascades, and token spirals that burn monthly budgets in minutes.
GraphRAG vs. Vector RAG: When Knowledge Graphs Beat Embeddings
Vector embeddings degrade to zero accuracy on multi-entity queries in compliance and enterprise domains. Here's when knowledge graphs are the right call — and the operational costs you're signing up for.
Where to Put the Human: Placement Theory for AI Approval Gates
The most common HITL mistake isn't skipping human review — it's placing it at the wrong point. A framework for classifying agent actions by risk and inserting approval gates exactly where they prevent irreversible damage.
When Embeddings Aren't Enough: A Decision Framework for Hybrid Retrieval Architecture
A practical framework for when to combine BM25 with dense embeddings, how to handle metadata filters without killing recall, and when cross-encoder reranking is worth the latency cost.
The Insider Threat You Created When You Deployed Enterprise AI
Giving employees AI coding assistants and document search agents also gives compromised insider accounts significantly amplified capability. Here's the threat model and the architectural controls that limit blast radius.
The Instruction Complexity Cliff: Why LLMs Follow 5 Rules Reliably but Not 15
Frontier models reliably satisfy around 3 stacked constraints and forget rules buried in the middle of long prompts. Here's what the empirical data shows about instruction compliance degradation — and the design patterns that keep system prompts reliable at scale.
The Jagged Frontier: Why AI Fails at Easy Things and What It Means for Your Product
AI's capability curve is jagged, not smooth — superhuman at some tasks, shockingly bad at adjacent ones. Here's how that creates invisible product traps and what to do about it.
The Knowledge Contamination Problem: When Your RAG System Ignores Its Own Retrieval
LLMs confidently answer from training memory even when retrieval provides better facts. Here's how to detect when a model ignores context versus when retrieval simply fails — and what to do about it.
Knowledge Cutoff Is a Silent Production Bug
A model's training cutoff is not a documentation footnote — it is a class of time-delayed production failure that conventional monitoring cannot see. Here is how to detect it, contain it, and design around it.
Live Web Grounding in Production: Why Calling a Search API Is Only the Beginning
Why 'just call a search API' produces a far worse pipeline than engineers expect — the latency math, failure modes, and architectural patterns that separate demo-quality from production-ready web grounding.
LLM-as-Annotator Quality Control: When the Labeler and Student Share Training Data
Using an LLM to label data for fine-tuning another LLM sounds efficient — until both models have absorbed the same internet text. Here's how shared pretraining creates systematic labeling failures, and the detection and mitigation strategies that actually work.
When LLMs Beat Rule-Based Systems for Data Normalization (And When They Don't)
LLMs handle the long tail of messy production data better than rules — but at a cost that surprises most teams. Here's the hybrid architecture, cost math, and validation patterns that actually hold up in production.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
