Page 87
12 articlesBias Monitoring Infrastructure for Production AI: Beyond the Pre-Launch Audit
Static fairness testing catches known problems against known datasets. Here's how to build the live monitoring infrastructure that catches the ones you didn't know to look for.
Cache Invalidation for AI: Why Every Cache Layer Gets Harder When the Answer Can Change
Traditional TTL and tag-based cache invalidation breaks down in AI systems. A breakdown of each cache tier — semantic caches, RAG knowledge bases, prompt caches, and embedding indexes — the failure modes specific to each, and the design patterns that keep them consistent in production.
Canary Deploys for LLM Upgrades: Why Model Rollouts Break Differently Than Code Deployments
Swapping an LLM version isn't a code deploy. Output semantics shift, downstream parsers break on subtly different schemas, and by the time your monitoring fires, thousands of users have already absorbed the failure. Here's the engineering discipline that makes model upgrades predictable.
The CAP Theorem for AI Agents: Choosing Consistency or Availability When Your LLM Is the Bottleneck
When an AI agent's tool call fails or the LLM times out, you face the same tradeoff distributed systems engineers know from the CAP theorem. Most agent frameworks silently choose availability — and pay for it in production.
Chunking Strategy Is the Hidden Load-Bearing Decision in Your RAG Pipeline
The chunk size and boundary strategy you commit to at index time sets a ceiling on your RAG system's quality. Here's how to tune it correctly and catch regressions before they become silent failures.
Communicating AI Limitations Across the Organization: A Framework for Engineering Leaders
Between 70 and 95% of enterprise AI initiatives fail — not because of bad models, but because legal, sales, and ops each build a different mental model of what the system does. A structured framework for engineering leaders to align stakeholders before miscommunication becomes a production crisis.
The Compound Accuracy Problem: Why Your 95% Accurate Agent Fails 40% of the Time
A 10-step agent pipeline where each step is 95% accurate succeeds only 60% of the time. Here's the math behind why, and the architectural patterns that actually bend the failure curve.
Contract Testing for AI Pipelines: Schema-Validated Handoffs Between AI Components
When one AI stage produces structured output consumed by the next, you've created a producer-consumer contract nobody tests. Here's the consumer-driven contract testing approach adapted for probabilistic AI outputs.
Conversation State Is Not a Chat Array: Multi-Turn Session Design for Production
The chat-history-as-array abstraction breaks in predictable ways at production scale. Here is the session design that actually holds up.
Cross-Lingual Hallucination: Why Your LLM Lies More in Languages It Knows Less
LLMs hallucinate 15–35% more in non-English languages, but aggregate benchmarks hide this gap. Here's why it happens, how to measure it, and the production architectures that reduce it.
The Data Flywheel Trap: Why Your Feedback Loop May Be Spinning in Place
The data flywheel sounds like a compounding advantage, but most implementations have at least three leakage points that silently corrupt the training signal. Here's the audit that separates real flywheels from their imitations.
Data Lineage for AI Systems: Tracking the Path from Source to Response
RAG pipelines without attribution metadata leave you blind when a response is wrong. Here are the lightweight span-tagging patterns that capture retrieval provenance and make hallucination debugging systematic.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
