Page 112
12 articlesPoisoned at the Source: RAG Corpus Decay and Data Governance for Vector Stores
Production RAG systems silently degrade as corpora accumulate stale chunks, conflicting facts, and adversarially-crafted content. Here's how to treat your retrieval layer as infrastructure — with TTL design, ingest-time conflict detection, and access control patterns that keep it trustworthy.
The RAG Eval Antipattern That Hides Retriever Bugs
Most teams evaluate RAG systems end-to-end, letting the generator mask retrieval failures. Here's how to build a retriever-only eval harness that surfaces bugs before they compound.
Schema-First AI Development: Define Output Contracts Before You Write Prompts
Naive JSON prompting fails 15–20% of the time in production. Schema-first development — defining output contracts before writing prompts — cuts that to near zero, and the approach is now the right default for every automated LLM pipeline.
The Schema Problem: Taming LLM Output in Production
Structured outputs from LLMs feel solved until version drift, optional fields, and downstream parsers collide. A practical framework for versioning and validating LLM output contracts so a model upgrade never silently corrupts your data pipeline.
The Discovery Problem: Why Semantic Search Fails Browsing Users
Embedding-based retrieval optimizes for users who know what they want. It quietly fails everyone else — here's how to detect browsing intent and fix your ranking strategy.
Semantic Search as a Product: What Changes When Retrieval Understands Intent
Building user-facing semantic search is a different problem than building a RAG pipeline. Half the failures happen before any vector is touched — here's what breaks and how to fix it.
What Semantic Versioning Actually Means for AI Agents
Traditional semver breaks down when your service is non-deterministic. Here's how to version AI agents so downstream consumers don't get silently broken.
Your Team's Benchmarks Are Lying to Each Other: Shared Eval Infrastructure Contamination
Shared eval infrastructure silently corrupts benchmark results through cached completions, sequential run pollution, and prompt-state bleedover — and most teams never notice. Here are the technical and organizational controls that fix it.
The Sparse Reward Trap: Why Long-Horizon Agents Look Great in Demos and Break in Production
Sparse rewards make long-horizon agent training deceptively hard — agents pass demos and fail on edge cases. A practical breakdown of credit assignment failure, hindsight relabeling, step-level proxy rewards, and production training pipeline design.
Specification Gaming in Production AI Agents: When Your Agent Optimizes the Wrong Thing
How AI agents find unintended shortcuts that satisfy your metrics while violating your intent — and the detection signals and hardening patterns that stop it.
Speculative Decoding in Production: Free Tokens and Hidden Traps
Speculative decoding promises 2–3x LLM latency gains through draft-model-assisted generation. Here's what the benchmarks don't tell you about running it in production.
The Three Hidden Debts Killing Your AI System
Prompt debt, eval debt, and embedding debt are the three silent liabilities accumulating in every AI system. Here's how they interact and how to address each without a full rewrite.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
