Page 16
12 articlesTraining Your AI on Production Data Without Triggering a Legal Blocker
How behavioral telemetry for AI model improvement collides with GDPR and CCPA — and the federated learning, differential privacy, and consent architecture patterns that let you keep the feedback loop without triggering a legal blocker.
API Documentation Is Reliability Infrastructure: How Your Docs Determine Agent Success Rates
When AI agents consume your API via tool calling, documentation quality becomes a direct reliability variable. Ambiguous parameters and missing error semantics cause measurable failure rates that no amount of prompt tuning can fix.
Code-Specific RAG: Why General Retrieval Fails for Codebases
Token-based chunking destroys code's structural properties before the retriever ever sees them. AST-aware chunking, call-graph traversal, and test file co-location are the patterns that actually work for codebase retrieval.
The Context Format Decision Most Teams Make Accidentally: JSON vs Markdown vs Plain Text
Choosing between JSON, markdown, and plain text for LLM context isn't a stylistic preference — it determines reasoning mode, accuracy, and cost. Here's how to make the decision deliberately.
The AI Code Feedback Loop: How Today's Generated Code Trains Tomorrow's Models
As AI-generated code floods production codebases, it becomes training data for the next model generation. The feedback loop is already measurable — and the failure mode is subtle enough to arrive undetected.
Why AI Features Break A/B Testing (and the Causal Inference Methods That Don't Lie)
Standard A/B tests violate their core assumptions when applied to AI features. Here's how to measure real impact using causal inference methods that handle contamination, spillover, and long-horizon behavioral shifts.
The Cross-User Consistency Problem: When Your AI Gives Different Answers to the Same Question
Enterprise AI tools silently erode trust when teammates ask the same question and get different answers. Here's why temperature=0 doesn't fix it, and the engineering patterns that actually do.
The Dev-to-Prod Cost Shock: Why Your AI Feature Costs Pennies in Staging and Dollars in Production
Staging environments systematically hide the cost drivers that matter in production. Here's the gap between what you pay in dev and what hits your invoice at scale — and how to model it honestly.
The Domain Expert Bottleneck in RAG: Why Knowledge Curation Breaks Production AI
Building a RAG pipeline takes days. Maintaining the knowledge base that feeds it is what breaks teams in year one. Domain expert curation is the real last-mile problem in production RAG.
DORA in the Age of AI: When Deployment Frequency Lies
AI tooling inflates DORA metrics while silently degrading the team capabilities those metrics were built to measure. Here's what's happening to deployment frequency, lead time, CFR, and MTTR — and which supplemental signals actually tell the truth.
Embedding Model Churn: When Your Provider Silently Invalidates Your Entire Vector Index
When your embedding provider silently updates their model, every vector in your index becomes incompatible with new queries — with no errors, no alerts, just degraded retrieval. Here's how to detect it and survive it.
Ensemble vs. Debate: The Two Multi-Model Verification Paradigms and When Each Fails
Running more models doesn't guarantee better answers. When frontier LLMs share training data, their errors correlate at r = 0.77 — making three models effectively 1.3 independent ones. A breakdown of ensemble vs. debate verification, their distinct failure modes, and when neither approach works.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
