Page 80
12 articlesData Provenance for AI Systems: Why Tracking Answer Origins Is Now an Engineering Requirement
When your LLM gives a wrong answer in production, can you trace exactly which documents contributed to it? If not, you're already behind. Here's how to build source lineage into AI systems from day one.
Goodhart's Law in Your LLM Eval Suite: When Optimizing the Score Breaks the System
How teams inadvertently game their own LLM evals, why benchmark scores diverge from production quality faster than you expect, and the meta-evaluation practices that keep your eval suite honest.
GPU Scheduling for Mixed LLM Workloads: The Bin-Packing Problem Nobody Solves Well
Serving multiple LLM models on shared GPU clusters wastes 30–50% of available compute. Here's why Kubernetes GPU scheduling fails for LLM inference and what actually works.
The Institutional Knowledge Drain: How AI Agents Absorb Decisions Without Transferring Understanding
When AI agents handle tasks end-to-end, the reasoning that once flowed through human conversation stops flowing. Here's what that costs engineering teams — and concrete patterns to stop the drain before it compounds.
Why Your Database Melts When AI Features Ship: LLM-Aware Connection Pool Design
AI features create bursty, long-running query patterns that exhaust connection pools designed for predictable web traffic. Pool segmentation, admission control, and the release-before-LLM-call pattern prevent AI workloads from starving your core product.
Machine-Readable Project Context: Why Your CLAUDE.md Matters More Than Your Model
Every AI coding tool reads a project-specific markdown file before responding. The quality of that file predicts output quality more reliably than the model behind it — yet most teams write them once, badly, and never update them.
MCP Is the New Microservices: The AI Tool Ecosystem Is Repeating Distributed Systems Mistakes
16,000+ MCP servers are live and growing — mirroring the microservices sprawl of 2016. A practical guide to the failure modes, gateway patterns, and maturity model that prevent your AI tool layer from becoming the next Death Star.
Measuring Real AI Coding Productivity: The Metrics That Survive the 90-Day Lag
Velocity proxies look compelling at day 30 but diverge from code quality by day 90. The lagging indicators and leading signals that reveal whether AI coding tools are compounding productivity or just moving debt downstream.
Phantom Tool Calls: When AI Agents Invoke Tools That Don't Exist
LLM agents sometimes fabricate tool calls — invoking functions that don't exist with plausible-looking parameters. Here's why it happens, the five failure categories, and the runtime defense patterns that catch phantom calls before they derail your workflows.
Quality-Aware Model Routing: Why Optimizing for Cost Alone Wrecks Your AI Product
Cost-optimized LLM routing saves money but silently degrades the queries that matter most. A practical guide to routing by task complexity, model capability, and production feedback — not just price per token.
When Your Database Migration Breaks Your AI Agent's World Model
A routine column rename can silently corrupt your AI agent's reasoning without triggering a single alert. Here's how schema-prompt contract testing and CI gates catch the drift before your users do.
Spec-to-Eval: Translating Product Requirements into Falsifiable LLM Criteria
Most AI features are specified in prose and evaluated in prose — which is why teams agree at standup and disagree at launch. A practical methodology for converting English requirements into concrete, falsifiable LLM evaluation criteria before writing a single prompt.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
