Page 148
12 articlesLLM Evals: What Actually Works and What Wastes Your Time
A practitioner's guide to LLM evaluation — why error analysis comes before infrastructure, when LLM-as-judge works, how to avoid benchmark score traps, and why evals are never truly done.
Systematic Debugging for AI Agents: From Guesswork to Root Cause
Debugging AI agents in production requires a fundamentally different approach than traditional software. Learn how trajectory normalization, executable constraints, and evidence-backed failure localization replace guesswork with systematic diagnosis.
Why Your LLM Evaluators Are Miscalibrated — and the Data-First Fix
Most teams write LLM evaluation criteria before reading the data — and that inversion is why their evaluators miss the failures that matter most. A data-first workflow, binary labels, and proper validation against held-out sets fix the root cause.
Harness Engineering: The Discipline That Determines Whether Your AI Agents Actually Work
Most teams deploying AI coding agents focus on model selection while ignoring the harness — the scaffolding, feedback loops, and invariants that determine real-world reliability. Here's what actually separates agents that ship from agents that drift.
Building Governed AI Agents: A Practical Guide to Agentic Scaffolding
Most AI agent deployments fail not because the LLM underperforms—but because the scaffolding around it lacks governance. Here's how to build agents that are secure, auditable, and trustworthy in production.
Context Engineering: The Discipline That Matters More Than Prompting
Why the real bottleneck in production LLM systems is context architecture, not prompt wording — and how to design context as a first-class system concern.
Your CLAUDE.md Is Probably Too Long (And That's Why It's Not Working)
Adding more rules to your CLAUDE.md often makes your AI coding agent follow fewer of them. Here's why instruction overflow happens and how to architect your agent files for reliable compliance.
Eval Engineering for Production LLM Systems
How to build LLM evaluation systems that actually catch failures—covering error analysis loops, eval cost hierarchies, LLM-as-judge methodology, CI/CD integration, and agent-specific pitfalls.
Multi-Agent Conversation Frameworks: The Paradigm Shift from Pipelines to Talking Agents
Most multi-agent failures aren't model failures — they're architecture failures. Here's how conversation-based agent frameworks work, where they win, and why unstructured agent networks can amplify errors 17x.
Agentic RAG: When Your Retrieval Pipeline Needs a Brain
Most RAG systems fail in production not because of bad models, but because engineers skip the control loop. A guide to agentic RAG architecture — routers, graders, hallucination checkers, and the failure modes that kill first deployments.
Why Your Existing Observability Stack Won't Save You When AI Agents Break
HTTP 200s and clean latency charts mean nothing when your AI agent is reasoning incorrectly. How execution-level tracing works, what to measure, and how the observability tooling landscape breaks down for production agent systems.
Token Economics for AI Agents: Cutting Costs Without Cutting Corners
AI agents consume 3–10x more tokens than chatbots — and the gap between an unoptimized and optimized deployment can reach 200x in cost. A practical guide to prompt caching, model routing, context compression, and hard limits that actually move the needle.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
