Page 113
12 articlesTesting the Untestable: Integration Contracts for LLM-Powered APIs
Deterministic test suites fail for non-deterministic LLM outputs. Learn property-based testing, behavioral invariant assertions, and semantic snapshot strategies that give you regression coverage without brittleness.
The Testing Pyramid Inverts for AI: Why Unit Tests Are the Wrong Investment for LLM Features
How the classic testing pyramid breaks for LLM features, why prompt-level unit tests give false confidence, and the test allocation strategy that matches how AI failures actually distribute.
Tokens Are a Finite Resource: A Budget Allocation Framework for Complex Agents
How to treat the context window as a scarce compute budget with explicit allocation across system prompt, memory injection, tool results, and scratch space — and what happens to agent reliability when you run out mid-task.
Vector Store Access Control: The Row-Level Security Problem Most RAG Teams Skip
Multi-tenant RAG systems silently serve the wrong documents when chunk-level authorization isn't enforced at query time. Here's why post-retrieval filtering is security theater, and the patterns that actually work.
When Your Agent Framework Becomes the Bug
High-level agent frameworks accelerate early prototyping but hide failure modes that surface in production — opaque retry amplification, invisible token costs, and debugging walls that require reading framework source. Here is how to recognize when your framework has become the bottleneck and how to migrate without a full rewrite.
Zero-Shot vs. Few-Shot in Production: When Examples Help and When They Hurt
The empirical case for when to use zero-shot vs. few-shot prompting — and why static examples at scale often make things worse.
Agent Fleet Observability: Monitoring 1,000 Concurrent Agent Runs Without Dashboard Blindness
Individual span trees per agent run collapse at fleet scale. Here are the fleet-level signals, sampling strategies, and behavioral fingerprinting techniques that actually work when you're running hundreds of concurrent agents.
Agent Identity and Least-Privilege Authorization: The Security Footgun Your AI Team Is Ignoring
When your AI agent calls internal APIs, whose identity does it present? Most teams give agents a broad service account token and move on. Here's why that's a security footgun and what production-grade agent authorization actually looks like.
The Agent Loading State Problem: Designing for the 45-Second UX Abyss
Users abandon silent UIs at ten seconds, but modern agents run thirty to one hundred twenty. The gap is a design surface most teams still fill with a spinner — here is what to ship instead.
Your Agent Traces Are Lying: Cardinality, Sampling, and Span Hierarchies for LLM Agents
Distributed tracing was designed for ~10 spans per request. A single agent run can produce hundreds, and default OpenTelemetry configurations systematically undercount the work. Here's the span hierarchy, tail sampling policy, and payload handling that survive production agent workloads.
Agentic Task Complexity Estimation: Budget Tokens Before You Execute
LLM agents commit resources before knowing how deep a task runs. Here's the complexity estimation layer — tiered routing, budget-tracker injection, plan template caching, and DAG-based decomposition — that prevents irreversible early mistakes and makes agent costs predictable.
When Your AI Agent Consumes from Kafka: The Design Assumptions That Break
Running AI agents on message queues breaks the assumptions baked into queue semantics. Here's how idempotency, ordering, and backpressure work differently when your consumer is stochastic.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
