Blog
- 73Apr 9, 202610 minagent-reliabilitydistributed-systems
The Retry Storm Problem in Agentic Systems: Why Every Failed Tool Call Burns Your Token Budget
Naive retry logic across chained agent tool calls creates exponential cost amplification — a $0.01 task becomes a $2 meltdown. A four-layer defense stack with tool budgets, agent budgets, orchestration backpressure, and error classification prevents cascading failures in production AI agents.
Read more → - 74Apr 9, 202610 minllm-inferenceself-hosting
Self-Hosted LLMs in Production: The GPU Memory Math Nobody Tells You
GPU memory planning for self-hosted LLMs is almost always wrong because teams size for model weights and ignore the KV cache. A breakdown of the math, quantization tradeoffs between INT4/FP8/FP16, framework selection, and the real break-even calculation for going off cloud APIs.
Read more → - 75Apr 9, 202610 minai-agentsself-improvement
The Self-Modifying Agent Horizon: When Your AI Can Rewrite Its Own Code
Self-modifying AI agents — systems that rewrite their own source code to improve benchmark performance — have jumped from research curiosity to reproducible result. Here is what the benchmark numbers actually mean, the failure modes buried in the papers, and the governance infrastructure you need before deploying any of this in production.
Read more → - 76Apr 9, 202611 minllmcaching
Semantic Caching for LLMs: The Cost Tier Most Teams Skip
Semantic caching eliminates LLM calls for semantically equivalent queries — but real production hit rates range from 10% to 70%. Here's the math, threshold tradeoffs, invalidation pitfalls, and failure modes to evaluate before you build.
Read more → - 77Apr 9, 20269 minai-reliabilityproduction-ai
The Semantic Failure Mode: When Your AI Runs Perfectly and Does the Wrong Thing
Production AI systems can return valid, confident responses while completely missing user intent. A practical framework for detecting and closing the gap between task completion and task correctness using implicit behavioral signals, trajectory analysis, and intent-alignment scoring.
Read more → - 78Apr 9, 202610 minai-agentsagent-reliability
The Stale World Model Problem in Long-Running Agents
Long-running AI agents silently accumulate stale assumptions about external state—files, APIs, databases—that diverge from reality mid-task. Here's how the failure compounds, why no framework solves it automatically, and five patterns to build in explicit freshness guarantees.
Read more → - 79Apr 9, 202612 minai-engineeringstreaming
The Streaming Infrastructure Behind Real-Time Agent UIs
Four ways agent streaming fails in production — and the server-side architecture decisions for SSE transport, backpressure, graceful cancellation, and browser-refresh reconnection that actually make real-time agent UIs reliable.
Read more → - 80Apr 9, 202610 minllmreliability
Structured Output Reliability in Production LLM Systems
Naive JSON prompting fails 15–20% of the time in production. Learn how constrained decoding, schema design patterns, and the validate-retry loop eliminate structured output failures before they propagate through your pipeline.
Read more → - 81Apr 9, 20269 minproduction-aillm-agents
The Sycophancy Tax: How Agreeable LLMs Silently Break Production AI Systems
LLM sycophancy is present in 58% of production deployments and evades standard evals — the flip test, pressure testing, and architectural patterns that catch it before it undermines your system's integrity.
Read more → - 82Apr 9, 202610 mintext-to-sqlllm
Text-to-SQL in Production: Why Correct SQL Is the Easy Part
LLMs score 86% on SQL benchmarks and 10% on your actual warehouse. The queries that fail don't error—they return wrong data. A taxonomy of silent failure modes and the layered architecture that catches them.
Read more → - 83Apr 9, 202610 minsecuritymulti-agent
The Three Attack Surfaces in Multi-Agent Communication
82% of frontier LLMs comply with malicious commands from peer agents even when refusing them from users. Here are the three distinct attack surfaces — prompt injection, agent spoofing, and memory poisoning — and the protocol-level defenses each requires.
Read more → - 84Apr 9, 20269 minfeedback-loopsrlhf
Why Your Thumbs-Down Data Is Lying to You: Selection Bias in Production AI Feedback Loops
Only 1–3% of users click rating buttons — and they are systematically different from everyone else. How selection bias distorts RLHF training data, amplifies preference collapse, and hides 80% of your quality problems, plus the five implicit behavioral signals that capture ground truth from every user.
Read more →
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
