Page 140
12 articlesThe Reasoning Trace Privacy Problem: How Chain-of-Thought Leaks Sensitive Data in Production
Reasoning models correctly identify sensitive data 98% of the time yet leak it in their chain-of-thought 33% of the time. Here's why the scratchpad is a distinct attack surface and what production teams need to do about it.
The Reasoning Trace Privacy Problem: What Your CoT Logs Are Leaking
Chain-of-thought traces leak more PII than final model outputs, create a readable attack surface for prompt injection, and turn your observability stack into a GDPR liability. Here's what to do about it.
The Retry Storm Problem in Agentic Systems: Why Naive Retries Burn 200x the Tokens
Naive retry logic in AI agent systems creates exponential token-cost amplification across chained tool calls and multi-agent delegation. A layered defense architecture — circuit breakers, conversation-level budgets, deadline propagation, and honest degradation — prevents a single flaky API from cascading into a full agent meltdown.
The Retry Storm Problem in Agentic Systems: Why Every Failed Tool Call Burns Your Token Budget
Naive retry logic across chained agent tool calls creates exponential cost amplification — a $0.01 task becomes a $2 meltdown. A four-layer defense stack with tool budgets, agent budgets, orchestration backpressure, and error classification prevents cascading failures in production AI agents.
Self-Hosted LLMs in Production: The GPU Memory Math Nobody Tells You
GPU memory planning for self-hosted LLMs is almost always wrong because teams size for model weights and ignore the KV cache. A breakdown of the math, quantization tradeoffs between INT4/FP8/FP16, framework selection, and the real break-even calculation for going off cloud APIs.
The Self-Modifying Agent Horizon: When Your AI Can Rewrite Its Own Code
Self-modifying AI agents — systems that rewrite their own source code to improve benchmark performance — have jumped from research curiosity to reproducible result. Here is what the benchmark numbers actually mean, the failure modes buried in the papers, and the governance infrastructure you need before deploying any of this in production.
Semantic Caching for LLMs: The Cost Tier Most Teams Skip
Semantic caching eliminates LLM calls for semantically equivalent queries — but real production hit rates range from 10% to 70%. Here's the math, threshold tradeoffs, invalidation pitfalls, and failure modes to evaluate before you build.
The Semantic Failure Mode: When Your AI Runs Perfectly and Does the Wrong Thing
Production AI systems can return valid, confident responses while completely missing user intent. A practical framework for detecting and closing the gap between task completion and task correctness using implicit behavioral signals, trajectory analysis, and intent-alignment scoring.
The Stale World Model Problem in Long-Running Agents
Long-running AI agents silently accumulate stale assumptions about external state—files, APIs, databases—that diverge from reality mid-task. Here's how the failure compounds, why no framework solves it automatically, and five patterns to build in explicit freshness guarantees.
The Streaming Infrastructure Behind Real-Time Agent UIs
Four ways agent streaming fails in production — and the server-side architecture decisions for SSE transport, backpressure, graceful cancellation, and browser-refresh reconnection that actually make real-time agent UIs reliable.
Structured Output Reliability in Production LLM Systems
Naive JSON prompting fails 15–20% of the time in production. Learn how constrained decoding, schema design patterns, and the validate-retry loop eliminate structured output failures before they propagate through your pipeline.
The Sycophancy Tax: How Agreeable LLMs Silently Break Production AI Systems
LLM sycophancy is present in 58% of production deployments and evades standard evals — the flip test, pressure testing, and architectural patterns that catch it before it undermines your system's integrity.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
