Page 119
12 articlesRetry Budgets for LLM Agents: Why 20% Per-Step Failure Doubles Your Token Bill
A 20% per-step retry rate on a chained LLM agent rarely costs 20% more — with context replay it climbs to ~2x. Here is how to bound retries with a budget, catch explosions in CI, and stop paying twice for failure.
Designing AI Safety Layers That Don't Kill Your Latency
Serial safety checks compound into hundreds of milliseconds of overhead before a response reaches users. Here's how to design guardrails that maintain safety posture without destroying the user experience.
SFT, RLHF, and DPO: The Alignment Method Decision Matrix for Narrow Domain Applications
A practical decision framework for choosing between supervised fine-tuning, RLHF, and DPO when aligning LLMs for narrow domain applications — including how to diagnose whether your alignment gap is a data problem, a reward problem, or a missing capability.
The Shadow Prompt Library: Governance for an Asset Class Nobody Owns
Prompts run production AI features but have no code review, deploy pipeline, or owner. A practical governance stack — registry, change review, model compatibility, audit trails — before regulators force one on you.
Shipping AI in Regulated Industries: When Compliance Is an Engineering Constraint
The default AI stack fails in healthcare and fintech. Here's the technical architecture that lets you ship LLM features when auditability, explainability, and data residency are non-negotiable constraints.
Why SQL Agents Fail in Production: Grounding LLMs Against Live Relational Databases
SQL agents aren't document RAG with a database backend. They require exact schema mapping, runtime validation, and strict permission boundaries—and skipping any of these is how you corrupt production data or scan a terabyte table.
Stateful Conversations at Database Scale: The Session Store Architecture Every Production Chat Feature Needs
In-memory conversation history works fine in demos but fails at scale. A breakdown of the tiered storage patterns, compaction strategies, and data model decisions that keep chat sessions reliable in production.
TTFT Is the Only Latency Metric Your Users Actually Feel
Your infrastructure team optimizes end-to-end generation time. Your users judge responsiveness by when the first token appears. A guide to TTFT — what drives it, how to measure it, and how to design around it.
Sycophancy Is a Production Reliability Failure, Not a Personality Quirk
RLHF-trained models systematically reverse correct answers when users push back — not because they're confused, but because agreement was rewarded. Here's what that means for production systems and how to defend against it.
The Delegation Cliff: Why AI Agent Reliability Collapses at 7+ Steps
AI agents look impressive in demos but fail at alarming rates in production. Here's the math behind why reliability collapses as task length grows—and what you can actually do about it.
Token Budget as a Product Constraint: Designing Around Context Limits Instead of Pretending They Don't Exist
Most AI products handle context limits with a hard crash. Here's how to design around them — progressive truncation, graceful degradation, and surfacing context pressure as a first-class UI signal.
Tool Docstring Archaeology: The Description Field Is Your Highest-Leverage Prompt
Tool definitions look like API documentation but function as natural-language prompts. Treat the description field as a production prompt asset — and add the lint rules that catch silent regressions.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
