Page 49
12 articlesModel Routing in Production: When the Router Costs More Than It Saves
Most teams deploy model routers expecting automatic cost savings. The counterintuitive reality: a poorly designed router can cost more than sending every request to the expensive model. Here's the decision framework that actually works.
How to Pick the Right LLM Before You Write a Single Prompt
Public benchmarks have saturated and can't tell you which LLM will work in your system. A practical framework for evaluating models on the dimensions that actually matter: function-call reliability, structured output compliance, refusal rate on your domain, and latency under real concurrency.
Preference Data on a Budget: Capturing RLHF Signal Without a Research Team
How to collect pairwise preference signal from real users using implicit behavioral telemetry, inline editing, and A/B prompts — plus the minimum viable reward model setup that works without PPO infrastructure.
Prompt Injection at Scale: Defending Agentic Pipelines Against Hostile Content
Prompt injection is the #1 vulnerability in production AI agents. Here's the attack surface, why instruction-level defenses fail, and the architecture that keeps systems useful under adversarial pressure.
Prompt Regression Tests That Actually Block PRs
Most teams claim to test their prompts. Almost none have CI gates that will fail a build. Here's the lightweight harness that changes that without burning your API budget.
Retrieval Debt: Why Your RAG Pipeline Degrades Silently Over Time
Your RAG pipeline was working fine at launch. Now answers feel slightly off and nobody can explain why. Here's how retrieval debt accumulates through stale embeddings, tombstoned chunks, and encoder drift — and how to stop it before users notice.
Sampling Parameters in Production: The Tuning Decisions Nobody Explains
Temperature, top-p, and top-k silently shape your LLM's output quality. Here's what engineers actually need to know about tuning them in production—including why temperature=0 isn't deterministic and how top-p and temperature interact.
Structured Outputs Are Not a Solved Problem: JSON Mode Failure Modes in Production
JSON mode feels like a solved problem until you hit deeply nested schemas, enum-heavy types, or long completions that truncate silently. A complete failure taxonomy and the validation patterns that catch breakage before it reaches users.
When Code Beats the Model: A Decision Framework for Replacing LLM Calls with Deterministic Logic
The 'just use the model' reflex is the main driver of unnecessary complexity in AI systems. A decision framework for recognizing when a regex, lookup table, or rule-based classifier outperforms an LLM call on accuracy, latency, and cost.
Writing Acceptance Criteria for Non-Deterministic AI Features
Standard acceptance criteria break when your system is probabilistic. Here are the eval threshold contracts, example-based specs, and measurement patterns that let product and engineering agree on 'done' for AI features.
Tracing the Planning Layer: Why Your Agent Traces Are Missing Half the Story
Agent observability tools give you complete tool-call logs and timing, but the planning and reasoning that drove those decisions stays invisible. Here's what planning-layer tracing looks like, why it catches a completely different failure class, and how to instrument it today.
Agentic Web Data Extraction at Scale: When Agents Replace Scrapers
AI agents solve real problems traditional scrapers can't, but the 'LLM reads the page' prototype collapses at 1,000 pages per hour. Here's the hybrid architecture, cost model, and monitoring design that actually works in production.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
