Page 139
12 articlesMultimodal LLMs in Production: The Cost Math Nobody Runs Upfront
How vision, audio, and video inputs change your LLM token budget — a breakdown of per-modality cost formulas, the multipliers that silently inflate production bills, and the architectural patterns teams use to control costs.
The N+1 Query Problem Has Infected Your AI Agent
The N+1 query problem from the ORM era has re-emerged at the AI agent tool call layer — sequential single-item fetches, redundant re-fetches, and over-fetching are silently inflating your latency and token costs. Here's how to diagnose it and fix it.
The Non-Determinism Tax: Building Reliable Pipelines on Probabilistic Infrastructure
Temperature=0 doesn't make LLMs deterministic. Batch composition, tensor parallelism, and floating-point non-associativity drive up to 72 percentage-point performance swings. Here's how to measure the variance and build application logic that's stable despite it.
Non-Deterministic CI for Agentic Systems: Why Binary Pass/Fail Breaks and What Replaces It
Binary pass/fail CI breaks down when every test run is non-deterministic. Statistical verdicts, graduated thresholds, trajectory fingerprinting, and sequential analysis catch real agent regressions without drowning teams in false failures.
Parallel Tool Calls in LLM Agents: The Coupling Test You Didn't Know You Were Running
Enabling parallel tool execution in LLM agents exposes hidden coupling in your tool design — the three silent failure modes, how to classify tools for safe parallelism, and when to consolidate instead of parallelize.
Production AI Incident Response: When Your Agent Goes Wrong at 3am
AI incidents don't look like software incidents — no stack traces, no 500 errors, just confident wrong answers and runaway loops. A practical guide to detection, triage, containment, and post-mortems for production LLM systems.
The Prompt Ownership Problem: What Happens When Every Team Treats Prompts as Configuration
A $340K production incident exposed what happens when prompts have no owner, no version history, and no review gate — and the lightweight governance model that prevents it.
Prompt Sprawl: When System Prompts Grow Into Unmaintainable Legacy Code
System prompts that start at 200 tokens and balloon to 4,000 silently degrade LLM performance. How to audit, decompose, and architect modular prompts that stay maintainable — applying DRY, separation of concerns, and version control to prompt management.
Why the Chunking Problem Isn't Solved: How Naive RAG Pipelines Hallucinate on Long Documents
Fixed-size and semantic chunking both fail in predictable ways on production documents. Here's what the research shows about RAG chunking failures, and the evaluation and architecture patterns that close the accuracy gap.
RAG's Dirty Secret: Your Retrieval Succeeds but Your Answers Are Still Wrong
Retrieval success doesn't guarantee correct answers. A third failure mode lurks between retrieval and generation—context sufficiency—where retrieved documents rank correctly but lack the specific information needed. Here's how to detect it and what to do about it.
The RAG Freshness Problem: How Stale Embeddings Silently Wreck Retrieval Quality
Semantic similarity has no temporal dimension — stale embeddings score just as high as fresh ones. The CDC pipelines, decay-weighted scoring, and monitoring stack that keep production RAG systems from silently serving outdated answers.
The Reasoning Model Premium in Agent Loops: When Thinking Pays and When It Doesn't
Reasoning models cost up to 86x more per query than standard models — and inside agent loops, that cost compounds with every iteration. A practical decision framework for when to route to reasoning models and when fast models are the better call.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
