Page 120
12 articlesMultimodal LLM Inputs in Production: Vision, Documents, and the Failure Modes Nobody Warns You About
A practical guide to the failure modes engineers encounter when deploying multimodal LLMs in production — from vision token cost quadratic scaling and OCR vs. native vision trade-offs to PDF table extraction, hallucination on degraded images, and composable pipeline architecture.
The Production Retrieval Stack: Why Pure Vector Search Fails and What to Do Instead
Pure vector search fails on exact keywords, rare terms, and multi-constraint queries. A practical guide to building a production retrieval stack with BM25 hybrid search, cross-encoder reranking, and stage-level metrics.
Prompt Versioning in Production: The Engineering Discipline Teams Learn the Hard Way
Managing prompt changes in production LLM systems without version control is how teams end up paged at 2am with no rollback path. A practical guide to the deployment pipeline that prevents it.
Semantic Caching for LLM Applications: What the Benchmarks Don't Tell You
Production semantic caches hit 20–45% of traffic, not the 95% vendors claim. Here's what the threshold tuning problem looks like, which failure modes practitioners miss, and when to skip semantic caching entirely.
Structured Concurrency for AI Pipelines: Why asyncio.gather() Isn't Enough
Parallel tool calls are one of the most useful LLM capabilities — but asyncio.gather() introduces orphaned tasks, silent failures, and resource leaks that only surface under production load. Here's how to do concurrency correctly in agent pipelines.
JSON Mode Won't Save You: Structured Output Failures in Production LLM Systems
Production LLM structured output fails in four distinct ways, and JSON mode only catches one of them. A breakdown of syntax, schema, semantic, and distribution failure layers — and the validation stack that handles all four.
Synthetic Training Data Quality Collapse: How Feedback Loops Destroy Your Fine-Tuned Models
Using LLMs to generate fine-tuning data creates feedback loops that amplify biases, narrow distributions, and cause irreversible model degradation — and most teams don't notice until it's too late.
The Tool Selection Problem: How Agents Choose What to Call When They Have Dozens of Tools
Why LLM agents fail at tool selection as inventories scale past a dozen tools — token explosion, retrieval failure modes, and the layered routing architecture that keeps selection accurate at 50+ tools.
Voice AI in Production: Engineering the 300ms Latency Budget
Why voice AI feels robotic even when the model sounds good — and the streaming pipeline architecture, turn detection strategy, and transport choices that get you under 300ms.
When Thinking Models Actually Help: A Production Decision Framework for Inference-Time Compute
A decision framework for when reasoning models like o1, o3, and Claude extended thinking actually improve production outcomes — and when they burn tokens without improving results.
The Three Memory Systems Every Production AI Agent Needs
A practical guide to episodic, semantic, and procedural memory in AI agents — and why treating all persistent state as a single vector store will eventually break your production system.
What Nobody Tells You About Running MCP in Production
A practical guide to MCP's hidden production challenges — transport selection, tool schema design, tool poisoning attacks, and the gateway pattern that actually scales.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
