Page 127
12 articlesValuable. Doable. Mine.
Reflecting on the historical lessons of Xerox PARC and Apple to explore how, in a rapidly changing technological environment, one can judge the true value of a technology and whether it aligns with personal business pursuits.
LLM Observability in Production: Tracing What You Can't Predict
Standard monitoring dashboards miss most of what goes wrong in LLM applications. A practical guide to distributed tracing, cost attribution, latency profiling, and debugging non-deterministic agent behavior at scale.
The Hidden Costs of Context: Managing Token Budgets in Production LLM Systems
Context windows aren't free storage — they're the biggest hidden cost in LLM systems. Learn how quadratic attention scaling, the lost-in-the-middle problem, and context length creep drive bills up, and the layered strategies that keep them under control.
Structured Output in Production: Getting LLMs to Return Reliable JSON
Getting LLMs to return valid, schema-compliant JSON in production is harder than it looks. Here's how constrained decoding, validation layers, and schema design decisions interact — and where each approach breaks down.
Prompt Engineering Deep Dive: From Basics to Advanced Techniques
A practical guide to prompt engineering for engineers building with LLMs in production — covering zero-shot vs few-shot tradeoffs, chain-of-thought benchmarks, structured output reliability patterns, and the five mistakes that break production prompts.
What AI Benchmarks Actually Measure (And Why You Shouldn't Trust the Leaderboard)
AI benchmark scores look objective, but data contamination, format sensitivity, and Goodhart's Law mean leaderboard rankings often tell you little about real-world performance. Here's what to watch for.
Tool Calling in Production: The Loop, the Pitfalls, and What Actually Works
A practical guide to tool calling in production LLM systems — covering the agentic loop, parallel execution formatting rules, writing effective tool descriptions, error recovery with is_error, and when tools add latency without value.
Why Multi-Agent Systems Break at the Seams: Designing Reliable Handoffs
Production multi-agent systems fail at the boundaries between agents, not inside them. A breakdown of the three dominant failure modes and the engineering patterns that prevent them.
Reasoning Models in Production: When They Help and When They Hurt
Reasoning models can solve problems that instruct models can't touch — but using them wrong costs 10x more and adds 10 seconds of latency to every request. Here's how to think about the tradeoff.
LLM Latency in Production: What Actually Moves the Needle
A practical breakdown of LLM latency — prefill vs decode phases, streaming, KV cache strategies, speculative decoding, and what to measure to ship faster AI applications.
Why Long-Running AI Agents Break in Production (And the Infrastructure to Fix It)
Long-running AI agents fail in predictable ways: compound error rates, synchronous timeouts, non-idempotent retries, and no plan for human interrupts. Here is the infrastructure that actually makes them reliable.
LLM Guardrails in Production: Why One Layer Is Never Enough
Five guardrails at 90% accuracy gives you 59% system correctness. A practical guide to tiered guardrail architecture—covering input and output validation, tool selection, latency tradeoffs, and why compound error rates are the hidden failure mode.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
