Page 121
12 articlesBeyond JSON Mode: Getting Reliable Structured Outputs from LLMs in Production
Prompt-only JSON extraction fails 5–20% of the time in production. A practical breakdown of all four generations of structured output techniques — from JSON mode to constrained decoding — with library recommendations and schema design rules.
What Your APM Dashboard Won't Tell You: LLM Observability in Production
Production LLM systems fail silently — green dashboards hide hallucinations, prompt drift, and wrong tool selection. Here's the instrumentation model that actually surfaces what's going wrong.
LLM Routing and Model Cascades: How to Cut AI Costs Without Sacrificing Quality
Running every query through a frontier model is the most common way teams overspend on AI. LLM routing and model cascades can cut costs by 45–85% while maintaining 95% of quality — here's how the patterns actually work in production.
Writing Tools for Agents: The ACI Is as Important as the API
How you design tools for AI agents — schemas, descriptions, return values, error messages — directly determines agent reliability. A guide to treating the agent-computer interface as seriously as any production API.
Fine-Tuning Is Usually the Wrong Move: A Decision Framework for LLM Customization
Most teams reach for fine-tuning too early. Here's a practical decision framework — backed by benchmarks and production examples — for when prompt engineering beats fine-tuning, when it doesn't, and the real economics of each approach.
Streaming AI Applications in Production: What Nobody Warns You About
Production streaming failures almost never come from the LLM itself — they come from NGINX buffering silently, load balancers timing out long-lived connections, and incremental JSON parsers degrading to O(n²). A practical guide to the infrastructure patterns that actually break at scale.
In Defense of AI Evals, for Everyone
Most teams that claim to skip evals are already doing evaluation — just badly. Here's why systematic AI evaluation matters, when lighter approaches are defensible, and how to run evals that surface real signal.
Data Flywheels for LLM Applications: Closing the Loop Between Production and Improvement
A production data flywheel turns user interactions into model improvements — but less than 1% of interactions yield explicit signal, and naively training on that 1% quietly poisons your system. Here's the architecture, feedback signals, and failure modes that determine whether your loop compounds or collapses.
Agentic Engineering Patterns That Actually Work in Production
What separates teams shipping real products with AI agents from teams stuck demoing impressive-looking outputs: TDD as a control mechanism, kill switches that live outside the reasoning path, and why code health is a precondition, not a byproduct.
Model Context Protocol: The Standard That Finally Solves AI Tool Integration
A practical guide to Model Context Protocol — how it works, where it wins over function calling, the security risks practitioners miss, and what to build with it today.
Reasoning Models in Production: When to Use Them and When Not To
A practical guide to deploying reasoning models in production — when the 5–10x cost premium is justified, how to build a routing architecture, and what metrics to track.
Structured Outputs in Production: Engineering Reliable JSON from LLMs
A practical guide to getting schema-valid JSON from LLMs in production — covering constrained decoding, provider APIs, schema design pitfalls, and the validation patterns that keep agent chains from falling apart.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
