Page 89
12 articlesThe Latency Perception Gap: Why a 3-Second Stream Feels Faster Than a 1-Second Batch
A 3-second streaming response often feels faster than a 1-second batch response. Here's the psychology behind it and the engineering patterns that exploit it.
Why Your LLM Alerting Is Always Two Weeks Late
LLM quality degrades silently while your infrastructure metrics stay green. Learn the specific signals — semantic drift score, output schema conformance, user-repair rate — and anomaly detection patterns that catch model degradation 11 days before users start filing tickets.
Your Model Is Most Wrong When It Sounds Most Sure: LLM Calibration in Production
LLMs trained with RLHF are systematically miscalibrated — highest verbal confidence often marks incorrect outputs. How to measure calibration error on your task and fix the routing logic that depends on it.
LLM Cost Forecasting Before You Ship: The Estimation Problem Most Teams Skip
Token counts in production depend on user behavior you can't predict at design time. Here's how to build a cost model that bounds variance before launch—through simulation, canary traffic, and framework-level budget enforcement.
Model Migration as Database Migration: Safely Switching LLM Providers Without Breaking Production
Switching LLM providers or upgrading model versions is more like a database schema migration than a config change. Here's the production playbook engineers actually need.
LLM-Powered Data Migrations: What Actually Works at Scale
A practitioner's guide to using LLMs for schema migrations and ETL automation — covering the silent failure modes, layered validation architecture, schema-based prompting, and when LLMs should not replace traditional pipelines.
LLMs as Data Engineers: The Silent Failures in AI-Driven ETL
LLMs handle messy data edge cases that hand-coded ETL pipelines miss — but they also produce confidently wrong transformations with no error signal. Here's the validation, sandboxing, and monitoring stack that makes AI-augmented ETL safe in production.
What Model Cards Don't Tell You: The Production Gap Between Published Benchmarks and Real Workloads
Model card benchmarks are measured under ideal conditions that rarely match production. Here's the gap every team discovers too late — and the internal benchmark suite that catches it before deployment.
Model Deprecation Is a Systems Migration: How to Survive Provider Model Retirements
When your inference provider sunsets a model, swapping the model ID is the least of your problems. Here's the engineering discipline that keeps production AI running through retirements.
The Model Portability Tax: How to Architect AI Systems You Can Actually Migrate
Every model swap is a partial rewrite if you didn't design for portability. Here's the abstraction layer, capability negotiation, and regression testing infrastructure that turns model migrations from crisis deployments into planned operations.
Model Upgrade as a Breaking Change: What Your Deployment Pipeline Is Missing
Foundation model updates silently break downstream systems through output format shifts, tone changes, and reasoning divergence. Here's the infrastructure to detect and manage it.
Multi-User AI Sessions: The Context Ownership Problem Nobody Designs For
When multiple users share an AI assistant, context becomes a shared mutable resource with no access control. Here's how context leaks, personalization bleeds, and race conditions appear at team scale — and the isolation patterns that actually prevent them.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
