Page 51
12 articlesCapacity Planning for AI Workloads: Why the Math Breaks When Tokens Are Your Resource
Traditional provisioning models fail for LLM workloads. Here's the forecasting methodology that accounts for token burstiness, KV cache pressure, and why GPU utilization is a misleading signal.
The Cognitive Load Inversion: Why AI Suggestions Feel Helpful but Exhaust You
Real-time AI suggestions paradoxically increase cognitive load by shifting work from generation to verification. Here's the research and the design patterns that actually help.
Compaction Traps: Why Long-Running Agents Forget What They Already Tried
Context compaction silently drops the failure records and constraint information that prevent agents from re-attempting operations they already know won't work. Here's how to design around it.
Compound AI Systems: When Your Pipeline Is Smarter Than Any Single Model
Composing retrievers, rerankers, code interpreters, classifiers, and LLMs into pipelines that reliably outperform any single component — and the emergent failure modes that appear when you don't engineer the seams.
Context Windows Aren't Free Storage: The Case for Explicit Eviction Policies
Teams routinely pack codebases, histories, and documents into context and absorb the cost and quality degradation without measuring it. Here's why LLM context deserves the same explicit management as CPU registers — and how to build eviction policies that make it work.
The Context Window Cliff: Application-Level Strategies for Long Conversations
What actually happens when your LLM context fills up mid-session, why most frameworks handle it badly, and the summarization, selective retention, and externalization patterns that keep long-lived conversations coherent.
Continuous Deployment for AI Models: Your Rollback Signal Is Wrong
HTTP error rates can't detect behavioral regression in LLM upgrades. Here's how to run blue/green and canary deployments with behavioral divergence as the real rollback signal.
The Conversation Designer's Hidden Role in AI Product Quality
UX writing in system prompts, error messages, and capability disclosures directly shapes model behavior and user trust — in ways most engineering teams never measure.
Corpus Architecture for RAG: The Indexing Decisions That Determine Quality Before Retrieval Starts
Most RAG failures are diagnosed at query time but caused at index time. A technical guide to the chunk size, overlap, hierarchy, and metadata decisions that silently determine retrieval quality.
Cross-Encoder Reranking in Practice: What Cosine Similarity Misses
Vector ANN search finds semantically adjacent chunks, not necessarily the most useful ones. Layer cross-encoder reranking, MMR, and BM25 hybrid scoring to close the retrieval quality gap—with latency math that tells you when it pays off.
The Data Quality Tax in LLM Systems: Why Bad Input Hits Differently
Traditional ML degrades gracefully on noisy data. LLMs hallucinate confidently, corrupt vector stores, and propagate errors downstream with apparent authority. Here's how to measure and mitigate the data quality tax.
Dead Reckoning for Long-Running Agents: Knowing Where Your Agent Is Without Stopping It
When an agent runs for hours, knowing where it is—and whether it's still on track—becomes a first-class engineering problem. These are the patterns that solve it.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
