Page 47
12 articlesContext Compression Artifacts: What Your Summarization Middleware Is Silently Losing
Context summarization is the standard fix for hitting context limits — but it destroys information non-uniformly. Negations, exact numbers, conditional dependencies, and tool-output attribution disappear first. Here's what practitioners need to know.
The Context Length Arms Race: Why Filling the Window Is the Wrong Goal
Every major model release now advertises a larger context window. But practitioners are discovering that filling that window degrades quality, inflates latency, and burns budget — while sparse, curated context consistently outperforms the naive approach.
The Context Limit Is a UX Problem: Why Silent Truncation Erodes User Trust
When an LLM silently drops earlier context to make room for new tokens, users don't see an error — they see a confused AI. This is a product design failure, not a model failure.
The Context Window Is an API Surface: Treat Your Prompt Structure as a Contract
Why treating your context window layout as a formal API contract — with named slots, versioning, and diff-friendly structure — makes LLM systems dramatically easier to debug and maintain.
Conversation-Aware Rate Limiting: Why Per-Request Throttling Breaks Multi-Turn AI
Per-request API throttling treats each conversation turn as an independent call, but a 10-turn debugging session is architecturally one task. Session budgets, semantic deduplication, and graceful degradation are the right primitives — here's why.
The Data Flywheel Assumption: When AI Features Compound and When They Just Accumulate Noise
Most teams believe more interaction data automatically makes their AI better. It doesn't. Here's what separates a real compounding flywheel from an expensive log file.
Data-Sensitivity-Tier Model Routing: Governing Which Model Sees Which Data
Most AI routing decisions optimize for cost and latency. But the privacy classification of your data should drive routing too — and getting this wrong creates silent compliance violations that only surface in audits.
Dead Letters for Agents: What to Do When No Agent Can Complete a Task
Message queues solved the stuck-message problem with dead-letter queues. Agent systems have the same problem but richer failure modes — here's how to adapt the pattern.
Diffusion Models in Production: The Engineering Stack Nobody Discusses After the Demo
Running diffusion models at scale exposes hard constraints that demos skip: GPU VRAM ceilings, LoRA hot-swapping architecture, a compliance stack for watermarking and NSFW moderation, and a cost-volume inflection where self-hosting beats every API tier.
End-to-End Latency Is Not P99 of Your LLM Call: The Multipliers Nobody Measures in Agentic Systems
Why the P99 latency of your LLM API call tells you almost nothing about what users actually experience in multi-step agent workflows — and the hidden multipliers that bridge the gap.
The Embedding Fine-Tuning Gap: Generic Vectors Don't Know What Relevant Means in Your Domain
Off-the-shelf embeddings optimize for semantic similarity, not domain relevance. Learn how contrastive fine-tuning with hard negatives, synthetic training data, and proper A/B evaluation closes the gap between benchmark scores and real retrieval quality.
Epistemic Trust in Agent Chains: How Uncertainty Compounds Through Multi-Step Delegation
When an orchestrator delegates to a subagent and accepts its answer, it inherits that agent's errors. How epistemic trust differs from authorization trust, why confidence compounds dangerously across agent handoffs, and what patterns actually address it.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
