Page 137
12 articlesThe Escalation Protocol: Building Agent-to-Human Handoffs That Don't Lose State
Most agent-to-human escalation breaks because teams treat it as an error state, not a designed workflow. A breakdown of the signal stack, state serialization format, oversight interface patterns, and the return path that preserves task continuity.
The Explainability Trap: When AI Explanations Become a Liability
Post-hoc AI explanations look authoritative but are structurally disconnected from model computation — how this creates regulatory exposure, misdirects users, and what honest explanation architecture actually looks like.
Fine-tuning vs. RAG for Knowledge Injection: The Decision Engineers Consistently Get Wrong
Fine-tuning teaches model behavior; RAG injects retrievable facts. Most teams confuse the two and spend months fine-tuning models that needed retrieval all along. Here's the decision framework that separates them.
Building GDPR-Ready AI Agents: The Compliance Architecture Decisions That Actually Matter
Four structural conflicts every regulated-industry engineer must resolve before shipping AI agents: right-to-erasure gaps in vector stores, audit trail requirements under the EU AI Act, data residency misconceptions, and the consent model that won't block future expansion.
GPU Memory Math for Multi-Model Serving: Why Most Teams Over-Provision by 3x
KV cache, not model weights, dominates GPU memory under concurrent load. The exact formulas for capacity planning, quantization tradeoffs (AWQ vs GPTQ vs GGUF), and bin-packing strategies that let you serve 4 models on hardware budgeted for 1.
Graph Memory for LLM Agents: The Relational Blind Spots That Flat Vectors Miss
Vector search retrieves similar facts but can't recover how facts relate — the structural blind spot that breaks agents handling multi-hop queries, evolving state, and long-horizon reasoning. Here's what graph memory fixes and what it costs.
Building a Hallucination Detection Pipeline for Production LLMs
A three-stage pipeline combining sentinel classification, token-level detection, and NLI verification catches LLM fabrications, contradictions, and outdated claims under 200ms P99 latency in production.
The Hidden Scratchpad Problem: Why Output Monitoring Alone Can't Secure Production AI Agents
Frontier models acknowledge the influence of sensitive inputs in their visible reasoning only 25–41% of the time. Here's why output-layer monitoring can't secure production agents—and how to build oversight that accounts for hidden computation.
The Hidden Token Tax: Where 30-60% of Your Context Window Disappears Before Users Say a Word
System prompts, tool schemas, chat history, and safety preambles silently consume 30-60% of your LLM context window before user content arrives — here's how to audit and reclaim it.
Hybrid Cloud-Edge LLM Architecture: Routing Inference Where It Actually Belongs
70-80% of production LLM queries never need a frontier model. A hybrid cloud-edge architecture routes each request to the cheapest tier that handles it well — using complexity classifiers, confidence cascading, and speculative decoding to cut costs 50-100x on the edge path without sacrificing quality.
Hybrid Cloud-Edge LLM Inference: The Routing Layer That Determines Your Cost, Latency, and Privacy Profile
A routing layer between edge and cloud inference cuts LLM costs 60–80% while improving latency and privacy — here's the engineering behind query-level routing, model compression, speculative decoding, and the orchestration that makes hybrid architectures work in production.
Hybrid Cloud-Edge LLM Inference: The Latency-Privacy-Cost Triangle That Determines Where Your Model Runs
A production guide to splitting LLM inference between on-device models and cloud APIs — covering the latency-privacy-cost triangle, compression techniques that preserve task accuracy, intelligent query routing, and the failure modes unique to hybrid architectures.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
