Page 77
12 articlesDocument Injection: The Prompt Injection Vector Inside Every RAG Pipeline
Every PDF, Word doc, and spreadsheet your RAG pipeline ingests is a potential attack surface. Here's how document injection works, what it's already broken in production, and the sanitization architecture that actually defends against it.
Why Gradual Rollouts Don't Work for AI Features (And What to Do Instead)
Feature flags and canary deploys assume deterministic code. AI features are stochastic, quality degrades silently, and there's no real-time ground truth. Here's the mental model shift required to deploy AI safely.
The HITL Rubber Stamp Problem: Why Human-in-the-Loop Often Means Neither
Most human-in-the-loop implementations don't produce oversight — they produce paperwork. Here's why reviewers stop scrutinizing, and the design patterns that keep HITL meaningful at scale.
The Hybrid Automation Stack: A Decision Framework for Mixing Rules and LLMs
Rule-based automation is brittle but auditable. LLM automation is flexible but opaque. A practical decision framework for which tasks belong in which paradigm—and how to architect the seam between them.
Latency Budgets for AI Features: How to Set and Hit p95 SLOs When Your Core Component Is Stochastic
LLM latency doesn't behave like database latency. Here's how to define realistic p95 SLOs for AI-powered features, decompose the latency budget, and use hedging, streaming, and speculative execution to actually hit them.
The Provider Reliability Trap: Your LLM Vendor's SLA Is Now Your Users' SLA
You're advertising 99.9% uptime while your critical path runs through an API with a 99.5% SLA — and provider incidents cluster during peak traffic. Here's how to close the gap before an outage closes it for you.
LLMs as ETL Primitives: AI in the Data Pipeline, Not Just the Product
Most teams build AI features into their products. The quiet transformation is happening inside the data pipeline, where LLMs classify, enrich, deduplicate, and route records at scale — creating compounding data assets that product-only teams can't replicate.
The Multi-Tenant Prompt Problem: When One System Prompt Serves Many Masters
B2B AI products let customers customize behavior, but layered system prompts silently override each other — and nobody notices until an enterprise customer files a ticket. Here's the explicit instruction hierarchy that makes conflict resolution auditable.
The Multi-Variable Regression Problem: Isolating AI Failures When Everything Changed at Once
When your AI feature regresses and the model version, prompt, retrieval corpus, and tool schemas all changed on the same Friday, attribution becomes nearly impossible. Here's the controlled experiment discipline and shadow evaluation patterns that prevent the worst outcome.
The Operational Model Card: Deployment Documentation Labs Don't Publish
Published model cards tell you whether a model is safe — not whether it will hit your p95 SLA, what context lengths it degrades at, or how often it produces malformed JSON. Here's the test battery for building the deployment documentation you actually need.
Prompt Linting: The Pre-Deployment Gate Your AI System Is Missing
Most teams ship prompt changes to production with less scrutiny than a CSS tweak. Static analysis for prompts — catching conflicting instructions, injection-vulnerable template slots, and positional traps — is the pre-deployment gate your AI system is missing.
Schema Entropy: Why Your Tool Definitions Are Rotting in Production
Tool definitions in production AI systems degrade silently over months. Here's how schema entropy forms, why agents can't self-correct, and the versioning and contract-testing practices that catch rot before it breaks live agents.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
