Page 108
12 articlesDocument AI in Production: Why PDF Demos Lie and Production Pipelines Don't
The gap between a working PDF demo and a reliable production pipeline is vast. Here's what breaks, how to detect it, and how to architect for 10,000+ documents a day.
Document Extraction Is Your RAG System's Hidden Ceiling
PDF-to-text pipelines silently discard tables, scramble reading order, and destroy section hierarchy before your embedding model ever sees the data. Here's how to find and fix the real failure layer in your RAG system.
Earned Autonomy: How to Graduate AI Agents from Supervised to Independent Operation
A framework for gradually expanding AI agent operational scope based on measured performance history, with rollback triggers and oversight mechanisms that prevent premature autonomy.
The Edge Inference Decision Framework: When to Run AI Models Locally Instead of in the Cloud
A practical decision framework for AI engineers: when on-device and on-premise LLM inference outperforms cloud APIs, and how to design the hybrid architecture that connects them.
The Enterprise AI Capability Discovery Problem
Enterprise users systematically underutilize AI features because they can't imagine the full capability surface from a chat box. Here are the design patterns that actually fix this.
Why Your Document Extractor Breaks on the Contracts That Matter Most
Fixed-layout extractors fail on the adversarial diversity of real enterprise documents. Here's the preprocessing pipeline that actually works in production, and the eval methodology that measures quality on the long tail.
Enterprise RAG Governance: The Org Chart Behind Your Retrieval Pipeline
40–60% of enterprise RAG deployments fail to reach production. The culprit is almost never the retrieval algorithm—it's governance: no document ownership, no access controls at query time, no PII handling, no freshness enforcement.
Eval Coverage as a Production Metric: Is Your Test Suite Actually Testing What Users Do?
A green eval suite can coexist with silently degraded production quality. Here's how to measure whether your evals actually represent real user intent—and what to do when they don't.
Event-Driven Agent Scheduling: Why Cron + REST Calls Fail for Recurring AI Workloads
Cron was built for sysadmin scripts, not autonomous agents. Here's what breaks when you use it for recurring LLM jobs—and the message queue architecture that actually works.
Why Your AI Model Is Always 6 Months Behind: Closing the Feedback Loop
AI models degrade silently because the gap between user failures and model updates spans months. Here's how to instrument implicit signals, run online evaluation, and use fast-path fine-tuning to compress that cycle from quarters to days.
The Feedback Loop Trap: Why AI Features Degrade When Users Adapt to Them
Self-induced distribution shift is the silent killer of production AI features. When users adapt their behavior to your AI's outputs, retraining on that adapted data makes the problem worse. Here's how to detect, measure, and break the loop.
Feedback Surfaces That Actually Train Your Model
Thumbs-up/down captures signal from the wrong users at the wrong moment. Here's how to design feedback surfaces that generate high-fidelity training data as a natural byproduct of product use.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
