Page 47
12 articlesYour RAG Knows the Docs. It Doesn't Know What Your Engineers Know.
Most enterprise RAG systems only index written documents, missing the tacit knowledge that actually drives decisions. Here's how to build systems that capture what your engineers know before they walk out the door.
Temperature Is a Product Decision, Not a Model Knob
LLM temperature controls output variance — and that variance directly shapes user trust, engagement, and behavior. Most teams treat it as a technical default. It isn't.
Text-to-SQL at Scale: What Nobody Tells You Before Production
Text-to-SQL demos are easy; production deployments are not. Schema ambiguity, privilege escalation, and the 80% benchmark gap expose the engineering layer most teams skip.
Adding AI to Systems You Don't Own: The Third-Party Model Integration Playbook
Building on external model APIs means rate limits, behavioral drift, and cost shocks are imposed on you. Here's the architecture that survives provider changes, outages, and silent model updates.
The Transcript Layer Lie: Why Your Multimodal Pipeline Hallucinates Downstream
Treating ASR and OCR output as ground-truth text silently poisons downstream LLM reasoning — and the fix isn't better models, it's keeping confidence scores alive through the pipeline.
The User Adaptation Trap: Why Rolling Back an AI Model Can Break Things Twice
When a model update introduces subtly wrong behavior, users adapt their workflows around it. By the time you catch it and roll back, you may have two groups of broken users instead of one.
The Vanishing Blame Problem in AI Incident Post-Mortems
When an AI system degrades, blame diffuses across model, prompt, retrieval, eval, and infrastructure simultaneously. Here is the attribution framework that pins incidents to a specific layer before your post-mortem devolves into 'the model just changed.'
Why Vision Models Ace Benchmarks but Fail on Your Enterprise PDFs
Vision models post impressive benchmark numbers on document understanding, but enterprise teams routinely see silent failures on real PDFs. Here's what breaks and how to build pipelines that survive contact with production documents.
Who Owns AI Quality? The Cross-Functional Vacuum That Breaks Production Systems
AI quality failures rarely stem from bad models. They stem from nobody claiming ownership. Here's how to fix the accountability vacuum before it costs you.
Agent Identity and Delegated Authorization: OAuth Patterns for Agentic Actions
When an AI agent books a calendar event or sends an email on your behalf, it operates under delegated authority. Here's how to design OAuth scope contracts, rotation lifecycle, revocation triggers, and audit trails for production agentic systems.
Agentic Data Pipelines: Offline Enrichment and Classification at Scale
How AI agents change the design of ETL and batch-enrichment workflows — variable compute per record, confidence thresholds as operational contracts, schema design for downstream consumers, and monitoring patterns that distinguish model uncertainty from data ambiguity.
AI-Native API Design: Why REST Breaks When Your Backend Thinks Probabilistically
REST was built for fast, deterministic backends. LLM services are slow, probabilistic, and long-running — and the interface patterns that actually hold up in production look nothing like conventional HTTP API design.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
