Page 83
12 articlesAI Content Provenance in Production: C2PA, Audit Trails, and the Compliance Deadline Engineers Are Missing
C2PA gives you cryptographic proof of who signed AI-generated content and when. But it doesn't survive your CDN, doesn't satisfy the EU AI Act alone, and won't tell you whether the content is truthful. Here's what production provenance actually looks like.
Why Users Ignore the AI Feature You Spent Three Months Building
AI features fail not because the model is bad but because users never discover them, trust them, or develop the habit of reaching for them. Here's how to fix that.
When Your AI Feature Ages Out: Knowledge Cutoffs and Temporal Grounding in Production
Products built on models with a fixed training cutoff break as the world diverges from training data. Here's how to detect cutoff-induced failures, manage RAG freshness, and design for temporal drift before it becomes a silent production regression.
The AI Feature Maintenance Cliff: Why Your AI-Powered Features Age Faster Than You Think
AI features don't just degrade — they degrade silently. Prompt drift, model updates, and distribution shift conspire to erode AI quality in production, and the dashboards stay green the whole time.
The AI Feature Retirement Playbook: How to Sunset What Users Barely Adopted
Most engineering teams know how to ship AI features. Almost none have a plan for retiring them. Here's the playbook for knowing when to quit and how to do it without burning users or accumulating compliance debt.
The AI Incident Response Playbook: Diagnosing LLM Degradation in Production
When your LLM feature degrades in production, standard SRE runbooks leave you blind. Here's the diagnosis tree, prompt rollback strategy, and postmortem template built specifically for AI systems.
The AI Incident Runbook: When Your Agent Causes Real-World Harm
When an AI agent causes real-world harm, your existing outage runbook will mislead you. Here is the playbook built for stochastic systems: how to bound blast radius without stack traces, preserve evidence before it disappears, and investigate beyond 'the model hallucinated.'
The AI Output Copyright Trap: What Engineers Need to Know Before It's a Legal Problem
Training data memorization, derivative works doctrine, and output ownership are live legal disputes with direct engineering consequences. Here's the risk surface and the controls that actually reduce liability.
The AI Taste Problem: Measuring Quality When There's No Ground Truth
How to evaluate AI outputs when accuracy metrics are meaningless — the engineering discipline behind pairwise studies, inter-rater reliability, and LLM-as-judge for copywriting, creative content, and design.
The Three Silent Clocks of AI Technical Debt
AI-specific technical debt—prompt drift, eval erosion, and embedding staleness—compounds invisibly unlike code debt. Here's how to detect each clock before it runs down.
The Annotation Economy: Why Every Label Source Has a Hidden Tax
A decision framework for choosing between human domain experts, crowd workers, synthetic LLM generation, and behavioral inference for eval label sourcing—and when annotation-free is actually right.
Annotation-Free Evaluation: Measuring LLM Quality Before You Have Ground Truth
A practical guide to measuring LLM output quality in week one — before you have labeled data. Covers self-consistency, constraint satisfaction, behavioral invariants, and LLM-as-judge, with the failure modes of each.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
