Page 79
12 articlesYour AI Chat Transcripts Are Evidence: Retention Design for LLM Products Under Legal Hold
Chat logs are ESI. Design retention in four tiers, build a hold registry before you need it, and tag provenance at ingestion — or pay for the same architecture in the middle of discovery.
The AI Interview Collapse: Engineering Hiring Has Lost Its Signal
Technical roles saw 48% AI-assisted cheating across 19,368 interviews and 61% of cheaters cleared the bar. A look at why detection cannot win, why no-AI policies punish honest candidates, and the interview formats replacing the broken ones.
The AI Observability Leak: Your Tracing Stack Is a Data Exfiltration Surface
Hosted tracing SDKs quietly ship full prompts and responses past your trust boundary. A compliance playbook for LLM teams: classify fields, scrub before egress, audit the SDK as policy.
Your AI Product Needs an SRE Before It Needs Another Model
Most struggling AI teams run frontier models on 2012-era operations. The next hire that fixes it is usually an SRE, not another applied scientist.
Async Agents Need an Inbox, Not a Chat
Chat UX collapses when agents run past thirty seconds. The inbox primitive — durable run IDs, completion notifications, result-over-progress framing — is the product shape long-running agents actually need.
The benchmark leak: how your eval set quietly joins the training corpus
Public LLM benchmarks quietly become training data and inflate scores by 5–15 points. A practical contamination audit (n-gram, canary, held-out) and the organizational reasons your eval team won't run it.
The Cancellation Tax: Your Inference Bill After the User Hits Stop
Hitting stop halts your UI, not the GPU. Most providers finish generating and bill you for tokens no user ever read. Here's how to measure and shrink the gap.
The Cascade Router Reliability Trap: When Cost Optimization Quietly Wrecks Your p95
Cascade routers cut LLM spend dramatically — and quietly degrade tail latency, poison your training data, and invalidate your A/B tests. Here's what to instrument before the cost win turns into a reliability bill.
Your Chain-of-Thought Is a Story, Not an Audit Log
Reasoning traces read like audit evidence but only describe intent — not what executed. Why compliance needs a runtime-emitted sidecar action log.
The Deadlock Your Agent Can't See: Circular Tool Dependencies in Generated Plans
LLM-authored agent plans routinely contain implicit cycles that classic deadlock detection cannot see. A static plan-graph pass plus a runtime watchdog catches them before tokens evaporate.
Your Clock-in-Prompt Is a Correctness Boundary, Not a Log Field
LLM agents have no clock — they trust whatever timestamp you injected. Treat the time-in-prompt as a correctness contract, not a log field, or keep shipping the Tuesday-vs-Wednesday bug.
Cold-Start Evaluation: How to Ship an AI Feature With Zero Production Traces
No production traces means no free eval signal — but waiting for real users is not the fix either. A four-layer cold-start eval stack: structured dogfooding, scenario simulation with personas, an expert-labeled seed set, and a public adversarial probe library, with explicit weights so the loudest internal user doesn't set the rubric.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
