Page 19
12 articlesThe AI Risk Register: What Your CRO Will Demand the Morning After
Your enterprise risk register has rows for cyber, vendor, regulatory — but no row for the autonomous agent that just took an action under your credentials and produced a customer-visible loss. Here are the five columns the CRO will ask for the next morning.
AI Shadow IT: When Product Teams Build Their Own LLM Proxy
Shadow LLM proxies bypass cost attribution, audit logs, and DPAs because the platform gateway loses to product deadlines. The fix is a paved road that beats the side-channel on time-to-first-token, capability parity, and developer experience.
Argument Hallucination Is a Drift Signal, Not a Model Bug
When the model invents an argument value, the cheapest hypothesis isn't 'the model failed' — it's 'the description you gave the model no longer matches the API on the other side of the wire.'
Why Your Bias Eval Passes in CI and Fails in Deployment
Static bias audits pass in CI and fail in production because input distributions drift. Continuous fairness monitoring with per-cohort SLOs and drift-aware release gates is the fix.
The 'Try a Bigger Model' Reflex Is a Refactor Smell
When every quality regression on your team gets routed to 'let's try the bigger tier,' you're paying capacity to mask an upstream bug. The discipline to break the reflex, and the gate to put in front of it.
Browser-Native AI Is a Per-Feature Decision: Four Axes Your Team Hasn't Priced
Browser-native AI is not a faster TensorFlow.js. It is a different runtime with a four-axis trade-off — latency floor, privacy, device fragmentation, capability ceiling — that does not collapse into a single answer.
Confidence Strings, Not Scores: Why Your 0.87 Badge Moves Nobody
A 0.87 confidence badge changes no user behavior. A natural-language hedge that names what the model didn't check changes a lot. Why probability scores are the wrong shape of signal, and how to ship uncertainty as content instead of a UI overlay.
Cost-Per-Correctness, Not Cost-Per-Token: The Unit Metric Your Bill Won't Tell You
Token spend is the numerator. Eval-graded outcomes are the denominator. Tracking only the bill is how cheap-tier migrations silently regress quality and inflate downstream support cost.
Cross-Team Agent SLAs Don't Compose: The 99% Math Your Org Forgot to Budget
When agents call agents across team boundaries, individual SLOs stop predicting end-to-end behavior. The four pieces that have to land before the composition math eats your reliability budget.
The Eval Bottleneck: Your Eval Engineer Is Now the Roadmap
In 2026 the throughput limit on AI features isn't model shipping or prompt iteration — it's eval engineering. Here's the staffing ratio, platform investment, and leadership reframing required before your only eval engineer quits.
Eval Differential as Branch Protection: Ship Score Diffs, Not Score Floors
Score floors let silent regressions ship while flagging real improvements. A baseline-aware, slice-level eval diff turns the eval gate into a regression detector your team can trust.
The Eval-Set Poison Pill: When Your Benchmark Becomes a Backdoor
Most teams trust the eval because nobody owns auditing it. The labeling pipeline is a human supply chain — and the gold set inherits whatever distortion the humans introduce.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
