Page 86
12 articlesMultilingual Eval Cost Amplification: Why Seven Locales Doesn't Cost 7×
Your English eval suite cost $40K. The seven-locale international launch will not cost $280K — the real curve is closer to N×L^1.3 because cross-locale comparison is a meta-eval that doesn't decompose.
Your On-Call Rotation Needs an AI-Literacy Prerequisite Before It Pages Anyone at 2am
Shared on-call rotations break the moment one of those services is an LLM-backed feature. Here is the literacy prerequisite, dashboard hygiene, and shadow-period playbook that keeps the AI team out of bed at 2am.
On-Device AI Needs a Fleet Manager, Not a Model Card
Shipping one on-device model to every user means you're either burning battery on flagships or shipping a degraded product on the long tail. The discipline that fixes it looks more like a CDN than a model registry.
Pagination Is a Tool-Catalog Discipline: Why Agents Burn Context on List Returns
Tools that return unbounded lists turn agents into the SELECT * antipattern of the function-calling era. Pagination is a load-shedding primitive — make it a convention in your tool catalog, not a per-tool decision.
Per-Vector Version Tags: The Missing Column Behind Every Embedding Migration
Vector stores ship without the migration tooling Postgres has had for two decades — no ALTER TABLE, no online schema change, no per-row version. The discipline that makes embedding upgrades survivable starts with a single column most teams forget to add.
Prompt Cache Thrashing: When Your Largest Tenant's Launch Triples Everyone's Bill
Prompt caching's discount is real until one tenant's launch evicts everyone else's prefixes. The shared inference cache is a tenant-coupling surface, and the bill lands weeks after the incident.
Prompt Deprecation Contracts: Why a Wording Cleanup Is a Breaking Change
A four-word edit to a system prompt can break parsers, judges, and chained agents that pinned the old wording. Prompts are APIs with silent consumers — and the discipline that keeps them stable looks a lot like REST endpoint deprecation.
Prompt Linting Is the Missing Layer Between Eval and Production
Behavioral evals catch what your model says; they cannot catch what your prompt is. A prompt linter — fast, deterministic, structural — closes the gap that ships eval-green and surfaces as a 11pm production incident.
Prompt Position Is Policy: The Silent Merge Conflict When Three Teams Co-Own a System Prompt
When a system prompt grows past 2K tokens, position bias makes a moved instruction as load-bearing as a rewritten one — and line-based diffs hide it. How three teams silently overwrite each other's intent, and the section-ownership and eval discipline that catches it.
The Customer Record Hiding in Your Few-Shot Prompt Template
The 'representative customer' you pasted into your few-shot prompt six months ago is still in production — re-identifiable, re-shipped, and invisible to DLP.
The Refusal Latency Tax: Why Layered Guardrails Eat Your p95 Budget
Layered safety pipelines silently triple p95 latency and cost on the long tail. Treat guardrails as a budgeted resource with tiered classifiers, parallel checks, and an honest latency contract.
The Reranker Is the Silent Second Model Your RAG Eval Never Measures
Most RAG pipelines run two models in series — retriever and reranker — but eval suites only grade the generator's output. When the reranker drifts, the dashboard shows answer quality dropping with no causal arrow. Here's how to build a reranker eval that catches the silent regressions.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
