Page 81
12 articlesYour Fine-Tuning Corpus Is a GDPR Data Artifact, Not Just an ML Asset
Fine-tuned weights encode customer PII that survives database deletion. A practical guide to treating training corpora as data artifacts under GDPR — lineage documentation, adapter isolation, and the compliance conversation to have before the first fine-tune ships.
First-Touch Tool Burn: Why Your Agent Reads Twelve Files Before Doing What You Asked
AI agents burn 60–80% of their token budget on reads before the first edit. Task-class routing, exploration budget caps, and plan-then-act gating cut the waste.
Free Tier Abuse Economics: When Your AI Generosity Gets Ratio'd by Bots
Free tier strategies built for SaaS quietly bankrupt AI products. Here's how bots monetize your generosity, and the rate limits, proof-of-work, and fingerprinting patterns that stop the bleeding.
GPU Starvation: How One Tenant's Reasoning Prompt Stalls Your Shared Inference Endpoint
One reasoning prompt can drag p99 latency for every other request on a shared inference endpoint. Here is why continuous batching and KV-cache pinning cause head-of-line blocking, the diagnostic signal nobody watches, and four mitigations — chunked prefill, priority scheduling, per-tenant token caps, and request-class isolation — ordered by how invasive they are.
The Hallucinated Success Problem: When Your Agent Says Done and Means Nothing
Agents that confidently report task completion without doing the work silently corrupt your dashboards. Here are the verification patterns that catch them before users do.
Human-in-the-Loop Is a Queue, and Queues Have Dynamics
Approval steps in agent workflows behave like production queues — with backlog growth, staleness, fatigue, and priority inversion. Here's how to design HITL that survives scale.
Your P99 Is Following a Stranger's Traffic: The Noisy-Neighbor Tax in Hosted LLM Inference
Hosted LLM APIs share GPUs, batches, and KV-cache budgets across tenants you never see, so your tail latency moves with strangers. Here is how to prove it, mitigate it, and decide when to flip to dedicated capacity.
Inference Is Faster Than Your Database Now
The model's share of request latency has collapsed. Your own feature store, auth, and Postgres calls are now the long tail — and most AI architectures haven't noticed.
Interview Mode vs. Task Mode: The Unspoken Contract Your Agent Keeps Breaking
Most 'asks too many questions' and 'didn't ask enough questions' complaints are the same bug — your agent picked the wrong contract. Here is how to detect and surface it.
LLM-as-Compiler Is a Metaphor Your Codebase Can't Survive
Framing LLMs as compilers quietly cancels the disciplines — review, refactoring, architectural judgment — that keep AI-generated codebases maintainable past the six-month wall.
LLM-as-Judge Drift: When Your Evaluator Upgrades and All Your Numbers Move
A regression suite that flips red without any prompt change is usually the judge, not the candidate. How evaluator drift fakes wins and losses, why pinned judges and calibration cadence matter, and what to log in eval metadata to stop the dashboard from lying.
Your LLM Span Is Lying: What APM Tools Don't Show About Inference Latency
Standard APM treats an LLM call as one opaque span — but prefill, decode, cache misses, and batch position all hide inside that duration. Here is the tracing surface you actually need.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
