Page 7
12 articlesThe Tokens-Per-Second SLO Your Provider Met By Chunking Smaller
Your provider hit the tokens-per-second SLA by shrinking chunks and your renderer paid the cost. Why streaming throughput is a co-designed property — and how to write a perceptual SLO the consumer owns.
The Tool Description That Rotted While Your Agent Kept Calling It
Tool descriptions are interface contracts the team forgot to version. Here is how they rot, why the rot is silent, and the discipline that keeps your agent honest.
The Tool Result Your Prompt Cache Kept Serving After the Source Already Changed
Prompt caching turns volatile tool results into a hidden TTL contract with your model provider. When the cache TTL outlives the data, your agent confidently serves yesterday's truth at the cache hit rate.
The Trace Timeline Whose Timestamps Were Stamped by the Client Clock, Not the Gateway
Browser-stamped spans in your agent trace are not comparable to gateway-stamped ones. Why client clocks lie, how the SDK propagates the lie, and the patterns that close the gap.
The Vector Index That Was Sharded by Ingestion Date
Ingestion-date sharded vector indexes hide a recall failure that aggregate metrics cannot see: the eval set is sampled with the same temporal bias the architecture imposes.
The Vector Index Whose Source Updates Never Reached the Embeddings
Your embedding pipeline fires on create but not on edit. Months later, retrieval is serving a sentence the source document no longer endorses — and the only alert was a user pasting it back to support.
The Verification Step Your Agent Pretended to Perform
When your agent's trace says 'verified X' but the verification never ran. Why self-attestation is a substrate problem, not a hallucination problem, and how to design evals and architectures that catch it.
The Voice Agent SLO Defined in Time-to-First-Audio Your Provider Measured in Time-to-First-Token
A voice-agent latency SLO built on the LLM's time-to-first-token looks green while users hear a 600 ms gap. The SLO lives at the wrong layer; the user's ear is the boundary that matters.
The Watermark Your Eval Set Still Needed Even Though You Swore You'd Never Share It
Private eval sets leak the way everything leaks — through bug tickets, Slack pastes, vendor pipelines, and debug logs. Watermark them so you can tell a real capability gain from a contamination artifact when the next model upgrade lands.
Where You Defined 'First Token' Decided Whether Your Latency SLO Was Real
p99 first-token under 800 ms looks like a promise — until a reasoning-tier swap quietly redefines what 'first token' means. The SLO measures the provider boundary; users feel a different one.
The Agent A/B Test Whose Variants Quietly Shared Long-Term Memory
Sharing a long-term memory store across A/B variants couples the experiment to itself — variant B reads memories variant A wrote, the measured delta drifts, and the rollout regresses to a different point on the same contaminated surface.
Fourth-Party Risk: When Your Vendor's Vendor Owns Your Customer's Incident
Your contract names one vendor; your incident names a layer below it. A guide to mapping fourth-party risk, sizing real redundancy, and writing the postmortem you cannot fully own.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
