Page 6
12 articlesThe Streaming Abort Your Provider Billed Anyway: A 14% Gap Hiding in Your Invoice
Your stop button fires AbortController.abort(). Your provider keeps generating until the next batch boundary, then bills the difference. The gap is a measurable engineering problem with a finance owner.
The Streaming UI That Committed a Partial Answer Your Model Never Finished
When a stream errors mid-generation, the optimistic write can become the canonical record. Why chat UIs ship partial answers as real ones, and the finalization contract that fixes it.
The Summarizer That Paraphrased Away the User's Literal Question
Conversational summarizers compress older turns into paraphrase that drops the verb the user's question hinged on, so the model on turn forty-one confidently answers the wrong question. Treat the summarizer like a model in the critical path: pin the user's exact words, evaluate entity coverage, and stop letting the token-budget knob decide what your product forgets.
The Supervisor Agent That Rubber-Stamped Its Subagent Because They Shared a Prompt Template
A supervisor-reviews-subagent loop with a shared prompt template is not an independent check — it is the same model agreeing with itself, and the high approval rate is the tell.
The System Prompt Your Screenshare Leaked to a Vendor on a Support Call
System prompt protection programs harden the application boundary and ignore the screen pixel. The disclosure surface is every screenshare, Loom, and vendor support call your engineers join.
The Thumbs-Up Button That Poisoned Your Eval Set Through the Back Door
Implicit thumbs-up feedback skews your eval set toward the cohort that clicks and away from the cohort that pays. A breakdown of how the leak happens and how to govern around it.
The Token Count Your Client Estimated And Your Provider Invoiced
A small drift between your local tokenizer estimate and the provider's billed tokens averages out to noise until it concentrates on multilingual content, pasted code, and tool schemas — at which point finance asks why the AI line item no longer matches your own logs.
The Tokenizer Upgrade That Invalidated Every Prompt Cache Prefix
A tokenizer change can collapse your prompt cache hit rate from 80% to single digits overnight while every other signal stays green. Here is what breaks, why it is invisible, and what to instrument.
The Tool Description That Drifted Out of Sync With the Tool It Described
When a tool's JSON schema and its prose description live in different places, they drift independently — and the model follows the description into bugs the schema would have caught. Why tool descriptions are part of your effective system prompt, and how to keep them honest.
The Tool Schema Migration That Broke Your Agent's Retries for Two Weeks
A six-week deprecation window worked perfectly for every human consumer and silently broke the agent for fourteen days. Why retry budgets, parsing errors, and graceful fallbacks compose into outages nobody pages on — and the metrics that catch them.
The traceparent header your gateway dropped between LLM call and tool execution
Production LLM agent stacks routinely drop the W3C traceparent header between the model call and tool execution, fragmenting one user turn into a forest of orphan traces — here is where the leak happens and how to seal it.
The Transcription Confidence Score Your Agent Trusted After the Vendor's Recalibration
A confidence threshold tuned to one ASR model is a contract with that specific calibration. When the vendor re-trains the confidence head, the same number means something different — and your gate silently loosens.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
