Page 30
12 articlesThe Tool That Worked Until Two Agents Called It At Once
Agent tools pass single-caller tests and break the day a second agent shows up. Why concurrency bugs are structurally invisible to serial evals, and how idempotency, locking, and load tests fix them.
The Distributed Trace That Goes Dark at the Agent Handoff
Your span tree is clean right up to the moment one agent calls another — then it goes dark exactly where the bug lives. Here is why agent handoffs break trace context, and how to make the handoff carry it.
When Two Agents Share a Tool: Concurrency Bugs in Multi-Agent Systems
Spinning up a second agent turns you into a distributed systems engineer. Race conditions, lost updates, and dirty reads return as silent corruption — and how to design the tool layer to stop them.
Halted Is Not a Status: Why Agents Need a Typed Terminal-Reason Protocol
Agent dashboards report completion rate, but a halted run that gave up looks identical to one that succeeded. A typed terminal-reason protocol makes how an agent ends a first-class, monitorable signal.
The Undo Button Your Agent Assumes Exists
AI agents plan as if every action can be undone, a habit learned in reversible code sandboxes. Encode reversibility tiers into your tools so one-way doors stay safe.
Your Vector Index Is a Cache With No Invalidation Strategy
A vector index is a derived copy of your source data, which makes it a cache that goes stale: edits never propagate, deleted documents leave ghosts, and revoked permissions leak. Why RAG reliability is a cache-invalidation problem, not a similarity-search one.
The Vector Index Has a Staleness SLO Nobody Set
A nightly re-indexing job is a freshness promise nobody wrote down. How to turn vector index lag into a measured SLO, surface data age to agents and users, and re-index by decay rate instead of by habit.
Warm Pools and Cold Truths: The Hidden Latency Floor of Serverless LLM Inference
Scaling GPU inference to zero converts a steady dollar cost into a spiky latency cost hidden in the p99 tail. Here is the break-even math and the mitigation toolkit.
When No One Answers the Escalation: Human-in-the-Loop Is a Staffing Problem
Human-in-the-loop assumes a person answers the escalation. In production it is a queue with arrival rate, service time, and abandonment — and an unanswered escalation is worse than none.
When the Cheap Model Is the Expensive One
Routing traffic to a smaller model lowers cost per token but can raise cost per finished task. Here is where the savings leak back out — and how to measure it before you ship.
Who Gets Paged When the Agent Is Wrong: On-Call for Non-Deterministic Systems
Agent failures don't reproduce, don't roll back, and stay green on every infrastructure dashboard. Here is how to rewrite the runbook, the alerts, and on-call expectations for systems you can't single-step.
Who Owns the Idle Cost of an AI Feature
Provisioned throughput, reserved GPUs, and warm vector indexes bill whether or not traffic arrives. Idle cost grows because it falls in the org seam between product, infra, and finance — here is how to make the gap visible and owned.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
