Page 69
12 articlesAgent Fleet Observability: Monitoring 1,000 Concurrent Agent Runs Without Dashboard Blindness
Individual span trees per agent run collapse at fleet scale. Here are the fleet-level signals, sampling strategies, and behavioral fingerprinting techniques that actually work when you're running hundreds of concurrent agents.
Agent Identity and Least-Privilege Authorization: The Security Footgun Your AI Team Is Ignoring
When your AI agent calls internal APIs, whose identity does it present? Most teams give agents a broad service account token and move on. Here's why that's a security footgun and what production-grade agent authorization actually looks like.
The Agent Loading State Problem: Designing for the 45-Second UX Abyss
Users abandon silent UIs at ten seconds, but modern agents run thirty to one hundred twenty. The gap is a design surface most teams still fill with a spinner — here is what to ship instead.
Your Agent Traces Are Lying: Cardinality, Sampling, and Span Hierarchies for LLM Agents
Distributed tracing was designed for ~10 spans per request. A single agent run can produce hundreds, and default OpenTelemetry configurations systematically undercount the work. Here's the span hierarchy, tail sampling policy, and payload handling that survive production agent workloads.
Agentic Task Complexity Estimation: Budget Tokens Before You Execute
LLM agents commit resources before knowing how deep a task runs. Here's the complexity estimation layer — tiered routing, budget-tracker injection, plan template caching, and DAG-based decomposition — that prevents irreversible early mistakes and makes agent costs predictable.
When Your AI Agent Consumes from Kafka: The Design Assumptions That Break
Running AI agents on message queues breaks the assumptions baked into queue semantics. Here's how idempotency, ordering, and backpressure work differently when your consumer is stochastic.
AI-Assisted Incident Response: How LLMs Change the SRE Playbook Without Replacing It
AI copilots in on-call workflows can surface correlated signals and draft runbook actions—but they introduce failure modes traditional SREs aren't trained to catch. A practical guide to integrating LLMs into incident response without making outages worse.
The AI Capability Ratchet: How One Smart Feature Breaks Your Entire Product
Shipping one impressive AI feature permanently raises user expectations for every other feature in your product — including ones you haven't touched. Here's the mechanism, real examples, and how to manage the expectation debt before it hits your support queue.
The AI Dependency Footprint: When Every Feature Adds a New Infrastructure Owner
Every AI feature you ship introduces new infrastructure dependencies — vector databases, embedding models, eval frameworks, GPU serving layers. The problem isn't the dependencies themselves. It's that nobody owns them.
AI Feature Decommissioning Forensics: What Dead Features Teach That Successful Ones Cannot
The AI features your company quietly killed contain the failure patterns your next launch will hit. A forensic template, a leading-indicator catalog, and how to read the evidence dead features leave behind.
The AI Incident Severity Taxonomy: When Is a Hallucination a Sev-0?
Traditional severity classification breaks for probabilistic AI systems. A multidimensional framework for classifying AI incidents — beyond binary broken/working to capture scope, reversibility, and compounding damage.
AI On-Call Psychology: Rebuilding Operator Intuition for Non-Deterministic Alerts
On-call for AI systems breaks standard SRE intuition. A practical taxonomy, rotation design, and training curriculum for operating stochastic production systems without burning out the team or missing real regressions.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
