Page 35
12 articlesThe Model Provider Webhook Surface You Forgot to Subscribe To
Major LLM providers broadcast incidents, deprecations, and account warnings through webhooks and emails — and most teams have those channels disabled. A look at the small integration that turns 'we found out from a customer' into 'we found out from the provider before our own monitors did.'
The Monday Morning AI Degradation Your Dashboard Treats As Noise
Most AI feature dashboards average away the one failure mode that costs real money — a weekly cyclic degradation that shows up only when you slice latency, cache hit rate, and retry counts by hour-of-day.
MTBF Is Dead When Your Agent Self-Heals
Self-healing agent runtimes have eaten the failure signal MTBF was built to count. Here is the metric set that replaces it: success-rate-after-recovery, recovery-cost-per-success, and recovery-attempt distribution per trace.
When Your Agents Disagree: Conflict Resolution Patterns for Parallel AI Systems
Parallel agents producing conflicting outputs is not an edge case — it's a guarantee at scale. Here are the patterns that prevent silent disagreements from shipping broken decisions.
Open-Weight Licenses Are a Compliance Minefield Your Team Hasn't Mapped
Open-weight model licenses propagate through fine-tune lineages, switch terms at scale, and ship audit-finding risk that surfaces years after the download. Why model provenance is engineering work, not legal's.
The Parallelism Trap in Agentic Pipelines: When Fan-Out Makes Latency Worse
Spawning parallel sub-agents looks like an obvious speedup, but hidden coordination overhead — context merging, deduplication, error aggregation — makes p99 latency worse even as p50 improves.
Per-User AI Quotas: The UX Layer Your Cost Dashboard Can't See
Engineering teams instrument total spend and per-tenant cost and call it a day. The user who hits a quota at 3pm Tuesday and gets a cryptic 429 never trusts the feature again.
Persona Drift in Long-Running Agent Sessions: Why Your Agent Forgets Who It Is
After 8–12 dialogue turns, agent persona self-consistency degrades by over 30%. Here's why transformer attention causes your agent to drift from its system prompt—and three production patterns that actually fix it.
Persona Overlays: When One Agent Needs Many Voices for Different Customer Cohorts
Shipping one agent persona to a cohort-spanning customer base costs renewals quietly. The fix is overlays, not forks — and slice-level evals that catch the regressions an aggregate score will hide.
The PRD for an AI Feature: Why Your Old Template Misses the Cliff
Deterministic PRDs have no field for what the model gets wrong, what eval score gates ship, or who owns the system prompt. Four sections that fix it.
The Pre-Launch Blast Radius Inventory: The Document Your Agent Team Forgot to Write
Most agent teams discover the absence of a blast radius inventory during their first incident. Here's the artifact, the columns it needs, and how to make it a CI merge gate so it stays accurate.
The Preprocessing Bottleneck That Kills AI Pipeline Throughput
Most teams profile the LLM call and declare victory. The real throughput killer is everything before the model sees a single token — parsing, chunking, embedding, and enrichment that quietly dominate end-to-end latency.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
