Page 14
12 articlesThe Production Logs Your Agent Cannot Read
Wiring an agent into your log stack gives it access, not understanding. The real integration work is teaching it what your data is called.
The Streaming Response That Committed Before the User Said Yes
Streaming UX inherits a reversibility contract that tool calls do not honor. Here is why the stop button can't unsend the email, and the framework changes that fix it.
When the Agent Asks Forgiveness Instead of Permission
The autonomy default that turns 'undo' into an aspirational verb — and the tool-layer, eval, and orchestrator discipline that keeps agents from acting first and apologizing later.
When the Intern Deploys an Agent on Day One
New-hire onboarding gives humans a tenure gradient; agents inherit production-grade permissions on day three. Inside the seam where security review, the tool registry, and the chargeback bucket meet — and why it has no owner yet.
When Two AI Features Compete for the Same Click
Two AI features can each pass their own A/B test and still make the product worse. Cross-feature attention competition is a portfolio problem no team-level dashboard will catch.
Your A/B Test Can't Tell Two Models Apart When the Metric Is Vibes
Click-through rate cannot distinguish a model your users love from one they tolerate. Before you trust an experiment to pick a model, prove the metric can detect a model you broke on purpose.
Your Agent's Audit Log Records Everything Except the Reason
A complete agent trace shows what happened and never why. Why observability is not explainability, why a logged chain-of-thought can be fiction, and how to capture decision rationale that survives a regulator.
The Chatbot That Inherited Your Support Team's Worst Habits
Fine-tuning on real customer-service transcripts transfers your team's tacit workflows along with the domain knowledge. Here is what your model actually learned, and the curation and evals that catch it.
Your Embeddings Don't Know the Contractor Was Off-Boarded
Your vector index is a permission cache nobody is invalidating. When access changes at the source, the embeddings keep answering as if nothing happened — and that is the leak nobody plans for.
Your Eval Set Only Has Problems You Already Solved
Your LLM eval score is climbing because survivorship bias filters out the users who never came back. Here is how to find the failures your suite cannot see.
From a Bug to a Behavior Rate: The AI Postmortem Without a Reproducer
An AI incident with no reproducer is not a debugging failure — it is the system telling you that one bad output is a sample from a distribution, not a deterministic bug. The postmortem has to change shape.
Your Model Router Is a Load Balancer That Cannot See the Load
A model router decides which model handles a query before the model has done anything — the difficulty signal it needs only exists in the answer. Why classifier accuracy lies, why misroutes look like mediocre quality instead of errors, and how to instrument the downstream signals that actually move with routing quality.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
