Tian Pan
Software Engineer
The AI Feature You Should Not Have Shipped: A Task-Shape Checklist
Most demos work. A meaningful fraction of shipped AI features are still task-shape mismatched — stochastic engines wired into deterministic-required outputs. A pre-build checklist and the roadmap pathway you need to redirect ideas that are not model-shaped.
The AI Interview Has No Signal: Why Your Loop Doesn't Identify People Who Ship LLM Products
Standard engineering interview loops select for deterministic-systems skills and miss the cluster — eval design, cost intuition, prompt debugging, recovery-mindedness — that predicts who ships LLM products. The fix is loop redesign, not another bolted-on AI round.
The Five-Surface Triage Tree: An AI On-Call Playbook for Pages That Don't Fit Your Runbook
Pages that say the model started lying do not fit a runbook designed for restart the service. Here is the five-surface triage tree, freeze button, and replay harness that make AI on-call its own discipline.
The Batch-Tier Inference Question: When 50% Off Reshapes Your Architecture
Provider batch APIs cut inference cost in half but reshape the engineering contract: job-level idempotency, freshness boundaries, late-result observability, and a tier-aware decision matrix that reroutes 30–50% of LLM spend on workloads the user was never waiting on.
Build vs Buy for Guardrails: The Moderation API Is Now on Your Safety-Critical Path
A hosted moderation API turns your safety control into a synchronous external dependency — the build-vs-buy decision, fail-open vs fail-closed tradeoff, and integration discipline that keeps a vendor on the safety-critical path from owning your incident response.
Calibrated Abstention: The Capability Every Layer of Your LLM Stack Punishes
Every default in the LLM stack — pretraining, RLHF, judge LLMs, user feedback — pushes the model toward confident wrong answers. Calibrated abstention only ships if you build the eval, rubric, and UI that pay for it.
Cancel-Safe Agents: The Side Effects Your Stop Button Already Shipped
Stop is a UI affordance, not a system guarantee. A practitioner's playbook for cancel-safe agents: durable side-effect ledgers, scoped authorization, compensating actions, and what the cancel UI should actually display.
The Chargeback Ledger for Compound AI Systems
A field guide to attributing cost across compound AI systems — per-span ledgers, on-behalf-of headers, settlement-currency mismatches, and the political surface that decides who pays for tool calls.
Chat History Is a Database. Stop Treating It Like Scrollback.
Treating conversation history as scrollback is why agents lose the thread by turn 8 and why context bills scale superlinearly. The fix is to call it what it is — a read-heavy database — and design accordingly.
The Coding Agent Autonomy Curve: Reading Is Free, Merging Is Incident-Class
A single autonomy switch is the wrong abstraction for coding agents. Map each tool to a blast-radius tier, scale gates to the tier, and match agent velocity to your rollback velocity.
Contract Tests for LLM Tool Surfaces: When the Vendor Changes a Field and Your Agent Silently Adapts
When a vendor renames a tool response field, your agent doesn't crash — it adapts and ships a degraded answer. Why microservices contract testing has to migrate to the agent stack, and how to wire it in.
Counterfactual Logging: Log Enough Today to Replay Yesterday's Traffic Against Next Year's Model
Production LLM logs answer 'what did the model say' well and 'what did the model see' poorly — and that gap is what breaks model-migration evaluations months later. A practical schema for replayable traces.