Page 57
12 articlesCohort-Aware Fine-Tuning: When One Model Isn't Enough But Per-User Is Too Much
Most fine-tuning lives at two extremes: one model for everyone, or one per customer. The middle — three to eight cohort-aware fine-tunes — is where the leverage hides.
The 90-Second Cold Start for Production Agents: When the LLM Isn't the Slow Part
Production agents spend 60–120 seconds on a cold start before the model even runs. The fix isn't a faster TTFT — it's treating cold-start latency as a first-class SLO with warm pools, snapshot/restore, lazy tool registration, and CI gates.
The Conversation Reset Button: UX Patterns for Starting Over Without Losing Your Artifacts
Most chat products bind conversation history and artifacts to the same lifetime, so hitting reset destroys the user's work along with the poisoned context. Decoupling them turns reset into a safe, recoverable action.
Your CS Team Built a Shadow Agent. That's Your Roadmap.
Your CS team's unsanctioned Slack bot is not a security incident. It is the most accurate AI roadmap signal your engineering org will get this year — and the four product questions it has already answered.
The Disable Switch Is the Real Product: Designing the Non-AI Fallback Path
Most AI features ship with a disable switch that returns an error. Treat the off-state as a real product, with capability-level flags, deterministic fallbacks, and the same eval discipline as the on-state.
Disconnected Agent Mode: Designing for the Network You Don't Have
Most AI agent stacks assume the network is always there. That assumption breaks on planes, in basements, and behind flaky Wi-Fi. Here is how disconnected-first architecture actually works.
Distillation Is a Product Decision, Not a Research Artifact
Distillation is a product decision about which capabilities you sacrifice to unlock a cost floor and a latency floor — not a research-team optimization. A frontier-model feature and its distilled variant are two products, not two implementations of one.
Personalization Belongs in a Dotfile, Not a Vector Store
Most agent personalization is tone, formatting, default tools, and project context — declarative settings the dotfile pattern solved decades ago. Reach for fine-tuning and persistent memory only after you have exhausted config.
Eval-as-Code: When Your Release Gate Is a Notebook on Someone's Laptop
The number deciding whether your model ships is a notebook on someone's laptop. Treat the eval suite like the production system it has quietly become — version it, gate it, give it SLOs.
The Eval Automation Trap: When Your Pipeline Drifts Away From What Users Actually Want
Automated eval pipelines can show improving accuracy while user satisfaction quietly drops. Here's how the drift happens and how to catch it before it compounds.
The Eval Migration Tax: Why a Prompt Schema Change Wrecks 800 Test Cases
Schema changes to AI prompts routinely break hundreds of test cases that have nothing to do with the change. Most teams treat eval suites as static fixtures instead of versioned data — and pay a hidden tax on every release.
The Fallback Cascade: Why Your AI Feature Needs Five Failure Modes, Not One
AI features that treat model failures as binary pass/fail events are one outage away from disaster. A five-level fallback cascade — from frontier model to human escalation — keeps your feature functional when individual levels break.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
