Page 51
12 articlesReranking Is the Real Work: Why Your Retrieval System's Bottleneck Is Never the Index
Most teams over-invest in vector index tuning and under-invest in the reranking layer. The ranking step — not the index — determines whether your RAG system delivers or hallucinates.
The Shadow AI Problem: Why Engineers Bypass Your Official AI Platform and What to Do About It
Nearly half of engineers use AI tools their employers haven't sanctioned. Blocking endpoints makes the problem worse. Here's why shadow AI is a platform design failure — and how to fix it.
The Stakeholder Explanation Layer: Building AI Transparency That Regulators and Executives Actually Accept
Most AI systems can explain themselves to engineers. Almost none can explain themselves to regulators, executives, or legal teams. Here's the architectural layer that bridges that gap — and why it's fundamentally an observability problem, not an interpretability one.
The System Prompt Is a Software Interface, Not a Config String
Most teams treat system prompts like config strings — unversioned, untested, and one bad edit away from silent failure. Applying software interface design principles to prompts is what makes LLM systems maintainable at scale.
Thinking Budgets: When Extended Reasoning Models Actually Make Economic Sense
Extended reasoning models can inflate inference costs 5–30x — or deliver genuine quality jumps on hard tasks. The difference comes down to routing: which queries actually warrant thinking tokens, how to set budget ceilings, and how to catch over-thinking before it hits your invoice.
Timeout-Aware Agent Design: How to Deliver Partial Results Instead of Silent Failure
Most AI agents fail completely when they hit a deadline. Here's how to design agents that surface the best available result instead of returning nothing.
Token Economics for AI-Powered API Products: Pricing What You Cannot Predict
Per-seat and per-query pricing both collapse under variable LLM costs. Here's how to think about hybrid pricing, token budgets, and margin math for AI-powered API products.
Tool Discovery at Scale: Why Embedding-Only Retrieval Fails Past 20 Tools
Flat embedding retrieval breaks as AI agent tool inventories grow past 20. Here's why it fails, what structured capability metadata looks like, and how hierarchical routers solve the problem that better descriptions cannot.
Tool Output Schema Design: How Your Tool Responses Shape Agent Reasoning
Poor tool output schemas cause agent reasoning failures that look like model problems. A practical guide to field naming, nullability, verbosity, error design, and output contract testing for LLM tool interfaces.
The Two-Sided Cost of AI Content Filters: Why Over-Refusal Is a Business Problem Too
AI content filters are routinely tuned to minimize false negatives while ignoring false positives — but blocking legitimate users has measurable business costs too. Here's how to calibrate both error types properly.
Vector DB Sharding: Why HNSW Breaks at Partition Boundaries and What to Do About It
HNSW graphs resist partitioning in ways that cause silent recall degradation at scale. Here's why it breaks, what the quality loss looks like in practice, and the operational patterns teams use to recover accuracy when they've outgrown a single node.
The Accountability Transfer Problem: Why AI Gets Blamed for Decisions It Was Never Designed to Make Alone
When AI-assisted decisions go wrong, organizations often blame 'the AI' — but the AI approved nothing. Here's why accountability transfer happens in production systems and the design patterns that prevent it.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
