Page 89
12 articlesThe Discovery Problem: Why Semantic Search Fails Browsing Users
Embedding-based retrieval optimizes for users who know what they want. It quietly fails everyone else — here's how to detect browsing intent and fix your ranking strategy.
Semantic Search as a Product: What Changes When Retrieval Understands Intent
Building user-facing semantic search is a different problem than building a RAG pipeline. Half the failures happen before any vector is touched — here's what breaks and how to fix it.
What Semantic Versioning Actually Means for AI Agents
Traditional semver breaks down when your service is non-deterministic. Here's how to version AI agents so downstream consumers don't get silently broken.
Your Team's Benchmarks Are Lying to Each Other: Shared Eval Infrastructure Contamination
Shared eval infrastructure silently corrupts benchmark results through cached completions, sequential run pollution, and prompt-state bleedover — and most teams never notice. Here are the technical and organizational controls that fix it.
The Sparse Reward Trap: Why Long-Horizon Agents Look Great in Demos and Break in Production
Sparse rewards make long-horizon agent training deceptively hard — agents pass demos and fail on edge cases. A practical breakdown of credit assignment failure, hindsight relabeling, step-level proxy rewards, and production training pipeline design.
Specification Gaming in Production AI Agents: When Your Agent Optimizes the Wrong Thing
How AI agents find unintended shortcuts that satisfy your metrics while violating your intent — and the detection signals and hardening patterns that stop it.
Speculative Decoding in Production: Free Tokens and Hidden Traps
Speculative decoding promises 2–3x LLM latency gains through draft-model-assisted generation. Here's what the benchmarks don't tell you about running it in production.
The Three Hidden Debts Killing Your AI System
Prompt debt, eval debt, and embedding debt are the three silent liabilities accumulating in every AI system. Here's how they interact and how to address each without a full rewrite.
Testing the Untestable: Integration Contracts for LLM-Powered APIs
Deterministic test suites fail for non-deterministic LLM outputs. Learn property-based testing, behavioral invariant assertions, and semantic snapshot strategies that give you regression coverage without brittleness.
The Testing Pyramid Inverts for AI: Why Unit Tests Are the Wrong Investment for LLM Features
How the classic testing pyramid breaks for LLM features, why prompt-level unit tests give false confidence, and the test allocation strategy that matches how AI failures actually distribute.
Tokens Are a Finite Resource: A Budget Allocation Framework for Complex Agents
How to treat the context window as a scarce compute budget with explicit allocation across system prompt, memory injection, tool results, and scratch space — and what happens to agent reliability when you run out mid-task.
Vector Store Access Control: The Row-Level Security Problem Most RAG Teams Skip
Multi-tenant RAG systems silently serve the wrong documents when chunk-level authorization isn't enforced at query time. Here's why post-retrieval filtering is security theater, and the patterns that actually work.
About Tian Pan
I'm Tian Pan, an engineer-founder focused on agentic engineering — building autonomous AI systems and scaling engineering teams. I write practical guides on system design, technical leadership, and shipping with AI agents. Previously an early engineer at Uber, Brex, and IoTeX.
