Skip to main content

Blog

Insights, analysis, and updates from the AI agent economy. Browse by tag.

·tian

Your Internal Framework Is a Low-Resource Language

Coding agents write flawless React but hallucinate your in-house ORM. Why internal frameworks behave like low-resource languages, how to measure the competence cliff, and when to conform your stack to the model's training distribution.

insider
ai-engineering
coding-agents
llm
+2
·tian

Your Model Thinks Your Stack Is Two Years Old

AI-generated code that looks idiomatic but targets a two-year-old version of your stack is a bug class review can't catch. Treat model knowledge staleness as a dependency-management problem: docs injection keyed to your lockfile, lint rules as the feedback channel, and a staleness audit.

ai-engineering
coding-agents
developer-tools
llm
·tian

Your Prompts Have Foreign Keys

Rename a column and the compiler catches every consumer except your system prompt. Prompt-embedded schemas, tool signatures, and few-shot examples are undeclared dependencies — here's how to give them manifests, deploy-time generation, and CI-enforced referential integrity.

ai-engineering
prompt-engineering
llmops
reliability
·tian

Your Secrets Manager Ends Where the Context Window Begins

A credential that transits an LLM's context can't be un-leaked — it is copied into traces, prompt caches, memory stores, eval fixtures, and provider logs before rotation ever runs. Why the tool boundary is the only real chokepoint, and how short-lived credentials cap the blast radius.

ai-security
llm-agents
secrets-management
ai-engineering
·tian

Your Tool Schema Validates Types, Not Units

A refund tool that takes cents will get dollars, because JSON Schema validates shape, not meaning. How unit confusion slips past types, schemas, and evals — and the naming, boundary-assertion, and eval patterns that stop 100x invoices.

ai-agents
tool-calling
llm-engineering
reliability
·tian

Your Users Drift While Your Model Stands Still

Quality metrics sag six weeks after launch with no deploy in sight — because users adapt faster than models change. How to instrument input drift, cohort-split your metrics, and refresh evals on the timescale of user habituation.

ai-engineering
llmops
evals
monitoring
·tian

Exactly-Once Was Hard Before Your Agent Could Retry Itself

Idempotency keys assume the caller repeats its request byte-for-byte. An agent paraphrases its own request on retry, which breaks that assumption. Here is how to build tool-level dedup that holds when the caller is a language model.

ai-agents
distributed-systems
idempotency
reliability
+1
·tian

Model Collapse Starts in Your Own Data Lake

Logged LLM outputs quietly become next quarter's training data. How model collapse takes hold inside your own data lake — and the provenance tagging, quarantine zones, and tail evals that break the feedback loop.

ai-engineering
llm
data-engineering
fine-tuning
·tian

When the Clock Is a Tool: Agents, Time Zones, and the Bug That Only Happens at Midnight

LLMs have no internal clock, so agents drift confidently onto stale time — and it surfaces as skipped schedules, wrong time-zone math, and cache-busting timestamps. Here is how to supply the current moment as infrastructure.

insider
ai-agents
llm
time-zones
+2
·tian

Conway's Law Comes for Your Agent Fleet

Your multi-agent topology copies your org chart by default — silos, lossy handoffs, and central bottlenecks included. Here's when that coupling becomes a bug and how to draw agent boundaries from the problem instead.

insider
ai-agents
multi-agent-systems
software-architecture
+1
·tian

Temperature Zero Is Not Determinism: Reproducing the Incident You Can't Re-Run

Setting temperature to zero forces greedy decoding but does not make LLM inference reproducible. Batch-dependent GPU numerics, not the sampler, flip your tokens — and here is how to build a post-mortem for the run you can't re-run.

llm
reproducibility
inference
debugging
+1
·tian

The Hidden Ops Org Behind Your Evals: Annotator Economics

Every eval score inherits the quality of a human labeling pipeline you don't budget for. How annotator economics, throughput-driven quality decay, and LLM judges shape the numbers you ship.

evals
rlhf
data-annotation
llm-as-judge
+1
Showing 121–132 of 2314 posts
Prev11 / 193Next