The Compiler Is the Cheapest Eval You'll Ever Run

Teams building with coding agents are spending real money on verification. Eval suites that replay curated tasks against every model upgrade. LLM judges that grade diffs. Sandboxed test runs that burn minutes of compute per iteration. All of it exists to answer one question: did the model write code that works?

Meanwhile, the cheapest eval most of these teams will ever have access to is sitting in their toolchain, and they configured it a decade ago without thinking about models at all. It's the compiler. A strict type checker is a free, instant, deterministic verifier that runs inside the agent loop on every single edit — and whether you have one is decided by your language choice, not your eval budget.

That reframing has an uncomfortable consequence. The stack decisions your team settled years ago — dynamic language for velocity, types optional, tests as the safety net — were optimized for human authors. When the author is a model, the tradeoffs reorder. The language your team is fastest in may no longer be the language your agents are safest in.

Every Check in the Loop Is an Eval — Price Them Like One

Strip away the terminology and an agent loop is a generate-verify cycle. The model proposes an edit; something checks it; the result feeds back as context for the next attempt. Everything sitting in the "check" position is an eval, whether you call it that or not. And evals differ on three axes that compound hard inside a loop: cost, latency, and determinism.

An LLM judge costs tokens, takes seconds, and gives you a probabilistic opinion. A test suite costs compute, takes seconds to minutes, and only covers the behaviors someone thought to encode. Runtime QA — deploy it and see — costs incidents. A type checker costs nothing per run, returns in milliseconds to seconds, and is fully deterministic: the same code produces the same verdict every time, with a file, a line, and a reason attached.

That last axis matters more than it looks. Agents iterate. A verifier that's wrong 5% of the time doesn't degrade your loop by 5% — it sends the agent chasing phantom failures or, worse, waves through broken code that a later, more expensive stage has to catch. Deterministic verifiers compound trust across iterations; probabilistic ones compound noise.

The research backs up how much of the failure surface this cheap eval covers. Work on type-constrained generation found that steering models with type-system information cut compilation errors by more than half and improved functional correctness by 3.5 to 5.5% across synthesis, translation, and repair tasks — and that held from 2B-parameter models up to 34B. The errors models make are disproportionately the kind a type checker catches instantly. In a dynamic language, those same mistakes don't disappear. They defer — to your test suite if you're lucky, to production if you're not.

What the Type Checker Knows That Your Test Suite Doesn't

A test suite verifies the behaviors someone thought to write down. A type checker verifies every typed interaction in the codebase, including the ones nobody anticipated — which is precisely where model-written code fails.

Human authors carry a mental model of the codebase between edits. A model has a context window. It will confidently call a function that was renamed last sprint, pass a naked string where a validated ID is expected, or handle three of the four variants your data can take. These aren't logic errors; they're coherence errors — mismatches between the edit and the reality of the surrounding code. Coherence is exactly what type systems check, exhaustively and for free.

There's a second, less obvious benefit: error messages are feedback, and typed toolchains produce dramatically better feedback. Practitioners consistently report the same pattern — hook an agent up to a language server, and it self-corrects through compiler errors in a few iterations, because "expected UserId, found string, at line 42" is an actionable instruction. Compare that to a dynamic language's runtime traceback, which tells you where the code died, not where the mistake was made — often several call frames and one serialization boundary apart. One error message localizes the fix; the other launches an investigation.

This is also why the "types slow you down" argument inverts with model authors. The historical cost of static typing was human keystrokes and human patience — annotations to write, generics to wrangle, compiler pedantry to satisfy. Models don't get impatient, and they emit annotations at the same speed as everything else. The cost side of the type-system tradeoff was always denominated in human effort. That currency just devalued.

The Stack Rankings You Settled in 2015 Just Reordered

A decade ago, choosing Python or Ruby or JavaScript over Java or Haskell was a defensible bet: developer iteration speed dominated, type ceremony was overhead, and the safety gap could be covered by tests and review. Every term in that equation assumed a human author. Change the author, and the equation reprices.

What appreciates when a model writes most of the code:

• Verification per keystroke. Strictness that was pedantic for humans is free signal for agents. Every constraint the compiler enforces is a check the model can't skip and you don't have to write.
• Build latency. The compile step now sits inside a loop that runs dozens of times per task. Go's near-instant builds mean the agent gets its verdict in seconds; a language whose builds take minutes turns every iteration into a coffee break the agent takes at your expense. Practitioners running agentic workflows report this as the difference between minutes-long and hour-long sessions on comparable tasks.
• Diagnostic quality. Rust's famously explanatory compiler errors were designed to teach humans. It turns out they steer models just as well — a precise diagnostic is a high-quality next prompt.
• One idiomatic way. Languages with enforced formatting and a single dominant style, like Go with gofmt, produce training corpora with less variance — and models generate more reliably when there are fewer plausible ways to write the same thing.

What depreciates:

• Terseness. Fewer keystrokes stopped mattering when the keystrokes became free.
• Runtime flexibility. Monkey-patching, dynamic dispatch, metaprogramming — power tools for expert humans, but they punch holes in exactly the static analysis your agents need. Every Any that flows through your codebase is surface area the cheapest eval can't see.
• "My team is fastest in X." Still relevant for review — humans read what agents write — but no longer the trump card, because the marginal line of code is increasingly not typed by the team.

None of this crowns a single winner. Rust maximizes constraint richness but pays in build times and a smaller training corpus. Go maximizes loop speed and uniformity but checks less per compile. TypeScript rides an enormous corpus and decent strictness but only if you actually enable it. The point is not that one language wins — it's that agent legibility is now a first-class column in the comparison table, and it wasn't even a row in 2015.

The Counterweights: Why You Shouldn't Rewrite in Haskell

Before this reads as a call to migrate everything to the most strictly typed language available, three forces push the other way — and they're strong.

Training data still dominates raw generation quality. Models generate the most probable code, and probability comes from exposure. Python and JavaScript's massive public corpora mean models know their idioms, their libraries, and their failure modes better than any boutique language's. A model writing OCaml gets a great compiler and a thin prior; a model writing Python gets a rich prior and a thin compiler. The sweet spot is where corpus size and static verification overlap — which goes a long way toward explaining the momentum behind TypeScript, Go, and Rust in agent-heavy shops.

Humans still review the code. Comprehension debt is real: if your team can't fluently read what the agent wrote, you've traded a verification problem for a worse one — plausible code nobody on staff can audit. A maximally expressive type system your team can't read fails the review step, and review is the one eval you can't automate away.

Verification lives in more layers than the compiler. A type checker validates coherence, not correctness. It will happily bless a well-typed function that computes the wrong thing. You still need tests for behavior, judges or review for intent, and monitoring for reality. The compiler is the cheapest eval, not the only one — its job is to make sure the expensive evals never see the failure modes a machine could have caught in milliseconds.

If You Can't Change Languages, Change the Dial Settings

Language choice is the headline decision, but most teams aren't choosing — they have a codebase. The good news is that "get a cheaper eval into the loop" is mostly a dial you can turn within your existing stack.

If you're on TypeScript, the distance between default and strict mode is the distance between decoration and verification. Turn on strict, ban escape-hatch types in CI, and type your module boundaries precisely — every interface the agent touches becomes self-checking. If you're on Python, a strict type checker like pyright or mypy plus disciplined annotations on public functions converts a fraction of your runtime failure surface into instant static feedback. It's not Rust, but it's deterministic signal at millisecond cost, which beats any judge you can buy.

Wherever you are, three moves generalize:

• Put the checker in the agent's loop, not just in CI. A diagnostic the agent sees immediately is a self-correction; the same diagnostic in a CI run twenty minutes later is a failed task. Language-server integration is the highest-leverage plumbing you can do for agent quality.
• Make invalid states unrepresentable at boundaries. Validated ID types instead of raw strings, exhaustive enums instead of stringly-typed variants, narrow interfaces at module edges. Each one converts a category of model mistake from "runtime surprise" into "compiler error with a location."
• Treat build speed as an eval-latency budget. Incremental compilation, module boundaries that localize rebuilds, caching — anything that shortens the edit-to-verdict gap multiplies across every iteration of every task your agents run.

And when you do get a greenfield call — a new service, a new tool, a component being carved out anyway — weigh the compiler as part of the eval budget. A language that verifies more per iteration, faster, is infrastructure for every agent task you'll run on that codebase for years.

The industry spent the last two years learning that agent quality is mostly a verification problem, then set about building verification out of the most expensive materials available: judges, sandboxes, human review. The deterministic checker in your toolchain does a meaningful share of that job for free — at millisecond latency, with perfect consistency, on every edit.

Language choice used to be about how fast your team could write code. It's becoming about how cheaply your tools can prove code wrong. The teams that internalize that will run more iterations, catch failures earlier, and ship agent-written code with fewer surprises than the teams still paying token prices for what a compiler does for free.