Skip to main content

Your Tool Schema Validates Types, Not Units

· 11 min read
Tian Pan
Software Engineer

A refund agent processes a customer request for $42. The tool call it emits is refund(amount: 4200) — wait, is that right? If the backend stores money in cents, it's exactly right. If the backend stores dollars, the customer just got a $4,200 refund. Both calls are syntactically perfect. Both pass JSON Schema validation in microseconds. The schema says amount is a number, and 4200 is unimpeachably a number.

This is the bug class that unit tests, schema validators, and most agent evals all sail past: unit confusion at the tool-call boundary. The model pattern-matches magnitudes from its training data — money in dollars, time in seconds, weight in whatever the surrounding prose implied — while your tool expects cents, milliseconds, and kilograms. Nothing in the type system objects. The invoice is just off by 100x.

Engineering has been here before. The Mars Climate Orbiter disintegrated in 1999 because one team's software emitted thruster impulse in pound-force seconds while the other team's software expected newton-seconds. The Gimli Glider ran out of fuel mid-flight in 1983 because a fueling calculation confused pounds and kilograms. In both cases, every individual number was locally plausible, every interface contract was satisfied at the type level, and the error only became visible when the physical world disagreed.

The new twist is that the caller is now a language model, and language models make this mistake by default, not by accident. Ask yourself what amount: 100 means in the training corpus. In millions of Stripe integration examples it means one dollar, because Stripe amounts are integers in the smallest currency unit. In millions of other snippets it means a hundred dollars, because the variable was a float. The model has seen both conventions thousands of times and will pick whichever the local context nudges it toward. Your API's convention is one vote among many.

Types Constrain Shape; Units Constrain Meaning

JSON Schema is genuinely useful at the tool boundary. It catches missing required fields, strings where numbers should be, malformed enums, hallucinated parameter names. Constrained decoding and structured-output modes have made "the model emitted invalid JSON" a mostly solved problem.

But a schema constrains the shape of data, not its meaning. {"type": "number"} accepts 42 and 4200 with equal enthusiasm. Even minimum and maximum bounds rarely help, because the legitimate range of a dollars-denominated field and a cents-denominated field overlap almost entirely — a $500 refund and a 500-cent refund are both routine values. The two interpretations are only distinguishable by intent, and intent is exactly what the schema doesn't encode.

This is why unit confusion is uniquely resistant to the standard defenses:

  • Type checking passes. Cents and dollars are both numbers. Seconds and milliseconds are both numbers.
  • Range validation passes. 4200 cents and 4200 dollars are both within any sane bounds for amount.
  • The model is confident. It isn't hedging or asking for clarification; it has seen this exact call shape before, just with a different unit convention attached.
  • The output looks right in review. A human skimming the tool call sees a plausible number next to a plausible field name.

Strongly typed languages solved this decades ago with dimensional types — F#'s units of measure, or the quantity libraries in Rust and Haskell, make Cents + Dollars a compile error. But the tool-call boundary is a JSON gap between two type systems: your backend's types end where the schema begins, and the model's "types" are statistical associations from training data. The boundary itself is untyped in the dimension that matters.

Why the Model Guesses Wrong, and When

The model isn't being careless; it's doing what it always does — inferring the most probable completion given context. Unit errors cluster in predictable places:

Money. The dollars-versus-cents split is the classic, but it's worse than binary. Stripe-style APIs use the smallest currency unit, which is cents for USD but whole yen for JPY, because yen is a zero-decimal currency. An agent that correctly learned "multiply by 100 for Stripe" will silently overcharge Japanese customers by 100x. Currency handling has edge cases that trip up human developers constantly; the model inherits every one of those confusions, weighted by their frequency online.

Time. Is timeout: 30 thirty seconds or thirty milliseconds? Unix timestamps come in seconds (classic), milliseconds (JavaScript), and microseconds (some databases), and all three appear abundantly in training data. A model asked to schedule something "in five minutes" might emit 300, 300000, or an ISO 8601 string, depending on which convention the surrounding conversation resembles.

Physical quantities. Weight in kilograms versus pounds, distance in kilometers versus miles, temperature in Celsius versus Fahrenheit. A logistics tool that takes weight will get pounds from a model that just processed a US shipping conversation and kilograms from one that just read a European invoice.

Percentages and ratios. Is a 15% discount 0.15 or 15? Both conventions are everywhere. This one is nasty because the error factor is only 100x in one direction and the values almost never overlap with sanity bounds — a discount: 15 meaning 1500% will often sail through.

The common thread: the model's prior is set by global convention frequency, but your API's convention is a local fact. Whenever the local fact is the minority convention — and cents, milliseconds, and minor currency units usually are minority conventions in prose — the model's default guess is wrong. The prompt might say "amounts are in cents" once, three thousand tokens ago; the training prior says "dollars" a million times.

Put the Unit in the Name

The single highest-leverage fix costs nothing: rename the parameter so the unit is inside the identifier the model must literally type.

amount becomes amount_cents. timeout becomes timeout_ms. weight becomes weight_kg. discount becomes discount_percent with an explicit 0–100 range in the description.

This works for a reason worth understanding. A description field saying "amount, in cents" is context the model read once and may have deprioritized by the time it generates the call. But the parameter name is regenerated inside the tool call itself — the model has to emit the token amount_cents immediately before emitting the value. The unit hint arrives at the exact moment of decision, at zero distance from the number being chosen. It converts a memory problem into a reading problem. Anthropic's tool-design guidance makes the same argument for user_id over user: ambiguous names invite the model to fill the gap with whatever its prior suggests.

A few refinements compound the effect:

  • Use quantity-typed formats where they exist. ISO 8601 durations (PT5M) and RFC 3339 timestamps are self-describing; there is no unit ambiguity in 2026-07-02T14:30:00Z. A string the model can't misinterpret beats an integer it can.
  • Make money a structured object. {"amount_minor": 4200, "currency": "USD"} forces the currency decision to be explicit and lets your backend apply the correct minor-unit rules per currency, including the zero-decimal ones.
  • Never accept bare floats for money. An integer minor-unit field can't carry the 42.00-versus-4200 confusion that a float invites, and it dodges floating-point rounding besides.
  • Kill unit-ambiguous defaults. A default of timeout: 30 is a landmine — the model will copy the pattern and reinterpret the unit. timeout_ms: 30000 teaches the convention by example.

Assert Plausibility at the Boundary

Naming shrinks the error rate; it doesn't zero it. The second layer is boundary assertions — checks that run inside the tool, after the model has committed to a value, asking not "is this a number?" but "is this number believable?"

Schema validation asks whether input is well-formed. Plausibility assertion asks whether it's likely. The difference matters because unit errors produce values that are wrong by a suspicious factor — 100x, 1000x — which is a detectable signature:

  • Cross-field consistency. A refund amount should not exceed the original charge. A scheduled time should not be 40 years in the future (a seconds-vs-milliseconds epoch bug produces exactly that). A package weight should be compatible with its declared shipping class. These checks live in your domain logic and no model error survives them silently.
  • Magnitude fences with escalation, not just rejection. If 99% of refunds are under $200, a $4,200 refund shouldn't be impossible — it should require a confirmation step or a human approval. Tiered thresholds turn the long tail of "large but legitimate" into a review queue instead of either an outage or an incident.
  • Round-factor suspicion. When a value is almost exactly 100x or 1000x the expected distribution for that field, the specific hypothesis "unit confusion" deserves its own error message.

That last point is where boundary assertions earn double duty: the rejection message goes back into the model's context. "Invalid amount" teaches the model nothing. "amount_cents=4200 would refund $42.00, but the customer requested $42 — did you pass dollars instead of cents? This field takes integer cents" lets the agent repair the call on the next attempt. At the tool boundary, your error messages are the only teacher the model gets mid-task, so make them state the unit, show the interpretation, and name the suspected confusion.

Why Your Evals Never Catch This

Here's the uncomfortable part: you can have a green eval dashboard and still ship 100x invoices, because unit errors are nearly invisible to standard agent evaluation.

Most tool-use evals check three things: did the model pick the right tool, did the arguments validate against the schema, and did the task complete. A unit-confused call passes all three. The tool was right, the schema accepted the number, and the task "completed" — a refund was issued, just the wrong one. An LLM-as-judge grader reviewing the transcript sees a plausible number next to a plausible field and approves, because the judge has the same training prior that caused the error. Grading dollars-as-cents with a model that also thinks in dollars is asking the fox to audit the henhouse.

Catching this class requires unit-aware assertions that most eval suites don't have:

  • Exact-value oracles, not plausibility grading. The test case must know the correct answer is 4200 cents, deterministically, and fail anything else — including 42, which looks more natural.
  • Cross-unit trap cases. Seed evals with tasks whose surrounding context uses the wrong convention on purpose: a user who says "refund my $42" against a cents API, a European weight against a pounds API, a JPY charge against your cents-assuming multiplication. If your eval set never crosses conventions, it's testing the happy path where the prior and the API agree.
  • Distribution monitoring in production. Unit bugs at scale produce bimodal value distributions — one cluster at the correct magnitude, one at 100x. That signature is trivially detectable in tool-call telemetry and invisible in any single trace. Log the arguments, not just the outcomes.

The reason to invest here is that unit errors have the worst severity-to-visibility ratio in the tool-call bug taxonomy. A hallucinated parameter name fails loudly and immediately. A unit error succeeds quietly and costs real money, and by the time finance notices the anomaly, the agent has repeated it a few hundred times.

Treat Every Bare Number as a Question

The discipline that falls out of all this is simple to state: at an agent-facing boundary, a bare numeric parameter is an unanswered question, and the model will answer it with a guess weighted by other people's codebases.

So audit your tool signatures the way you'd audit them for security. Every number field gets one of three treatments: the unit goes into the name (_cents, _ms, _kg), the value becomes a self-describing format (ISO 8601, RFC 3339, structured money), or the field gets a boundary assertion with an escalation path. Fields that touch money, time, or physical quantities get all three. Then add the trap cases to your evals — the ones where the conversation's convention and the API's convention disagree — because that disagreement is the actual production condition, not the edge case.

Twenty-five years ago, unit confusion between two teams' software destroyed a spacecraft, and the industry's answer was better interface contracts. The agent era reintroduces the same failure with a twist: one side of the interface is now a statistical model of how everyone else's software works. Your contract isn't just with a caller anymore — it's with the entire training corpus's idea of what amount means. Write your schemas like the corpus is wrong about you, because it probably is.

References:Let's stay in touch and Follow me for more thoughts and updates