The Warm Sandbox Pool: Infrastructure Economics When Every Agent Task Gets Its Own Machine

If you run coding agents at any real scale, you own a fleet of ephemeral virtual machines. You may not have signed up for that. It happened the moment you decided — correctly — that untrusted, model-generated code should never execute inside your application's trust boundary. Every task gets its own sandbox, every sandbox is a microVM or hardened container, and suddenly the platform team that thought it was building "an agent product" is operating something that looks suspiciously like a miniature AWS Lambda: pool warming, snapshot pipelines, bin-packing schedulers, and a reaper process for the environments nobody came back for.

The trap is assuming your container orchestration instincts transfer cleanly. Some do. But Kubernetes grew up scheduling long-lived, homogeneous services, and agent sandboxes are the opposite: short-lived, wildly heterogeneous, and created at rates that make a deployment rollout look leisurely. The teams that struggle are the ones that treat sandbox infrastructure as "just containers with extra steps." The interesting engineering — and almost all of the cost — lives in four problems: cold starts, filesystem state, packing density, and abandonment.

Why Every Task Gets Its Own Machine

The per-task isolation model won for a simple reason: an agent is an untrusted code generator wired to an execution loop. OpenAI's Codex runs each task in its own cloud container, preloaded with the repository, destroyed when the task completes, with network access off by default. Anthropic's cloud offerings and every serious sandbox vendor — E2B, Daytona, Modal, Fly.io — converged on the same shape. Session-scoped, isolated, disposable.

Shared environments fail in two directions at once. Security: model-generated code can rm -rf, exfiltrate credentials, or corrupt state that a later task depends on. Correctness: two agents installing conflicting dependency versions into a shared environment produce failures no one can reproduce. Per-task isolation eliminates both classes of problems by construction, which is why nobody serious argues against it anymore.

But it moves the cost somewhere else. A service handling 10,000 agent tasks a day is now booting 10,000 machines a day. The unit economics of that fleet — how fast machines start, how much memory each one idles at, how tightly you pack them — become product-level concerns. A sandbox that takes 30 seconds to provision is a 30-second tax on every task, and agent loops that spin up sandboxes for a single tool call pay it dozens of times per session.

Cold Starts: The Pool Is a Bet on Your Traffic

The raw numbers explain the entire warm-pool industry. A Firecracker microVM cold boot — minimal kernel, userspace init, network setup — lands around 110–200ms at the median, with p99s past 300ms. That's before you've cloned a repo, installed dependencies, or started a language server. A realistic "cold" agent environment, built from scratch, is tens of seconds. Restoring the same environment from a memory snapshot takes single-digit milliseconds for the VM itself; Lambda SnapStart's production numbers are around 3ms at p50. Providers quote end-to-end figures like Daytona's sub-90ms sandbox creation from warm pools and Fly.io's ~300ms checkpoint/restore for its agent-focused machines.

That's a three-orders-of-magnitude spread between "fully cold" and "restored from snapshot," and the pool is how you buy the fast end. You pre-boot N environments to a checkpoint — OS up, runtime installed, common dependencies present — then either keep them running (paying for idle RAM) or snapshot and park them (paying for storage and a restore penalty).

The sizing problem is where it gets interesting, because it's a bet on traffic you don't control:

• Too small, and burst traffic falls through to cold path. Cold-start latency compounds under concurrency — a queue of tasks each waiting 30 seconds isn't a latency problem, it's an outage.
• Too large, and you're paying for idle memory around the clock. Unlike CPU, RAM doesn't oversubscribe gracefully; a warm-but-idle pool is nearly pure cost.
• Too generic, and warmth doesn't help. A pre-booted Ubuntu image saves you 200ms of kernel boot but not the 40 seconds of npm install that dominates the actual start time.

That last point is the one teams miss. The valuable snapshot isn't "a booted VM" — it's "a booted VM with this repo's dependency tree already installed." Which means your warm pool isn't one pool; it's a cache keyed by environment fingerprint, with all the cache-management problems that implies: invalidation when the lockfile changes, popularity-based retention, and a long tail of environments that are never worth pre-warming. Suspend/resume pricing models — where a parked machine bills like storage instead of compute, as Fly's suspended machines do — exist precisely to make a large, mostly-idle cache affordable.

Filesystem State: Copy-on-Write Is the Whole Game

The second cost center is getting a working tree into the sandbox. Naive approach: git clone plus dependency install on every task. For a large monorepo that's minutes, and it's minutes of wasted work because 10,000 tasks a day are materializing nearly identical filesystems.

Loading…