NVIDIA Vera Doubles Modal Sandboxes Speed: Up to 4,281 Tasks per Minute on 150 Containers

Processors are usually debated in terms of single-thread speed. But if you are running hundreds of AI-agent sandboxes on a single machine, that is not the right metric. The Modal team got its first server with the Arm-based NVIDIA Vera processor and showed why behavior under load matters more.

Why Sandboxes Are a Special Case

An agent's sandbox sits idle most of the time, waiting for an LLM response. Then suddenly an npm install or a heavy grep happens, and CPU load jumps tenfold. It is impossible to know in advance what the LLM will send for execution, so resource sizing has to be guesswork.

Modal allows bursting: you reserve capacity for the typical load (P50) rather than the worst moment, and peaks are absorbed by headroom on the host. This saves customers money, but it amplifies the noisy-neighbor problem: one tenant slows down another.

Latency is expensive here. Background agents such as Ramp Inspect run faster, which makes users happier and consumes less CPU time. In RL training, the synchronous scheme is still more common: the GPU waits until the whole fan of rollouts in sandboxes finishes. A second of CPU time is effectively a second of GPU idle time.

How It Was Measured

There is no ready-made benchmark for agentic workloads, so Modal built its own: Agentless Agent Bench. It consists of 50 reference solutions from Terminal-Bench with their checks, from which all network calls were removed. What remains are purely CPU-bound operations: compilation, file search, git, device emulation. All of it runs in parallel Docker containers.

The first test, single tasks on an idle machine, disappointed the authors themselves. Vera was often better, but not always and not by a wide margin.

The second test, throughput at 50, 100 and 150 containers, painted a different picture. On a single logical core, Vera completes 86 tasks per minute versus 51 for Intel GNR and 59 for AMD Zen 5. On 79 cores it reaches 4,238 tasks, with a peak of about 4,281 at 99 cores. Intel GNR plateaus at around 2,500, AMD Zen 5 at around 1,500. Graviton 5 reaches roughly 3,470 on 96 cores.

A False Trade-off

Usually, the more densely you pack tasks, the higher the throughput, but the worse the latency of each one. Modal's scheduler is constantly tempted to repack hosts. According to the authors, this effect is weaker on Vera: under high load the chip holds up better than its competitors.

Limitations and Takeaway

This is a benchmark by the vendor-consumer itself, not an independent test, and it is synthetic: network calls have been cut out. The charts, judging by the data, are not smooth everywhere: for example, Intel GNR in a VM shows a dip at 124 cores. Moreover, Vera is not yet in production at Modal. Supporting Arm will require changing the infrastructure and the image build system, since the company has always run only on x86.

What practitioners should do. If you are building a platform for agents or RL rollouts, do not choose hardware based on single-thread tests. Run your own mix of real tasks under dense packing and count the cost of a completed task, not the price per core. Also check that your images and dependencies build for arm64: that migration will be the main line item in your costs.