Modal advances gVisor: isolating AI workloads with sandbox startup in 25 ms

If your AI agent can run Python, install packages, and execute commands, you are already building a service that runs other people’s code. A standard container helps separate environments but leaves applications sharing the Linux kernel. A bug in that kernel can turn a failed script run into a problem for the entire machine. Modal suggests taking a look at gVisor: according to the company, a sandbox starts in around 25 ms with roughly 12 MB of additional memory overhead.

The news comes amid a change in project governance. According to Modal’s post, Google is transferring gVisor, including its name and trademarks, to the CNCF, while technical leadership is passing to the maintainers. Modal, Ant Group, and Tines have committed to joining their ranks. For infrastructure developers, the most important issue is continued support for GPUs, networking, and compatibility with familiar tools.

What happens between the script and the server

Imagine a service where an agent downloads a user’s repository and runs tests. It may contain unknown dependencies, installation scripts, and arbitrary commands. In a standard container, their system calls are handled by the host kernel—the same one used by neighboring workloads.

gVisor places its own software kernel, Sentry, written in Go, between the application and the host. It implements the Linux system call interface itself and accesses the real kernel through a limited set of operations. The program gets a familiar environment, but its access to the server passes through an additional check.

Sentry itself is also constrained by Linux mechanisms: namespaces, resource limits, and seccomp filters. These provide several layers of protection, rather than a promise of absolute security. Hardware virtualization is not required, so the approach also works on cloud virtual machines where nested virtualization is unavailable.

Modal says it launches more than 40 million gVisor containers per day. Workloads include inference, training across multiple machines, and agent sandboxes. This is one provider’s experience, but it shows that the technology is used well beyond demonstration setups.

GPUs inside the sandbox: where things get complicated

Isolating a script is not enough: it often needs CUDA, multiple GPUs, and fast data transfers. For NVIDIA, gVisor provides nvproxy. It parses and validates calls to the driver interface while preserving native compute execution and GPU memory operations.

Modal describes changes submitted upstream to support new drivers, NVLink, CUDA MPS, Nsight profilers, and monitoring. The practical benefit is straightforward: moving a workload into a sandbox causes less disruption to an engineer’s usual workflow. Still, support for a particular GPU, driver, and tool needs to be checked in your own environment.

For training across multiple servers, the company added rdmaproxy. It validates RDMA control operations while leaving data transfers a direct path through the hardware. In NCCL all-reduce tests, Modal achieved around 99% of native throughput. This is the result of a specific benchmark: it does not mean that every training workload loses only 1% of its speed.

Another notable result concerns transferring data packets through shared memory: after a change to the memory allocator, throughput in one test rose from 2.9 to 22 GB/s. That increase shows how much performance depends on the workload and the implementation of individual operations.

What to test in your own environment

For a team running user or agent code, gVisor warrants a dedicated trial. Choose a real workload: installing dependencies, traversing a large repository, running tests, or reading a dataset. Measure startup time, memory usage, file operations, and compatibility with the libraries you need. Fast sandbox startup alone does not guarantee a fast product response.

For inference, state snapshots are an interesting option: you can save an already initialized process and restore it on another machine. Modal reports around 670,000 container restores from snapshots every day. However, snapshots of programs using multiple GPUs are still in development; GPUDirect Storage and starting large inference servers in under a minute are also described as planned features.

The key takeaway for a founder or product manager is that executing generated code requires a separate decision about isolation. Moving gVisor to maintainer-led governance broadens the group of companies responsible for the project. Adoption decisions should be based on your own testing of security, compatibility, and execution costs.