NVIDIA Releases Molt: A PyTorch Framework for Agentic RL in 8,600 Lines of Code

On August 1, 2026, NVIDIA released Molt, a PyTorch framework for agentic reinforcement learning developed by the NeMo team. Its defining feature is size: about 8,600 lines of RL code, 7 times less than verl, with performance comparable to the Megatron stack.

Why does code size matter to researchers?

Molt is designed so that a researcher can hold the entire codebase in their head, and an AI assistant can read and reason about all of it. In academic and corporate RL projects, every algorithm iteration requires changes to the trainer, the distributed backend, and the rollout plumbing, and each change costs time. Molt tackles this problem with compactness: 8,600 lines versus 62,000 for verl, 25,000 for slime, and 7,200 for OpenRLHF.

• Molt RL code size: ~8,600 lines (vs 62,000 for verl, 25,000 for slime, 7,200 for OpenRLHF)
• License: Apache 2.0, with ready-made Slurm scripts and a prebuilt container
• Hardware requirements: 16 NVIDIA H100 GPUs across two nodes (8 for training, 8 for rollout)
• Components: Ray (orchestration), vLLM (rollout), NVIDIA AutoModel + FSDP2 (training)
• Supported scenarios: multi-turn tool-use agents, code execution, vision-language environments, LLM-as-judge, on-policy distillation into a smaller model

How does Molt work under the hood?

Molt combines three components without forking any of them: Ray handles placement and asynchronous queues, vLLM runs the rollout, and NVIDIA AutoModel with FSDP2 handles training. Upstream updates arrive by changing the container pin, with no rebasing.

"The codebase should be compact enough for a researcher to hold in their head, and for an AI assistant to read and reason about in its entirety," says NVIDIA

NeMo's Molt technical documentation.

An agent in Molt is an ordinary Python program: the researcher points to a module containing an AgentRunner, and the reward function is written as standard code. Two modes are supported: Env (the framework drives the LLM loop in the style of Gymnasium) and ChatAgent (the developer drives it through the standard OpenAI or Anthropic SDK). The framework starts a loopback server that supports both wire protocols, and every request is decoded on the server side into the exact sequence of tokens.

Three correctness invariants shape the design: token identity, meaning the generated tokens define the trajectory, not a retokenized transcript; policy-version semantics, meaning the behavior policy's log-probabilities are stored at the token level; and forward consistency, meaning rollout and actor must run the same model. For MoE (mixture-of-experts) policies, Molt applies rollout routing replay: vLLM returns the expert IDs for each token, and the training pass replays them, eliminating routing discrepancies.

Who can use Molt today?

Molt is aimed at research groups with access to H100 or H200 clusters. According to NVIDIA, the framework's performance is statistically comparable to the Megatron stack, so researchers do not sacrifice speed for compactness. The target audience: frontier labs and funded AI startups working on model post-training; corporate research groups in finance, healthcare, and robotics with access to multi-node GPU clusters; and academic labs with H100/H200 hardware.

What this means

Molt lowers the cost of iteration in RL research: 8,600 transparent lines speed up algorithmic experiments, and support for standard SDKs means existing agent code can be trained without rewriting. NVIDIA is building an open stack for training agents, competing with verl and OpenRLHF not only on speed but also on ease of research.

Frequently asked questions

How many GPUs are needed to run Molt?

The standard recipe requires 16 NVIDIA H100 GPUs across two nodes: 8 handle training, and 8 handle generation (rollout).

How does Molt differ from verl and OpenRLHF?

Molt contains about 8,600 lines of RL code versus 62,000 for verl and 7,200 for OpenRLHF. None of its three components (Ray, vLLM, NVIDIA AutoModel) is forked: updates arrive by changing the Docker container pin.