NVIDIA выпустила Molt: PyTorch-фреймворк для агентного RL в 8600 строк кода

NVIDIA released Molt on August 1, 2026 — a framework for agentic reinforcement learning on PyTorch, developed by the NeMo team. Key characteristic: approximately 8,600 lines of RL code, which is 7 times less than verl, while delivering performance comparable to the Megatron stack.

Why code size matters to researchers?

Molt is designed so that a researcher can keep the entire codebase in their head, and an AI assistant can read and comprehend it in full. In academic and enterprise RL projects, each algorithm iteration requires changes to the trainer, the distributed backend, and the rollout wrappers — and every change is costly in terms of time. Molt solves this problem through compactness: 8,600 lines versus 62,000 in verl, 25,000 in slime, and 7,200 in OpenRLHF.

• Molt RL code size: ~8,600 lines (vs 62,000 in verl, 25,000 in slime, 7,200 in OpenRLHF)
• License: Apache 2.0, with ready-made Slurm scripts and a pre-built container
• Hardware requirements: 16 NVIDIA H100 GPUs across two nodes (8 for training, 8 for rollout)
• Components: Ray (orchestration), vLLM (rollout), NVIDIA AutoModel + FSDP2 (training)
• Supported scenarios: multi-turn tool-use agents, code-execution, vision-language environments, LLM-as-judge, on-policy distillation to a smaller model

How is Molt structured internally?

Molt connects three components without forks: Ray handles placement and async queues, vLLM performs the rollout, NVIDIA AutoModel with FSDP2 handles training. Upstream updates arrive via a container pin change — no rebase required.

"The codebase should be compact enough for a researcher to keep in their head, and for an AI assistant to read and comprehend it in full," — states the

Molt technical documentation from NVIDIA NeMo.

An agent in Molt is an ordinary Python program: the researcher specifies the module with AgentRunner, and the reward function is written in standard code. Two modes are supported: Env (the framework manages the LLM loop in Gymnasium style) and ChatAgent (the developer controls it via the standard OpenAI or Anthropic SDK). The framework runs a loopback server with support for both wire protocols, and each request is decoded server-side into the exact token sequence.

Three correctness invariants organize the design: token identity — generated tokens define the trajectory, not a re-tokenized transcript; policy version semantics — the behavior policy's log-probabilities are preserved at the token level; forward consistency — the rollout and the actor must operate on the same model. For MoE policies (mixture-of-experts), Molt applies rollout routing replay: vLLM returns expert IDs for each token, and the training pass replays them to eliminate routing divergences.

Who can access Molt now?

Molt is designed for research groups with access to H100 or H200 clusters. According to NVIDIA, the framework's performance is statistically comparable to the Megatron stack — researchers do not sacrifice speed for compactness. Target audience: frontier labs and funded AI startups working on model post-training; corporate research groups in finance, healthcare, and robotics with access to multi-node GPU clusters; academic labs with H100/H200.

What this means

Molt lowers the cost of iteration in RL research: 8,600 transparent lines accelerate algorithmic experiments, and support for standard SDKs means existing agent code can be trained without rewriting. NVIDIA is building an open stack for agent training, competing with verl and OpenRLHF not only on speed but also on research convenience.

Frequently Asked Questions

How many GPUs are needed to run Molt?

The standard recipe requires 16 NVIDIA H100 GPUs across two nodes: 8 for training and 8 for generation (rollout).

How does Molt differ from verl and OpenRLHF?

Molt contains approximately 8,600 lines of RL code versus 62,000 in verl and 7,200 in OpenRLHF. None of the three components (Ray, vLLM, NVIDIA AutoModel) are forked: updates arrive via a Docker container pin change.