AWS and NVIDIA launch large-scale Unitree H1 robot training on SageMaker AI

AWS, together with NVIDIA, has shown a complete pipeline for training control policies for the Unitree H1 humanoid robot directly in the cloud, with no GPU cluster of your own.

Why the cloud for robots

Reinforcement learning for physical robots requires billions of simulation steps, and that is no exaggeration. For a humanoid to learn to walk forward without falling, the neural network must go through tens of billions of interactions between a virtual agent and its environment. Doing this in the real world is expensive and dangerous: one failed experiment can mean thousands of dollars in repairs, and the process itself would take years instead of hours. That is why the industry is betting on physical simulation.

The race for a "Moore's law for robots" has already begun: Tesla, Figure, Boston Dynamics and dozens of startups are investing hundreds of millions in building synthetic training environments.

NVIDIA Isaac Lab is a GPU-accelerated simulator that can run thousands of copies of a virtual environment at once on a single node. Until now it was used mainly in large corporate and university labs with expensive hardware.

Isaac Lab is now integrated directly with Amazon SageMaker AI. That means a request for hundreds of GPUs is fulfilled in minutes, and the engineer does not have to think about infrastructure, only about the policy code and the task configuration.

Two ways to run it

AWS offers two modes for different use cases:

• SageMaker HyperPod is a persistent managed cluster; the infrastructure is kept between runs, which suits multi-week research and iterative hyperparameter tuning
• SageMaker Training Jobs is a one-off managed run; resources are allocated strictly for the task and released automatically when it finishes, which makes budget control easier
• p4d and p5 series instances are supported, with NVIDIA A100 and H100 respectively
• Isaac Lab is deployed in a standard Docker container; model weights and checkpoints are saved automatically to Amazon S3
• Training metrics (reward, episode length, entropy loss) stream to Amazon CloudWatch in real time

The key advantage of both options is that they remove the operational burden. You do not have to configure Kubernetes yourself, manage the InfiniBand network between nodes, or balance load across GPUs by hand.

How Unitree H1 training works

Unitree H1 is one of the most accessible production humanoids: about 180 cm tall, 47 kg, with 19 degrees of freedom. That makes it a popular platform for academic research on motion control.

In the Isaac Lab simulation, thousands of virtual copies of the robot learn to walk in parallel using the Proximal Policy Optimization (PPO) algorithm: they fall, get up, adjust their balance and earn a reward for stable forward movement. The trained policy is only as good as the reward function is at describing the desired behavior.

On a single H100 node, Isaac Lab can run up to 4096 parallel simulations at once. When scaling to several nodes, distributed training via PyTorch DDP is used, and gradient synchronization between GPUs happens automatically.

"Scaling to hundreds of GPUs through

SageMaker cuts training time from several days to several hours," the authors of the AWS blog post note.

Once training is done, the policy is exported to ONNX or TorchScript and can be transferred to real hardware through NVIDIA Isaac ROS.

What this means

Cloud reinforcement learning for robots is moving beyond labs with multimillion-dollar hardware budgets. Any small team with an AWS account can now run a serious humanoid training experiment without capital investment in infrastructure.

That changes the economics of robotics: the barrier to entry falls, iteration speed rises, and the next breakthroughs in controlling physical robots may well come from surprisingly small teams.