AWS launches hourly GPU reservations for ML testing and release prep

AWS has introduced EC2 Capacity Blocks for ML, along with SageMaker training plans that integrate with it, as a new way to reserve GPU capacity for short periods. It addresses a core pain point for ML engineers: GPUs are scarce, and teams often have to pay for long contracts even when they need compute for only a few hours. Now you can reserve exactly as many GPUs as you need, for exactly the time you need them.

When you need GPUs for a short time

In practice, these scenarios come up far more often than you might think. Load testing before a feature launch requires the full infrastructure, but only for a day or two; once testing is done, there's no point in paying for it. Model validation, such as checking a new prompt or a fine-tuned model against real data, often takes 4-8 hours. Team workshops that train engineers on frameworks (PyTorch, TensorFlow) need GPUs for the length of the session, not permanently. Before a major release, teams have to prepare inference infrastructure: spin up servers, warm caches, and run smoke tests. Then there are temporary traffic spikes during peak hours, when you need extra compute that you won't need once demand drops.

• Load testing before feature launches
• Model validation after fine-tuning
• Team training and workshops
• Preparing inference capacity before a release
• Handling temporary traffic spikes

How Capacity Blocks works

The logic is simple. Instead of a Reserved Instance (a monthly or yearly contract) or On-Demand (expensive for sustained use), you reserve a block of GPUs for a specific window, from a few hours to several days. AWS guarantees the capacity will be reserved and available during exactly the window you choose. That gives engineers predictability: you know the GPU will be ready when you've scheduled it. The service integrates with SageMaker Training Plans, so you can launch a training job without worrying that GPUs will run out mid-training.

EC2 Capacity Blocks supports several GPU types: NVIDIA H100 (for LLMs), A100 (the general-purpose option), and L4 (compact, for inference). You pick a configuration based on your workload. Everything is managed through the familiar AWS interface, with integrations for SageMaker, CloudFormation, and other tools.

Pricing and flexibility

Until now, the options weren't great. You could commit to a Reserved Instance a year in advance, which is cheap but inflexible. Or you could pay On-Demand by the hour, which is flexible but costs 3-4 times more. Capacity Blocks sits between those two extremes: it's cheaper than On-Demand, with no long-term contract. Most importantly, you don't pay for time you don't use.

For businesses, this means more accurate infrastructure budgeting and no overpaying. Engineers won't request GPUs "just in case" and inflate costs in the process. DevOps teams can scale infrastructure flexibly ahead of critical moments such as releases, conferences, and marketing campaigns, knowing the exact price of every hour.

What it means

Cloud services are increasingly adapting to the real needs of ML work. The era of booking GPUs in advance and paying for idle time is fading. Instead, you pay only for what you use, at the exact moment you use it. That's cheaper, makes more sense, and cuts waste in infrastructure projects.