Modal releases a GPU calculator: serverless wins when peak demand exceeds the reservation discount

Modal, a serverless GPU compute platform, has published an interactive model that compares two ways of buying accelerators: pay-as-you-go and a long-term reservation. By default, the calculator prices the B200 at Modal's current rates and at three-year reserved pricing from hyperscalers. The company's main conclusion: when the ratio of peak to average demand is above 5:1, serverless comes out cheaper despite a higher price per GPU-hour.

The article aims to reconcile two camps inside companies. Finance teams want to control spending but don't always understand the application's workload. Engineers understand the workload but can't see market prices. Users can pick a ready-made scenario built from customer experience, or draw their own traffic curve with the mouse.

One formula instead of arguments

The model is simple. GPU demand over time is denoted demand(t). With serverless, the cost equals the rate Rs multiplied by actual demand at each moment, which amounts to the average demand over the period. With a reservation, you have to buy capacity for the peak for the entire contract term: the rate Rr × maximum demand × duration T.

Divide one cost by the other and what remains is a comparison of two demand statistics, the peak and the average. Serverless is cheaper when the peak-to-average ratio is greater than the rate ratio Rs/Rr. The article gives an example: if the reservation is 10 times cheaper and the application peaks at 10 times its average, the costs are equal.

The figures Modal cites:

• typical reservation discounts: 2–5×;
• peak-to-average for inference, training and agentic development: 5–10×, sometimes higher;
• utilization of reserved capacity, per industry surveys: below 30%, often below 10%;
• peak-to-average for spikes above baseline load: 10–100×;
• Enterprise-tier scale: hundreds and thousands of GPUs.

Where the model lies, and in whose favor

The authors candidly list their assumptions. First: demand is perfectly forecast for the entire reservation term. This favors reservations. In practice, underestimating demand causes queues, SLO violations and urgent purchase negotiations, while overestimating it means money already wasted.

The second assumption favors serverless: GPUs are provisioned and released instantly. Modal acknowledges a caveat here: the goal is for SLOs to be met during autoscaling. The company says even a vLLM or SGLang server starts in seconds, and for everything else there are warm pools and buffers, which add cost but get cheaper per unit as scale grows.

Third: the strategy is either entirely serverless or entirely reserved. Some companies cover baseline load with a reservation and send spikes to serverless. The same arithmetic applies to the excess above the baseline, and that is exactly where peak-to-average ratios reach 10–100×. Modal is only exploring a hybrid model; it does not yet offer one as a single provider.

Fourth: GPU cost is not the only factor. Modal points to the speed of its cold starts, which lets teams develop on the same infrastructure they run in production. The article gives no quantitative estimate of this effect.

How to use it

Keep in mind that this is a vendor's tool: the rates and parameters are tuned to Modal's product, and the customer cases are not disclosed. Still, the logic itself is universal, and you can use it to test any offer, including competing ones.

The practical procedure is as follows. Take a real hourly export of your GPU usage covering at least several months and calculate the peak and the average. Divide the serverless rate by the reservation rate, accounting for the contract term and the actual hourly price rather than the list price. If the peak-to-average ratio is noticeably higher, serverless deserves serious consideration. If the load is flat, for example continuous training on a fixed cluster, a reservation remains a sensible choice.

Separately, check the utilization of reservations you have already bought. If it is closer to 10% than to 30%, you can start the conversation about spikes with your own bills. And don't forget to budget for the cost of warm buffers: they determine whether serverless can hold your latency SLO.