a16z: Token Prices Fall, but H100 Rental Rates Hold Steady — the Jevons Paradox in AI

Analysts at a16z have spotted a classic Jevons paradox in the AI market: token prices keep falling, while hourly rental rates for H100 GPUs hold steady or rise. Cheaper inference does not reduce the load on hardware. It increases it, because demand grows faster than the per-unit price falls. For Jensen Huang, this is the best possible scenario. But it rests on a single assumption: that the demand curve does not flatten.

The Jevons paradox in the world of tokens

William Stanley Jevons described this effect in 1865, using coal as his example. Steam engines became more efficient, and the country began burning more coal, not less. The same mechanics apply to tokens. When a query to a model costs pennies, people send many times more of them: they build AI into every product, run agents that make dozens of calls per task, inflate context windows and turn on reasoning chains. The price per token falls, the number of tokens grows even faster, and total compute consumption rises.

H100 rental rates serve here as a direct indicator of scarcity. The H100 is a 2022-generation chip, and in a normal life cycle its rental price should have dropped as new architectures arrived. If the price holds steady or rises, there is still not enough free capacity.

What the data shows

The Decoder's summary publishes conclusions, not the full table of figures. The picture comes down to four points:

• the price per token from model providers continues to fall;
• H100 rental costs are stable and, in some cases, rising;
• demand for tokens is growing faster than their price is falling;
• the main risk is a potential demand plateau, which would put pressure on the entire compute supply chain.

For exact percentages and observation periods, it is better to consult the original a16z source. The conclusion does not change: cheaper AI is not yet freeing up capacity but absorbing it.

Who wins and who is at risk

The chain is long. It includes accelerator makers, cloud providers, specialized GPU clouds, data center builders and the lenders who finance all of it against expected utilization. Each link is betting that demand will keep growing at a pace no lower than today's.

As long as tokens get cheaper and rental rates do not fall, the model works at every level. Model developers cut prices and attract new customers. Customers consume more. Clouds keep their margins on hardware rental. Nvidia sells new accelerators for a growing volume of demand.

If growth slows, rental prices will fall first, then cluster utilization. Operators who bought GPUs on credit and counted on current rates would suffer. The structure is sensitive to any cooling of demand, because capacity is ordered years ahead, while demand can be lost in a quarter.

What teams and investors should do

For companies building products on LLMs, the takeaway is simple: plan the inference budget around usage volume, not the price list. A lower token price will not shrink the bill if the number of agent calls and the context length are growing at the same time. It makes sense to calculate the cost of the whole task rather than the cost of a single request, and to optimize routing between large and small models in advance.

Those who buy compute should watch the H100 rental price, not just token prices. It is an earlier signal: if it starts to fall, supply has caught up with demand, and the terms of long-term contracts can be revisited. Infrastructure investors should look at cluster utilization and rental rates before vendor revenue. Token growth still looks solid, but this pair of metrics will be the first to show the moment the paradox stops working.