Four-parameter SIMPLE law predicts inference energy of 13 LLMs on H100 without calibration

AI is billed per token, but tokens do not all cost the same. Every new token goes into the KV-cache and makes all subsequent tokens more expensive, so energy grows nonlinearly along a sequence even though the tokens themselves look identical. A group of researchers at the University of Bologna (the DEI and DISI departments) has proposed the SIMPLE law: a compact four-parameter formula that expresses inference energy through the architecture's own hyperparameters.

They validated it on traces from 13 dense LLMs from six families (OPT, LLaMA, Gemma, Falcon, Qwen2 and Granite) running on an NVIDIA H100 GPU. Measured by MAPE, the mean relative error of the prediction, SIMPLE proved more accurate than the previous SOTA. It does so without per-model calibration, which means it also applies to models that nobody has measured.

Why existing models are no good for planning

Inference, not training, consumes most of a model's energy: the authors cite an estimate of up to 90% of lifecycle energy. A model is trained once and then served continuously. Yet for an LLM that has not been deployed yet, there is only one way to learn the price of a token: build it, run it and measure. By that point, the architecture can no longer be chosen.

The target is also moving. Reasoning models spend extra tokens to gain accuracy, and agentic workloads have raised the median prompt length from 750–1050 to 68 thousand tokens. Price tags mislead too: a model with a lower list price per token can end up costing more in total than a nominally pricier one.

Previous analytical models fall into two camps. One group empirically fits energy to input and output length, using linear formulas and formulas with an input-output interaction term. These describe the measured regime but do not explain it. The most accurate known model derives decode in closed form from compute and memory accesses, but its coefficients are fitted to a single LLM. The number of layers, the hidden dimension and the attention design do not appear in its formula explicitly. It therefore has to be recalibrated for each model, and it knows nothing about the differences between MHA and GQA.

What the law contains

The authors derived closed-form expressions for computational complexity and the number of memory accesses during inference, separately for Multi-Head Attention (MHA) and Grouped-Query Attention (GQA). Using these, they compared a family of analytical models of varying granularity. SIMPLE showed the best balance of accuracy and compactness: its four parameters separate prefill from decode, and decode is further split into the KV-cache cost and the computation of the next token (the forward pass through the FFN).

The evaluation used traces from 13 dense LLMs from six families on an NVIDIA H100. The law's MAPE is lower than that of the previous SOTA, and this holds without per-model calibration. The specific percentage reduction in error is not given in the available text of the paper.

What the law explains

The formula yields several practical conclusions:

• Prefill energy grows quadratically with the hidden dimension, while network depth contributes linearly. As a result, large models suffer most from long inputs.
• The more aggressively GQA groups keys and values, the lower the energy per token compared with standard MHA. For an architect, this is one of the most effective levers for saving energy.
• The law makes it possible to find analytically the output length at which energy per token is minimal. Raising the reasoning token budget pushes the length past this minimum, and energy grows superlinearly. Accuracy also rises with the budget, but energy rises faster, so the Intelligence per Joule metric (accuracy divided by the energy of a request) peaks at an intermediate budget and falls at the edges of the range.
• Agentic requests are noticeably more expensive than ordinary ones: both their prefill term and their KV-cache term grow. Prefix caching removes most of the prefill cost, but the KV-cache term remains. So even a cached agentic request costs more than a non-agentic one.

Limitations

The work covers dense models and two attention types, MHA and GQA. The authors only treat MLA as a variant of GQA. The measurements were made on a single GPU type, the H100, in single-card mode, without accounting for batch size or distributed inference. MoE architectures were not part of the evaluation. For practitioners, the takeaway is this: the law is suitable for estimating energy and cost before a model is run and for comparing architectures. It helps choose a reasoning budget and work out at what context length an agentic workload stops being cost-effective. The authors have also released an interactive implementation of the law.