University of Michigan and NVIDIA to Build an Energy-Saving Method Into Megatron-LM, Cutting Up to 50% of Power Use

About half of the energy spent on AI training can be saved without losing speed or model quality. That is the conclusion of engineers at the University of Michigan, who built a set of coordinated software tools to achieve it. They are now working with NVIDIA to build the approach into Megatron-LM, the company's open platform for training large models.

Where the energy is lost

Training a large model looks like thousands of GPUs running flat out, nonstop. In practice, some of that work is unnecessary. Accelerators wait for one another, data arrives late, and some computations are not on the critical path and could run more slowly with no effect on total training time. If every card still draws maximum power in the meantime, the electricity bill grows for nothing.

The original announcement does not disclose technical details, but the claimed result is clear: with proper coordination, roughly half of the energy spent on training turns out to be excess. The same research group has previously published work on reducing "energy bloat" in large-model training. That work reported smaller figures, around 30%. The claimed figure of up to 50% points to a broader set of tools, but the source does not explicitly describe how the two projects are related.

Why integration into Megatron-LM matters

Research code rarely makes it onto real clusters. To get there, it has to fit into a stack that people already use. Megatron-LM is exactly that kind of stack: training pipelines for models with tens and hundreds of billions of parameters are built on it and its components. If the method becomes part of the platform, it will not have to be brought to each team separately. Enabling an option or updating the version will be enough.

NVIDIA has a direct interest here. Power and cooling have long been a constraint for new data centers, and the power available at a site often determines cluster size better than the hardware budget does. Software that delivers the same training speed at half the power draw effectively expands the capacity of infrastructure that is already built.

What is known so far

• Claimed energy savings in training: up to 50%.
• Training speed and quality, according to the authors, do not suffer.
• Developer: University of Michigan, engineering school.
• Partner: NVIDIA.
• Target platform: open-source Megatron-LM.
• Status: integration is in progress; the announcement gives no timeline.

It is also worth noting what the announcement leaves out. It does not specify the sizes of the models on which the savings were measured, the type and number of GPUs, or the measurement method: whether only the accelerators' energy was counted or that of the whole system. These details determine how reproducible the figure is on someone else's cluster. The words "up to 50%" denote an upper bound, not a typical result.

What teams that train models should do

Until the integration is finished, it makes sense to prepare a baseline for comparison now. If you train models on Megatron-LM, start measuring power consumption at the GPU level alongside step time and throughput. Without this data, you will not be able to verify the claimed savings or assess how much you actually gained.

Watch the Megatron-LM releases and the research group's repositories: that is where both the code and the conditions under which the method applies will appear. For those planning new sites, this is a reason to recalculate the cost model. If the effect is confirmed on real workloads, the power needed for training may turn out to be noticeably lower than estimated, which would change both power-supply specifications and contracts with energy suppliers.