T-Bank Fine-Tunes LLMs With Gradient-Free Evolution: 7.8x Faster on 128 GPUs

A T-Bank team has shown that a language model can be fine-tuned without gradients, that is, without calculating a correction for each parameter based on the size of the error. That calculation is exactly what forces the GPUs in a cluster to exchange data constantly, and synchronization consumes a noticeable share of resources. The new method, ESSA, uses an evolutionary algorithm instead. On math tasks it matched the quality of GRPO, which fine-tunes the whole model, even though it tunes roughly a million times fewer parameters. The speedup grows with the size of the cluster: about 2x on 16 GPUs and up to 7.8x on 128.

Another paper from the same group, F-GRPO, improves GRPO itself. Both papers were presented at EMNLP, an A*-rated natural language processing conference.

How a model learns without gradients

Conventional fine-tuning works like this. The model attempts a task, the system computes the error and derives a gradient: a hint about which way to shift each parameter so the answer improves. When training runs on dozens or hundreds of GPUs, they exchange these gradients at every step. The computation is fast, but the network between the cards often becomes the bottleneck.

ESSA works differently, and its logic resembles selective breeding:

• take the original model and create many slightly different versions of it; • test each version on a set of tasks and count how many it solved; • the most successful ones become the basis for the next generation; • repeat the cycle step by step.

What changes is not the whole model but only a small set of LoRA adapter parameters. LoRA consists of small additional matrices attached to the main model, which itself stays untouched. That is why ESSA operates on roughly a million times fewer parameters than GRPO. GRPO is a popular reinforcement learning method: the model produces several candidate answers to a task, and their rewards are compared against each other.

Why a large cluster helps evolution

Each version of the model is evaluated independently of the others. The GPUs barely need to talk to each other: it is enough to collect the evaluation results at the end. As a result, the network between the cards stops slowing down training.

The researchers give the following speed figures relative to training with GRPO:

• about 2x on 16 GPUs; • up to 7.8x on 128 GPUs.

The picture is straightforward. The more cards, the more expensive synchronization becomes for conventional training, and the bigger the advantage of a method that has almost no synchronization. For large clusters, that is a serious argument.

A caveat: the quality comparison was made on math tasks. The description does not make clear how ESSA will perform on other types of tasks, such as code or long dialogues.

F-GRPO: how to avoid losing solution diversity

The second paper tackles a different problem with GRPO. After fine-tuning, a model often gets used to a few correct ways of solving a problem and picks others less often. For a single attempt this is not a problem, but when you need to try many attempts and find at least one correct solution, diversity becomes important.

F-GRPO changes parameters less on easy tasks that the model already solves with confidence. This keeps it from locking in a single habitual path and preserves different ways of answering.

The result was tested on Qwen2.5-7B with the pass@256 metric, which is the probability that at least one correct answer appears among 256 generated ones. On math tasks, the score rose from 64.1% to 70.3%. The method adds no extra training cost.

What this means for engineers and business

As long as fine-tuning is limited by the number of GPUs and the speed of the network between them, teams have two options: buy more hardware or change the algorithm. These papers show the second option from two angles.

• ESSA reduces dependence on data exchange between cards and gains the most on large clusters. • F-GRPO raises the chance of finding a correct answer among many attempts without making training more expensive. • Both approaches preserve or improve quality without requiring the whole model to be rebuilt.

The results should be assessed with caution: the figures come from math, and other tasks may turn out differently. But the direction is clear. Fine-tuning can be improved by preserving diverse solutions and by updating the model in a different way, not only by buying new GPUs. Primary sources: arXiv 2507.04453 and arXiv 2602.06717.