How to Train Gemma-3 on Math with GRPO and LoRA Adapters

On July 5, 2026, researchers published a detailed workflow for fine-tuning Google's Gemma-3 on the GSM8K math dataset using the GRPO algorithm, LoRA adapters and the Tunix framework.

What GRPO is and how it differs from classic methods

GRPO (Group Relative Policy Optimization) is a reinforcement learning algorithm adapted specifically for language models. Unlike classic PPO, which requires a separate critic model, GRPO compares several generated answers against each other within a single group and updates the policy in favor of the better solutions. This reduces memory use and makes training more stable.

This approach suits math problems particularly well: an answer is either correct or it is not, so the reward signal is computed automatically and unambiguously. GSM8K (Grade School Math 8K) is a standard benchmark of 8,500 grade-school math problems with multi-step solutions. It requires the model not just to output a number but to reproduce the chain of reasoning, which makes it an ideal testing ground for GRPO.

• Base model: Google's Gemma-3
• Dataset: GSM8K (8,500 grade-school problems)
• Algorithm: GRPO (Group Relative Policy Optimization)
• Framework: Tunix on top of Hugging Face Transformers
• Fine-tuning: LoRA adapters, with the main weights left unchanged

How the pipeline is built, step by step

The workflow is split into logical stages. First, the environment is set up and authentication on Hugging Face Hub is completed, since that is where the Gemma-3 weights are downloaded from. Then each GSM8K example is wrapped in a "reasoning + answer" template: from the first training steps, the model sees the expected output structure and learns to follow it.

The central element is two reward functions. The first checks format compliance: whether the response contains a reasoning block and a numeric result. The second scores numeric accuracy, that is, whether the final number matches the reference answer from the dataset. GRPO optimizes both signals at once, generating a group of candidates and selecting the best.

To avoid retraining the entire model, LoRA adapters are attached to Gemma-3. These are compact low-rank matrices that are trained in place of the frozen main weights. As a result, the number of trainable parameters drops sharply, and GPU memory requirements fall far enough that the workflow becomes feasible on consumer hardware.

After training, the authors suggest measuring accuracy with the adapters and comparing it with the base checkpoint. Optionally, you can merge the adapters into the main weights and export the final model for deployment.

Why this matters for developers

"LoRA adapters make the whole process feasible without a data center,"

the workflow description says.

Just a few years ago, fine-tuning large models required tens of gigabytes of video memory and access to professional clusters. Combining GRPO with LoRA changes that: the algorithm is already more memory-efficient than classic PPO, and LoRA further cuts the number of trainable parameters by 10 to 100 times, depending on the adapter's rank.

The practical upshot: a developer with a single modern GPU can take an open model like Gemma-3, specialize it for a particular domain such as math, law or medicine, and end up with a specialized model that shows measurable improvement on the target benchmark. Tunix acts as a convenient wrapper that automates GRPO configuration and adapter management; the same approach applies to any dataset with verifiable answers.

What this means

GRPO fine-tuning with LoRA on domain datasets is moving out of academic papers and into practical guides. The authors demonstrated the full cycle, from download to export, on a real model and a public benchmark, lowering the barrier for developers who want to improve an open model for a specific task without costly infrastructure.