Lean4AgenticRL: LAR-27B Scores 77.3% on TerminalBench-2.1 with Formal Rewards

The Lean4AgenticRL team has proposed a way to train LLM agents on the quality of intermediate steps, not just the final result. The steps are verified formally, through a Lean4 schema. The authors trained two models with the framework, LAR-9B and LAR-27B. The larger one scored 77.3% on TerminalBench-2.1. On average, the models beat the base model by 9.89% and the strongest model of comparable size known to the authors by 5.07%.

The problem: the agent learns from a final score

Agentic tasks in software development and terminal work are long. The agent reads a repository, runs commands, edits files, runs tests, backs out, and tries again. Modern LLMs for these scenarios usually go through large-scale RL fine-tuning. According to the authors, outcome reward dominates that process: a reward for the end result, such as passing tests.

This signal has an obvious flaw. It arrives at the end and says nothing about which of dozens of steps was a good one and which was superfluous. Tool calls at the lowest level and workflow stages at the middle level get almost no reward of their own. The authors argue this makes training less efficient.

How Lean4AgenticRL works

The approach is built around a Lean4 formalization. The authors say it is the first framework in which multi-level process rewards for open agentic RL rest on a single Lean4 schema. The pipeline looks like this.

First, the framework builds a problem-specific workflow, a process tailored to the particular task. It has to balance the agent's own capabilities against controllability: the agent is not put on rigid rails, but it gets a structure against which its behavior can be checked.

Next, the workflow is formalized and rewards are assigned at several levels. This makes it possible to verify the agent's actions inside a general-purpose harness. Training uses GRPO with normalization by workflow stage. The idea is to compare and average the signal within a stage rather than across the whole trajectory at once.

Key parameters of the work:

• framework: Lean4AgenticRL, with rewards at three levels (tool call, workflow stage, final outcome);
• algorithm: workflow-stage-normalized GRPO;
• models: LAR-9B and LAR-27B;
• best result: 77.3% for LAR-27B on TerminalBench-2.1;
• gain over the base model: 9.89% on average;
• margin over the second-strongest model of comparable size: 5.07%.

What the numbers show

77.3% on TerminalBench-2.1 looks like a strong result for a 27B-parameter model. But the abstract lacks the details needed for a precise assessment. The base model is not named. The full set of benchmarks behind the average is not specified. It is not disclosed whether 9.89% and 5.07% are percentage points or relative figures. It does not say which model is the "second-strongest." Until the full paper has been examined, these numbers are best read as the authors' claim, not as an independent comparison.

Still, the direction is clear. If step-level rewards can be obtained verifiably, without a trained reward model and without noisy evaluation by another LLM, the process signal becomes cheaper and more reliable. That is the bet behind formal methods.

What teams should do with this

For teams that fine-tune agents on SWE and terminal tasks themselves, the main takeaway is an engineering one: look at the structure of the trajectory, not just the final pass/fail. Even without Lean4, it helps to label typical stages (exploration, editing, verification) and measure progress against them.

It is worth waiting for the full paper and, if the authors release them, the code and weights of LAR-9B/27B. Then it will be possible to check how much work formalizing a workflow for a new class of tasks requires, and whether the gains carry over beyond terminal benchmarks. The cost of that labeling will determine whether the approach becomes a working tool or stays a research demonstration.