MolLean Makes LLMs Check Their Chemistry: Wins on 12 of 12 Tasks

Chemical reasoning from language models often looks convincing yet contains invented claims: a model confidently describes a structure that does not exist, or assigns a molecule properties it does not have. The authors of MolLean propose checking such claims programmatically. They built a process-verifiable language in which every reasoning step is tied to an explicit molecular state and passes through a Lean-based kernel. On 8,800 paired examples from 12 tasks with Qwen3.6-27B, the whole-trace refinement mode beat plain chain-of-thought on every task.

A molecule as a state that cannot be corrupted

In plain chain-of-thought, the model writes its reasoning as text, and nobody checks whether it matches a real molecule. In MolLean, reasoning becomes an executable trace. There is a current molecular state, and every step changes it under strict rules.

The kernel is built in layers. Primitive edits to the molecular graph sit at the bottom, and composite transformations are assembled above them. On top of that, claims about molecular properties are checked. If the model states that a molecule gained a particular group after an edit, or that some property changed, the kernel checks the claim against the state rather than taking it on faith.

The system follows an important rule: a rejected action leaves the state untouched. A faulty step does not contaminate the reasoning that follows. The model gets a refusal and tries a different move from the same point.

Witnesses for external measurements

Not everything in chemistry can be derived from the structure of a graph. Some properties come from external calculations or measurements. To avoid mixing what is verified with what is taken on trust, the authors introduced a typed-witness protocol. It separates native structural checks, which the kernel performs itself, from data supplied from outside.

As a result, the trace shows which claims are proven inside the system and which rest on an external source. For auditing, that difference matters: a reviewer or chemist can see at once where the check is strict and where trust has been delegated.

Three modes and 8,800 examples

MolLean offers three modes of operation:

• single-pass trace execution: the model writes the entire reasoning, and the kernel runs it;
• whole-trace refinement: after verification, the model receives the result and rewrites the entire reasoning;
• interactive mode through MolLean-MCP: the model acts step by step and receives feedback during generation.

MolLean-MCP is a stateful tool interface. It exposes state transitions, check results, and molecular properties to the model. Based on these, the model decides what its next action will be.

The comparison was run against direct chain-of-thought on 8,800 paired examples. They cover 12 tasks: molecule generation, editing, optimization, and reactions. The base model is Qwen3.6-27B.

Key results:

• whole-trace refinement beats direct chain-of-thought on all 12 tasks;
• MolLean-MCP supplements this mode with step-by-step feedback;
• on reaction tasks, the interactive mode gives the best result.

The last point makes sense. A reaction is a chain of transformations in which an early error breaks everything after it. A check after every step catches the error immediately, rather than during a final review of the whole trace.

Where this is useful in practice

The 27-billion-parameter model here serves more as a demonstration: the verification layer does not depend on model size and can accompany other LLMs. For teams building assistants for medicinal chemistry or synthesis planning, the approach suggests a concrete architecture. The model should not just produce text, but call tools that keep state and reject invalid steps.

The pattern itself is useful for agent developers. Transactional state with rollback of rejected actions, a formal verification kernel, and a split between native checks and external data can carry over to other fields where reasoning has a formal meaning. These include materials science, biology, and circuit design.

The limitations are also clear. The gain was measured on one model and on a fixed set of 12 tasks. Verification of external measurements still depends on the quality of the measurements themselves. Even so, the shift in approach is noticeable: a model's chemical claims turn from free text into commitments that can be checked and revised.