SLPO: A Method That Brings Verifiable-Reward RL to Latent Reasoning Models

In July 2026, researchers published SLPO (Surrogate Latent Policy Optimization) on arXiv, a method that brings outcome-reward reinforcement learning to autoregressive latent reasoning models and teaches them to choose the length of their "thinking" on their own.

What is latent reasoning

Latent reasoning stores a model's intermediate computations as continuous vectors instead of decoding every step into a text token. According to the arXiv abstract, this approach already matches, and sometimes beats, explicit Chain-of-Thought (CoT) reasoning at noticeably shorter compute horizons.

The savings are direct: classic CoT reasoning unrolls a thought token by token, and every intermediate step costs a separate decoding pass. A latent model travels the same path inside its internal representation space, which is faster and cheaper in compute.

Why hasn't RL worked with latent models?

Standard reinforcement learning with verifiable rewards (RLVR) could not be applied to latent models because of two technical barriers. Explicit CoT reasoners have already moved beyond imitating training data thanks to outcome rewards, while latent models have so far remained "tied to imitation."

The cause, the preprint's authors write, lies in the nature of latent trajectories:

• they have no computable per-step likelihood, so there is nothing to attach a per-step reward to;
• they have no adaptive stopping interface under a fixed "thinking" budget;
• as a result, outcome reward did not trigger test-time scaling.

What SLPO proposes

SLPO removes both barriers with two components. The first is an empirical surrogate policy density over latent transitions: it provides a probability distribution through which credit for success can be assigned at the level of the whole trajectory. The second is a stopping head trained on a correctness signal; optimizing against the outcome reward turns it into a variable-horizon policy that decides for itself when to stop reasoning.

Key facts about the work:

• The method is SLPO (Surrogate Latent Policy Optimization), an arXiv preprint from July 2026.
• The goal is to bring outcome-reward RL to autoregressive latent reasoning models.
• Component 1 is a surrogate policy density for assigning reward at the trajectory level.
• Component 2 is a stopping head trained on correctness, which yields variable-length reasoning.
• The result is higher Pass@k under parallel sampling, with more compute going to hard examples.

According to the abstract, in continuous and "soft" thinking modes SLPO raises Pass@k under parallel sampling and allocates longer latent computation to hard problems, while also improving deterministic accuracy.

"SLPO improves

Pass@k under parallel sampling and allocates longer latent computation to hard examples with higher deterministic accuracy," from the preprint's abstract on arXiv.

What this means

SLPO removes the main limitation of latent reasoning models: they can now be fine-tuned with an outcome reward, as text-based chains of reasoning long have been. If the result holds up on larger models, it points toward reasoning models that think more cheaply than standard CoT and dose their reasoning depth to the difficulty of the task on their own.

Frequently asked questions

What is latent reasoning in plain terms?

It is when a model carries out intermediate computations in its internal vector space instead of writing out each step in words. According to the arXiv abstract, this approach is already comparable to explicit Chain-of-Thought (CoT) reasoning while requiring shorter compute horizons.

How does SLPO differ from standard RL for CoT?

Standard RL with verifiable rewards relies on the per-step likelihood of text tokens. Latent trajectories have no such likelihood, so SLPO introduces a surrogate policy density and a trainable stopping head. Only this way does outcome reward start to work for latent models.