SLPO: метод переносит RL с проверяемой наградой на латентные модели рассуждений

In July 2026, researchers published an SLPO (Surrogate Latent Policy Optimization) preprint on arXiv — a method that transfers outcome-reward reinforcement learning to autoregressive latent reasoning models and teaches them to choose the length of their "thinking" on their own.

What is latent reasoning

Latent reasoning stores a model's intermediate computations as continuous vectors instead of decoding every step into a text token. According to the arXiv abstract, this approach already matches, and sometimes surpasses, explicit chain-of-thought reasoning (Chain-of-Thought, CoT) at noticeably shorter computation horizons.

The savings here are direct: classic CoT reasoning unrolls a thought token by token, and each intermediate step costs a separate decoding pass. A latent model travels the same path within an internal representation space — faster and cheaper computationally.

Why didn't RL work with latent models?

Standard RL with verifiable reward (RLVR) could not be applied to latent models because of two technical barriers. Explicit CoT reasoners have already moved beyond imitating training data thanks to outcome-based rewards, while latent models remained "tied to imitation."

The reason, as the preprint's authors write, lies in the nature of latent trajectories:

• they have no computable per-step likelihood — there's nothing to attach a step-by-step reward to;
• there is no adaptive stopping interface under a fixed "thinking" budget;
• because of this, the outcome reward failed to trigger test-time compute scaling.

What SLPO proposes

SLPO closes both barriers with two components. The first is an empirical surrogate policy density over latent transitions: it produces a probability distribution that lets the method distribute "credit" for success at the level of the entire trajectory. The second is a "stopping head," trained on a correctness signal; optimizing for the outcome reward turns it into a variable-horizon policy that decides on its own when to stop reasoning.

Key facts about the work:

• Method — SLPO (Surrogate Latent Policy Optimization), arXiv preprint, July 2026.
• Task — transfer outcome-reward RL to autoregressive latent reasoning models.
• Component 1 — surrogate policy density for assigning reward at the trajectory level.
• Component 2 — a stopping head trained on correctness → variable-length reasoning.
• Result — higher Pass@k under parallel sampling; more compute goes to harder examples.

According to the abstract, in continuous and "soft" thinking modes SLPO raises the Pass@k metric under parallel sampling and allocates longer latent computation to harder tasks, while also raising deterministic accuracy.

"SLPO improves

Pass@k under parallel sampling and allocates longer latent computation to harder examples with higher deterministic accuracy," — from the preprint's abstract on arXiv.

What this means

SLPO removes the main limitation of latent reasoning models: they can now be fine-tuned with outcome-based reward, the way explicit text chain-of-thought models have long been. If the result holds up on larger models, it points toward reasoning models that think more cheaply than standard CoT and dose their own reasoning depth to match task difficulty.

Frequently asked questions

What is latent reasoning in simple terms?

It's when a model carries out its intermediate computations in an internal vector space instead of writing out every step in words. According to the arXiv abstract, this approach is already comparable to explicit chain-of-thought reasoning (CoT) while requiring shorter computation horizons.

How is SLPO different from standard RL for CoT?

Standard RL with verifiable reward relies on the per-step likelihood of text tokens. Latent trajectories have no such likelihood, so SLPO introduces a surrogate policy density and a trainable "stopping head" — only this way does outcome-based reward start working for latent models.