ELR teaches one model to loop: Koi-4B scores 18 and 29

The authors of an arXiv preprint have proposed Elastic Latent Recurrence (ELR), a training recipe for looped language models that do not lose to regular models at one pass. On a 4B dense model, ELR at one pass scores 0.52 points below a comparable baseline on the average of ten tasks. On a 30B-A3B hybrid MoE, it scores 1.04 points above. Four passes add 7.25 and 4.75 points, respectively. Post-training produced Koi-4B: about 18 points on the Artificial Analysis Intelligence Index at one pass, on par with Qwen3.5-4B, and 29 at four.

The one-pass tax

A latent recurrent model applies the same block of layers several times in a row. The number of repeats becomes a tuning knob: set one loop for speed, or more loops for accuracy. A single checkpoint serves both cheap and hard queries. That works in theory. In practice, such models at one pass are noticeably weaker than a regular model of the same size. The authors call this the "one-pass tax." As long as it exists, a separate non-recurrent model is the more sensible choice under strict latency requirements, and the whole idea of one flexible checkpoint loses its point.

ELR targets exactly this tax. At one pass, the model costs the same to serve as a regular one and, trained on the same data, reaches comparable accuracy. Each additional loop buys extra accuracy with extra inference compute.

Three design decisions and several engineering tricks

The working recipe rests on three decisions. First, a gated state update with a learnable step size. Second, the number of loops is mixed within each optimizer step rather than fixed for the whole run. Third, each pass is conditioned on the loop index and the total budget. The first two preserve quality at one loop; the third lowers loss at each additional loop.

The authors also tackled training cost. Mixed-depth training can be switched on late in pretraining. If a zero-initialized conditioning is added at the switch, this late schedule reaches lower loss than mixed training from scratch. In MoE models, reusing the first loop's expert routes in subsequent loops cuts the mixed update time by 11.7% with no measurable loss of accuracy. At inference, storing the attention KV-cache from only the last loop holds memory at the level of a single pass and keeps most of the gain from additional loops.

The numbers

• 4B dense: −0.52 points versus the baseline at one loop, +7.25 at four.
• 30B-A3B MoE (hybrid): +1.04 points at one loop, +4.75 at four.
• MoE, reusing the first loop's routes: −11.7% mixed update time.
• Last-loop KV-cache: attention memory as at one pass.
• Koi-4B: about 18 on the Artificial Analysis Intelligence Index at one loop (estimate) and 29 at four.

Where this is useful

The main practical effect is in serving. Teams that need both fast answers and deep reasoning currently run two models, or a router between different sizes. With an ELR-style model, one set of weights is enough: chat and autocomplete run on one loop, hard tasks on four. That means fewer distinct checkpoints in memory and simpler deployment. Koi-4B's jump from 18 to 29 on the AA Index shows that a small model with headroom in loops can climb toward heavier classes.

There are limitations too. Extra loops cost compute, and the measurements come from the authors themselves, on two architectures and one evaluation index. There are no independent tests on real-world workloads yet, and it is unclear how well the recipe scales beyond 30B. But for anyone choosing between speed and quality on local hardware or in an expensive cloud, the approach deserves a place on the radar: it removes the main argument against looped models.