Of 1 598 RL Papers, 37 Made It Into Frameworks: An Audit of Seven Open-Source Stacks

A new analysis of seven open-source frameworks for RL post-training of language models shows how narrow the path from paper to working code is. The authors went through the commit histories and matched them against 6 939 selected arXiv papers from January 2025 to June 2026. Of the 1 598 papers that propose a general RL training method, only 37 ended up in at least one framework. That is about 2.3%.

For practitioners the reason this matters is simple: if a method is in a framework, it is available with a single line in a config. If it is not, you have to rewrite it yourself, and at the scale of tens of billions of parameters that is expensive and risky.

Papers keep piling up, but the queue for inclusion is not moving

The authors split the papers into five quarterly cohorts. Cohort size grew from 72 to 386 papers, a factor of 5.4. Meanwhile, the number of methods that frameworks merged within 180 days stayed in a band of 5 to 7. A rough estimate from the edges of that band: at the start it is on the order of 7–10% of a cohort, and by the end of the period only 1.3–1.8%.

The bottleneck, it seems, is not idea generation but maintainer resources. Every new implementation needs tests, support and compatibility with the rest of the code.

What gets through

Included papers differ noticeably from the other papers published in the same quarters:

• Policies of 30B parameters and up: 41% of included papers vs. 10% of the rest.
• Changes to a coefficient, clip, mask or weight in an existing loss function: 29% vs. 9%.
• Code release: equally common in both groups, so an open repository gives no advantage.
• Both gaps persist when you exclude papers from companies that maintain the frameworks themselves.

The last point answers the obvious objection that frameworks simply adopt their "own" work. The pattern is broader: methods that get in are ones that were tested on large models and can be added with a small change. If a paper requires a new component or a rebuilt pipeline, its chances are lower.

Rejections are almost invisible

The most unusual finding concerns public discussions. In issues and pull requests, methods are rarely rejected openly. For more than nine in ten general methods that did not make it into any framework, there is not a single thread that cites or names them. Many of the proposals that do appear stay open without a response. A method does not get turned down; it simply never comes into view.

Two additional checks fill out the picture. Users rarely report training failures publicly, so maintainers get few signals about which methods are actually needed. And the technical reports of 13 large open-weight models name 8 of the 13 widely adopted methods. That suggests the practice of large labs is a strong lever, not just the quality of the paper itself.

A tracker for those choosing a method

The authors are publishing the data as a tracker. It links training failures to framework implementations, and it also lists each project's contribution rules and earlier method proposals.

For teams that train models, several takeaways follow. Before adopting a method from a fresh paper, check whether it exists in a framework and what status its issue is stuck in. A missing implementation does not mean the method is weak, but it does mean you will be validating it on your own. Researchers who want to see their ideas in these stacks would do well to frame a method as a small change to a coefficient, clip or mask in an existing loss and test it on models of 30B and up. And framework users should describe failures publicly more often: that almost never happens now, and maintainers are choosing methods blind.