OpenRSI Index Hits 133,000 Views: Measuring AI Against Scientists

The OpenRSI Index project, which tries to answer the question "how far is AI from the level of frontier researchers," has drawn more than 133,000 views on its launch post on X. The topic also appeared in the platform's Today's News section. The Chinese outlet Jiqizhixin reported this. The available excerpt of the material does not disclose the index's methodology, so below we separate what is known from what remains to be verified.

What is known so far

There are few facts, and they mostly concern audience reaction rather than the project's substance:

• name: OpenRSI Index (the publication also writes it as OpenRSI-Index);
• launch venue: X, with a launch post that has more than 133,000 views;
• a separate signal: a mention in Today's News, where the platform surfaces notable discussions;
• framing of the question: how much time remains until AI surpasses frontier research scientists.

From the name, one can guess that RSI stands for recursive self-improvement, meaning AI that speeds up the development of AI itself. The source does not confirm this expansion in the excerpt provided, so this is a hypothesis.

Why the conversation shifted from leaderboards to science

The original headline contrasts "gaming leaderboards" with "doing science." Behind it lies a familiar problem: classic benchmarks are saturating. Models hit the ceiling, and a difference of a couple of percentage points says little about the ability to conduct research.

The industry has already tried to move toward more realistic tasks. OpenAI released MLE-bench, built on 75 Kaggle competitions, and PaperBench, where an agent reproduces papers from the ICML conference. METR proposed RE-Bench: a set of environments for research engineering, where agents' results are compared with those of human experts. All of these projects test fragments of a researcher's work. A single index that claims to compare AI with frontier scientists has to cover more: forming a hypothesis, designing an experiment, analyzing failures, and choosing the next step.

What to look for in the methodology

High interest does not yet say anything about measurement quality. To judge OpenRSI Index, several questions need answers:

• What tasks make up the index, and how closely do they resemble real research work?
• Who counts as a "frontier researcher," and how is their baseline defined: average, best, time-limited?
• How is test data leakage into training sets ruled out?
• Who evaluates the results: automatic metrics, experts, or another model?
• Are the data and code open (the word Open in the name hints at it), and can the numbers be independently reproduced?
• What exactly counts as "surpassing": speed, accuracy, novelty of the result?

Without answers, any headline figure like "AI is 40% of the way there" remains attractive but hard to verify.

Practical takeaway

For teams that build products on agents or plan research workflows with AI, three things make sense right now. First, wait for the published methodology and check whether its task set matches yours. Second, do not transfer the overall index to your own work; instead, run models on an internal set of real cases and measure time and error rate. Third, when reading any claims about "surpassing scientists," ask what time horizon the tasks were evaluated on: hours and weeks of work give different pictures.

Interest in OpenRSI Index shows that the question of AI researchers has already moved beyond a narrow circle of specialists. How far the index deserves trust will become clear from details that are not yet publicly available.