RSI Becomes the New Goal of AI Labs, but Its Definition Remains Blurry

Recursive self-improvement is rapidly displacing AGI on AI labs' agendas. The concept remains as blurry and elusive as its predecessor.

What is RSI

RSI (Recursive Self-Improvement) is the hypothetical ability of an AI system to analyze and improve its own code without human intervention. The system grows more powerful and capable until it reaches a point beyond which humanity can no longer control it or understand what exactly happened.

In theory this sounds simple and logical: to improve is to work faster, more accurately, more efficiently and more reliably. In practice, it is extremely hard to say whether a system has really improved or has simply grown more confident in its own abilities. How do you tell real progress from an illusion of progress? Who checks?

Why AGI is giving way to RSI

A few years ago, AGI seemed like the inevitable end point of AI development: artificial general intelligence, human-like and universal, able to solve any task at human level or above. But the definition was too sharp and unreachable, like a horizon. Decade after decade, AGI kept receding into the future.

RSI offers a different path. There is no need to wait for AGI; it is enough for a system to learn to improve itself. It is an intermediate goal that looks both more realistic and, to investors and politicians, more frightening.

The advantages of RSI are obvious:

• A more realistic intermediate goal than AGI
• Easier to convince investors the project is viable
• No need to wait another two to three decades
• You can start with small steps of self-improvement right now
• Looks less threatening than AGI

That is why even conservative labs have switched to RSI in their long-term plans and public statements.

The definition problem (which nobody is solving)

It would seem everything is simple and clear. But here the problems begin. One researcher talks about self-improvement through optimizing a neural network's weights. Another means automatic debugging and bug fixing. A third means rebuilding the model's architecture entirely. A fourth simply says the system gets better at its main task.

Each of them is right, but each is talking about something entirely different. And all of them call it by one name: RSI.

"Everyone means something different by RSI.

It's exactly what happened with AGI ten years ago: an existential risk from something nobody can define."

When the definition is blurry, it becomes impossible to set concrete goals, measure progress or even assess the real risk. Labs talk about RSI as if it were a single goal and a single finish line, when in fact it is ten different projects going by one name.

What this means

RSI is neither malicious intent nor a conspiracy, but an honest sign that AI labs need an intermediate milestone between today's LLMs and the philosophical AGI. History repeats itself: when one vague goal (AGI) becomes unreachable, another (RSI) is born. The question stays the same: how do you measure something that is not defined? How do you manage the risks of something that can mean any action at all?

So far there is no better answer to this question. Meanwhile, labs talk about RSI as if it were already a solved problem.