A New Frontier for AI: From Data to Interactive Experience

The last decade, progress in artificial intelligence was measured by scale: larger models, bigger datasets, and more computing power. This approach led to stunning breakthroughs in large language models (LLMs). In just five years, AI made a leap from models like GPT-2, which could barely imitate coherence, to systems like GPT-4, which can reason and engage in meaningful dialogue. And now early prototypes of AI agents that can navigate codebases or browse web pages point to an entirely new frontier.

But scale alone can only take AI so far. The next leap won't come from just scaling up model sizes. It will come from combining increasingly high-quality data with the worlds we build to train models. And the most important question becomes: what do classrooms for AI look like?

In the past few months, Silicon Valley has made its bets, and labs are investing billions in building such classrooms, called reinforcement learning (RL) environments. These environments allow machines to experiment, fail, and improve in realistic digital spaces.

The history of modern AI has unfolded in eras, each defined by the type of data that models consumed. First came the era of pretraining on internet-scale datasets. This public data allowed machines to imitate human language by recognizing statistical patterns. Then came data combined with reinforcement learning from human feedback — a method that uses crowdsourced workers to rate LLM responses — which made AI more useful, responsive, and aligned with human preferences.

Today, data remains the foundation. It is the raw material from which intelligence is built. But we are entering a new phase where data alone is no longer enough. To unlock the next frontier, we must combine high-quality data with environments that allow unlimited interaction, continuous feedback, and learning through action. RL environments don't replace data; they amplify what data can do by allowing models to apply knowledge, test hypotheses, and refine behavior in realistic conditions.

In an RL environment, a model learns through a simple loop: it observes the state of the world, takes an action, and receives a reward that indicates whether that action helped achieve the goal. Over many iterations, the model gradually discovers strategies that lead to better outcomes. The crucial shift is that learning becomes interactive — models don't just predict the next token, but improve through trial, error, and feedback.

For example, language models can already generate code in a simple chat setting. Put them in a live coding environment where they can get context, run their code, debug errors, and refine their solution, and something changes. They move from advising to autonomous problem-solving.

In a software-driven world, the ability of AI to generate and test production-level code in sprawling repositories will be a serious change in capabilities. This leap won't happen just from scaling up datasets; it will happen because of immersive environments where agents can experiment, stumble, and learn through iteration — much like human programmers do. The real world of development is messy: programmers have to deal with poorly defined errors, tangled codebases, and vague requirements.

Training AI to handle this messiness is the only way it will ever move from producing error-prone attempts to creating consistent and reliable solutions.

Web navigation is also messy. Pop-ups, login walls, broken links, and outdated information are woven into everyday browsing workflows. Humans handle these glitches almost instinctively, but AI can only develop this capability by training in environments that mimic the unpredictability of the Internet. Agents need to learn to recover from errors, recognize and overcome UI obstacles, and perform multi-step workflows in widely used applications.

Every major leap in AI development has relied on invisible infrastructure, such as annotators labeling datasets, researchers training reward models, and engineers building scaffolds for LLMs to use tools and actions. Finding large volumes of high-quality datasets once was a bottleneck in AI, and solving it sparked the previous wave of progress. Today, the bottleneck isn't data — it's creating RL environments that are rich, realistic, and truly useful.

The next stage of AI progress won't be a matter of scale luck. It will be the result of combining a solid foundation of data with interactive environments that teach machines to act, adapt, and reason in complex real-world scenarios. Coding sandboxes, OS and browser playgrounds, and safe simulation will turn prediction into competence.