Scaling laws on autopilot: AI is now teaching humans how to build neural networks

Imagine you're building a skyscraper, but you don't know for sure whether the foundation can support ten more floors. Large language models work in much the same way. Engineers spend hundreds of millions of dollars on training, hoping that adding another couple thousand GPUs will make a model smarter rather than just more expensive.

These unwritten rules of the game are called scaling laws, and until now, finding them has looked like modern alchemy. That era of guesswork may be ending: researchers from Peking University and Stanford decided to hand this tedious, expensive work to a neural network. The problem is that finding these laws is a painful and extraordinarily expensive process.

Remember DeepMind's famous paper on the "Chinchilla" model (Chinchilla scaling laws). The researchers had to train dozens of small models, collect data on their performance, and try to derive a formula that would predict how the "big brother" would behave. A miscalculation at this stage doesn't just cost time; it costs a fortune.

The new project, working title "AI Scientist," fundamentally changes how this works. Instead of having people pick coefficients and plot graphs by hand, the researchers built a system that analyzes the results of trial runs on its own and formulates the mathematical relationships. The most ironic part is that this virtual scientist did the job better than human experts.

In tests, the system predicted model accuracy with an error significantly lower than that of experienced data scientists. This isn't only about speed or convenience. We tend to think of scientific discovery and intuition as humanity's last strongholds, but it turns out that when it comes to finding hidden patterns in huge datasets, our brains lean too heavily on simplification.

AI doesn't look for "pretty" numbers or simple linear plots; it finds the relationships that actually hold in a multidimensional parameter space. Why does this matter right now? We've reached the point where simply adding compute no longer delivers explosive quality gains.

The industry increasingly whispers about a "plateau," and moving forward will take not just more teraflops but surgical precision in architecture. OpenAI or Google could once afford to burn a city's worth of electricity on an experiment; now investors demand efficiency. Automating the search for scaling laws essentially gives a navigation system to people who used to fly on instruments through thick fog.

We can now know in advance whether it's worth feeding a model another trillion tokens or whether it has already hit its limit. What does this mean for the industry's future? Most likely, development cycles will speed up sharply.

Testing a fundamental hypothesis used to take months; now an automated system can run thousands of scenarios in a matter of hours. That brings us closer to the moment when neural networks start designing their own next generations with almost no human involvement. We still have our hand on the switch, but someone else is already drawing the blueprints.

And that "someone" clearly understands the math of training better than we do. The bottom line: AI has definitively stopped being just a "smart chatbot" and has become a tool for fundamental scientific discovery. If neural networks have learned to optimize their own training better than their creators, then the arrival of full-fledged AGI becomes only a matter of time and the right formula, which, in all likelihood, won't be found by a human either.