Transformer no longer needed: ex-OpenAI vice president builds a billion-dollar new empire

The corridors of OpenAI seem to be getting roomier. The exodus of key staff from Sam Altman's company has gone from an annoying brain drain to the formation of a whole new industry. This time the news feeds were shaken by a former VP of Research who did not simply leave "for nowhere" but announced a startup with billion-dollar ambitions. His main goal sounds almost sacrilegious to the modern AI community: he intends to challenge the Transformer architecture, the foundation of everything we call modern artificial intelligence.

Let's be honest: the Transformer, given to the world by Google researchers in 2017, became the gold standard. GPT, Claude and Gemini are essentially variations on the same idea. But this architecture has fundamental problems with memory scaling and computational efficiency. The longer the context, the harder it is for the network to "breathe." Former OpenAI leaders who were there when the most powerful models were first trained understand perfectly well that endlessly adding more GPUs is a dead end. Reaching true artificial general intelligence takes something more elegant and efficient than the crude "attention" mechanism that holds up today's tech stack.

The 1 billion dollars the new project plans to raise is not just a nice figure for headlines. It is the price of admission to the major leagues. In a world where training a single model costs hundreds of millions, building an alternative architecture demands enormous resources for experiments with hardware and data. We have already seen attempts to introduce State Space Models (SSM) and Mamba-type architectures, but so far none has managed to dethrone the king. The fact that the person taking this on was responsible for post-training at OpenAI suggests he has a concrete understanding of exactly where the old architecture starts to break.

This departure is part of a broader trend. We are watching the "OpenAI mafia" spread across Silicon Valley, creating the kind of competition Microsoft and Google could only dream of. Anthropic was the first warning bell, and Ilya Sutskever's SSI (Safe Superintelligence) the second. Now comes a third wave: people who want to change not just training methods or safety questions, but the mathematical basis of neural networks itself. If this startup can prove its approach works better over long distances, OpenAI will look like a company that poured billions into perfecting the steam engine just as the internal combustion engine arrived.

Investors seem ready to take the risk. The Valley is currently gripped by a strange mix of euphoria and fear of missing "the next big thing." Everyone understands that the current success of LLMs may be a local maximum. And while Sam Altman is busy turning OpenAI into a commercial corporation and hunting for trillions for chips, his former engineers are trying to reinvent the wheel itself. It is a classic David and Goliath story, except David has 1 billion dollars of venture capital in his pocket and the best experience the industry has to offer.

What does this mean for us? Most likely, we are on the verge of a paradigm change. If the new architecture proves more efficient, AI will become not only smarter but also cheaper, more accessible and, possibly, more autonomous. We should get used to the idea that the acronym GPT could become as much of an archaism as Netscape or AltaVista. In AI, half a year is an epoch and a year is an eternity. And that eternity will, it seems, belong to those who dared to press the "delete" button on the Transformer code.

The key question: Will the new architecture be able to scale as predictably as the Transformer, or are we about to see yet another bubble of ambition that bursts against the harsh reality of distributed computing?