LLM с квадриллионом параметров: когда ждать — и почему ответ «никогда»

Models with a quadrillion parameters (10^15) will not appear in the foreseeable future — physical and computational constraints make this scale practically unachievable within the current paradigm.

How far are we from a quadrillion today?

We are a thousand times further from that coveted number than it seems. The best language models of 2025–2026 operate in the range of hundreds of billions to a few trillion parameters. According to SemiAnalysis analysts, GPT-4 is built on a Mixture of Experts architecture with a total weight count of approximately 1.76 trillion; Google's Gemini Ultra is around 1.56 trillion. To reach a quadrillion (10^15), the scale would need to grow another 700–1,000 times.

• The range of frontier models today: ~100 billion — ~2 trillion parameters
• A quadrillion (10^15) is 700–1,000 times larger than GPT-4
• Training GPT-4, according to analyst estimates, cost over $100 million
• Training Llama 2 70B, according to Meta's technical documentation, required ~539 MWh
• Just storing the weights of a 1Q model in int4 format would take approximately 125 TB

Why a 1,000-fold increase is not just a matter of "waiting"?

A quadrillion-parameter model requires not just more money — it requires fundamentally different physics.

NVIDIA's flagship H100/H200 GPUs hold 80–192 GB of memory per card. Even in aggressive 1-bit quantization, a 1Q model would require ~125 TB just for weights — without activations, gradients, or the optimizer. In standard BF16 format, that is already ~2 petabytes. Such a volume requires building fundamentally different infrastructure that no company in the world currently possesses.

"We see diminishing returns from pure parameter scaling — data and computational efficiency matter more than raw size," — according to the

DeepMind team's research publications on Chinchilla optimization.

Energy consumption adds to the physics challenge. Estimated training of a 1Q model at current chip efficiency would require tens to hundreds of TWh — comparable to the annual electricity consumption of an average European country.

Why there isn't enough data for a quadrillion?

According to DeepMind research (Hoffmann et al., 2022), for compute-optimal training, a model needs to be trained on approximately 20 tokens per parameter. Therefore, a 1Q model would need to be trained on 20 quadrillion tokens. According to various research estimates, all digitized text on the internet amounts to approximately 10^13–10^14 tokens — 200–2,000 times less than what is needed.

Synthetic data partially alleviates the constraint, but not at a thousandfold scale and not without quality degradation in looped generation. In other words, even if the computational infrastructure existed — there would be nothing to feed such a model.

What this means

Quadrillion-parameter LLMs are not a matter of time and funding, but a matter of fundamental constraints: semiconductor physics, the volume of existing data, and diminishing returns from scaling. Industry progress is moving not toward "one enormous model," but toward more efficient architectures — Mixture of Experts, Retrieval-Augmented Generation, and specialized agentic systems.

Frequently Asked Questions

What is the size of the largest open LLM today?

As of 2026, the largest fully open model remains Grok-1 from xAI — 314 billion parameters in a Mixture of Experts architecture, released into open access in March 2024. Closed frontier models from major labs, according to SemiAnalysis analyst estimates, exceed 1 trillion parameters.

Will quantum computers help in training such models?

Quantum computers are effective for a narrow class of tasks — factorization and molecular simulation — but not for matrix multiplication, upon which transformer training is built. In the foreseeable future, quantum systems will not replace GPUs and TPUs in LLM training tasks.

*Meta is recognized as an extremist organization and is banned in Russia.