Prometheus: Majestic Labs' AI server with 128 TB of memory

AI hardware startup Majestic Labs is developing a new AI server, Prometheus, with up to 128 terabytes of memory. That is more than 60 times the memory of Nvidia's flagship DGX B300, one of the most powerful off-the-shelf systems for AI workloads on the market. According to IEEE Spectrum, the company is betting on an architecture built entirely around DRAM to address the so-called "memory wall", one of the main limits on inference performance for large language models.

Why memory became the main bottleneck for AI

• Product: the Prometheus server from Majestic Labs
• Memory: up to 128 terabytes
• Comparison: more than 60 times the memory of the Nvidia DGX B300
• Architecture: unified and built entirely on DRAM (specifically LPDDR6), unlike Nvidia's combination of HBM and DRAM
• Key figure: Sha Rabii, co-founder and president of Majestic Labs

According to an influential research paper cited by IEEE Spectrum, token generation in large language models is inherently limited by memory speed. How fast a model produces text depends on how quickly data, above all the model's weights, can be read from memory, not on how fast the compute units run. The larger the model, the worse the problem gets. The result is a "memory wall" that caps inference performance no matter how much processor compute grows: however many cores and tensor units engineers add, output speed ends up bound by memory bandwidth and the amount of memory available.

How

Prometheus's DRAM architecture differs from Nvidia's approach

Sha Rabii, co-founder and president of Majestic Labs, argues that memory scale is what gives the company its competitive edge. In his words, Nvidia "did a phenomenal job building a system that can scale," but as models grow, that approach becomes less and less economical: the system "ends up over-provisioned on compute and starved for memory." Nvidia's servers today use fast high-bandwidth memory (HBM), typically to hold model weights, plus a separate, usually larger but slower pool of DRAM that serves the rest of the LLM's needs and the server's overhead.

Majestic Labs took a fundamentally different route: a unified architecture built entirely on DRAM, specifically LPDDR6 memory, with no split between fast and slow pools. According to Rabii, most memory interfaces are designed to work over very short physical distances, sometimes just a few millimeters, and that is what physically limits how much memory can fit in a system at all while keeping bandwidth high.

What this could mean for the AI infrastructure market

If Majestic Labs' bet pays off, the company will offer the market an alternative source of compute for workloads that today run short on memory in particular: long-context work, mixture-of-experts models with a large total parameter count and sparse activation, and heavy KV-cache use when serving many parallel requests. This challenges the common assumption that the AI infrastructure race comes down mainly to the number of GPUs and the amount of compute measured in flops: Prometheus shifts part of the competitive focus to memory capacity and architecture. Given Nvidia's dominant position in the AI server market, the startup's bet on an architecture built around memory rather than compute stands out among most players, who by and large follow the compute-first roadmap Nvidia has set over the past several years.

The very existence of such a project shows that the "memory wall" has stopped being a niche topic in academic papers and become a commercial incentive for launching a new hardware startup. If Majestic Labs takes Prometheus to volume production, data center operators and cloud providers will have a real alternative to the standard "GPU plus HBM" package for the first time in a long while, and competition in AI infrastructure will no longer boil down solely to comparing the number and generation of GPU accelerators.