Google's TurboQuant Algorithm Knocks Memory Maker Stocks After Research Release

Google Research has unveiled TurboQuant, a compression algorithm for AI models that cuts the volume of key working memory at least sixfold without noticeable quality loss. The market reacted instantly: after the publication on March 24, 2026, investors began to reassess how much memory the generative AI industry will actually need.

What Google showed

TurboQuant targets a narrow but expensive part of large language model inference: the key-value cache, or KV cache. It stores context so the model does not have to recompute tokens it has already processed. The longer the prompt, document or conversation, the faster this cache grows and the more GPU memory it takes up.

According to Google, the new method compresses the KV cache to 3 bits per value instead of the standard 16 bits, cutting memory use by at least a factor of six.

In practice this matters as much as it does in research. Freed-up memory lets the same hardware serve more simultaneous requests, run longer context windows, or use larger models without expanding the accelerator fleet. The authors write that TurboQuant requires no retraining or fine-tuning and will be presented at ICLR 2026. The tests used the Gemma, Mistral and Llama model families, along with standard long-context benchmarks.

How the algorithm works

TurboQuant uses a two-stage scheme. First, the PolarQuant method converts vectors to a polar representation, removing the extra overhead data that normally eats into the gains of traditional quantization. Then QJL kicks in, a technique that encodes the residual error with just one additional bit per dimension and reduces distortion in attention. As a result, most of the bit budget goes to preserving the meaning of the original data rather than technical overhead.

Google calls the KV cache a "high-speed digital cheat sheet" for the model.

• KV cache compression from 16 to 3 bits
• at least a 6x reduction in memory
• up to 8x faster attention computation on Nvidia H100 in 4-bit mode
• works without training or fine-tuning
• applicable not only to LLMs but also to vector search

Google says that on Needle in a Haystack tasks, TurboQuant kept a perfect result even with the cache compressed sixfold. On LongBench and ZeroSCROLLS, the method also matched or beat KIVI, one of the well-known baseline approaches to KV cache quantization. Separately, the company tested TurboQuant for vector search and got higher recall without large codebooks or tuning to a specific dataset. That is directly relevant to search, recommendation and advertising systems.

Why the market reacted

The stock market read the publication not as academic progress but as a signal of possibly lower demand for memory in AI infrastructure. Within a few hours of the release, Micron shares fell 3%, Western Digital 4.7% and SanDisk 5.7%.

The logic is simple: if a key component of inference suddenly needs several times less memory, future purchases of HBM, DRAM and storage may look less linear than investors had assumed.

But that does not mean the industry will suddenly need six times less hardware. Memory is only one line item in data center costs, and models' appetite for compute grows faster than any local optimization. Even analysts caution against drawing conclusions too directly: compression algorithms have existed before, but they did not crash demand for infrastructure as a whole. The history of computing more often shows the opposite effect: once resources get cheaper, companies build heavier and more widespread systems on the same budget.

What this means

TurboQuant is not a reason to write off memory makers, but an early indicator of a new stage in the efficiency race. The winners will no longer be only those who buy more GPUs, but also those who can compress inference harder without losing quality. For AI products it is a chance to lower the cost per request, and for the market a reminder that software already affects the market capitalization of hardware.