Replacing LLM weights with a recipe: how 16 GB of matrices are being folded into formulas

An 8B-parameter model in fp16 weighs about 16 GB: eight billion numbers at two bytes each. The 70B version already comes to 140 GB. Quantization to 4 bits shrinks the 8B model to roughly 4–5 GB, but these are still the same matrices, just stored at reduced precision. Habr AI has published a breakdown of a more radical idea: store not the weights themselves but a small recipe from which they can be computed.

The announcement of the piece covers several families of such methods. Implementation details and benchmarks appear only in the author's full article, so below we describe the approaches themselves and their known limitations.

A network that draws a network

The best-known variant is hypernetworks. The idea was described in the 2016 HyperNetworks paper: a small network takes, for example, a layer number and a learnable vector as input, and outputs the weights of a large network. Only the generator and a set of short vectors need to be stored.

The approach has an obvious cost. The generator produces weights from a narrow family, so if a real task requires "inconvenient" matrices, accuracy drops. Training is also harder: the gradient flows through two levels of networks rather than one.

Formulas and fractals

The second branch is analytical descriptions. If a layer's weights fit a smooth function well, it is enough to store its coefficients and compute the values from the coordinates of each matrix element. Implicit neural representations work this way, for example SIREN for images and NeRF for scenes.

The fractal idea comes from image compression in the late 1980s. Blocks of data are described as transformed copies of other blocks, and instead of an array, a set of self-similarity rules is stored. For weights, this is possible as long as they contain repeating structure. Trained LLMs are far from always showing it.

Methods based on pseudorandom number generators follow a similar logic. In the 2024 SeedLM paper, a block of weights is approximated by a combination of a sequence produced by a short seed and a few coefficients. The seed is stored, and the rest is recomputed at load time.

What you pay

In this area, compactness is almost never free. The general trade-offs are as follows:

• Memory versus compute. Weights are not read from memory but generated, which adds operations at every layer.
• Accuracy. Any approximation introduces error, and errors accumulate with model depth.
• Training. The model is either trained directly in the compressed parameterization or compressed after training, and in the latter case quality depends on the properties of the specific weights.
• Hardware. GPUs are optimized for dense matrix multiplication. Nonstandard on-the-fly weight generation can eat up the memory savings.

The author compares the path of LLMs to the fate of the original Doom, which has been run on calculators, printers, and other household electronics. For now, its LLM counterpart does not live on every kettle: even a heavily compressed model runs into memory bandwidth, not just memory capacity.

Practical takeaway

For engineers deploying models on devices, compression via recipes remains a research direction. In production, quantization, LoRA adapters, and distillation still do the work. Hypernetworks and seed-based methods are worth watching for those constrained by distribution size: for mobile apps, in-browser inference, and embedded systems, even a twofold saving on storage can decide whether a model can run at all. Before adopting anything, check three things: quality loss on your own tasks, latency on the target hardware, and the complexity of fine-tuning.