DeepSeek-V4-Pro Shrunk 50x and Run on a Single Free T4 on Kaggle

An experiment with DeepSeek-V4-Pro shows that even a 1.6-trillion-parameter model can be brought to a working state without an H100 cluster, provided you give up on running it in its original form. Instead of full inference, the project's author built an extremely aggressive approximation: compressing the weights with low-rank decomposition, streaming gigantic shards through the pipeline and manually adapting the architecture to existing tools. The result is far from production-ready, but running it on a free NVIDIA T4 on Kaggle is a strong demonstration of how much math and engineering ingenuity can now accomplish.

The original description calls DeepSeek-V4-Pro a 1.6-trillion-parameter MoE model with more than 800 GB of weights. Systems of this class usually need very different infrastructure: several H100s, plenty of video memory, fast links between nodes and a decent amount of local disk. Against that background, choosing a free Kaggle instance with a T4 with 16 GB of VRAM and roughly 50 GB of disk is not an attempt to replicate a standard deployment but an experiment at the edge of what is possible. The framing of the task matters too: the goal was not to preserve the model intact, but to test how much useful structure survives radical compression.

The key move in the project is dropping ordinary 4-bit quantization in favor of an SVD "transmutation", that is, low-rank decomposition of the weight matrices. According to the author, rank 64 gave roughly 50x compression. This scheme keeps the main dependencies between parameters but discards a lot of detail, and some quality with it. For a giant model it is a harsh trade: accuracy drops, but there is a chance of fitting the system onto accessible hardware. In essence, this is no longer the original model in the full sense, but its mathematical skeleton, which can still retain some context and associative links.

The second important element is handling the weights in a near-emergency MLOps mode. Rather than storing the full parameter set locally, the author processed shards sequentially via `safe_open`: download one file, extract the needed tensor, compress it in RAM, push the result to a repository and fully clear the cache before the next step. That let a set of weights that would not normally fit on a free machine pass through the disk limit. The author also stresses that RAM usage never rose above 4 GB. This detail matters because such tasks are constrained not only by VRAM but by file logistics, when the model physically cannot even be unpacked without intermediate tricks.

The third layer is architectural identity theft. As the author states, the transformers library did not yet support DeepSeek-V4, so the configuration had to be disguised as DeepSeek-V2, and MoE routing patched separately via monkey patching. From an engineering standpoint this is a brittle trick: it depends on library versions, config formats and the design of the expert router. But this step shows that some of the limits around large models come not only from hardware but from tooling compatibility. If the stack does not yet know a new architecture, researchers often have to adapt the framework to the model first and only then work on output quality.

The result is a version of the model that, according to the author, fits into the memory of a single T4 and can sustain context, but degrades noticeably in quality. Reported side effects include hallucinations and a mix of Russian, English and Chinese in a single answer. That makes the system a poor candidate for reliable applied use, where accuracy, stability and predictability matter. As a proof of concept, though, the project works: it shows that even ultra-large open-weight models can be discussed not only in terms of data centers but also taken apart into more accessible, if heavily cut-down, configurations.

The main takeaway is not that the T4 has suddenly become a replacement for a modern GPU cluster. Quite the opposite: the experiment shows the price of such compromises and the boundary beyond which running a model means not full inference but research reconstruction. Still, projects like this push forward the practice of compression, approximate inference and accessible MLOps. The more such workarounds appear, the lower the entry barrier for people who want to experiment with large models without a corporate budget.