A Home AI Lab for 200,000 Rubles

The Tesla V100 cost 8,000 dollars in 2017. Today it sells on the secondary market for 1,500. That number has broken the economics of cloud GPUs. On Amazon or Vast.ai, renting a V100 runs 0.40-0.80 dollars an hour, or 300-600 a month. Buy two V100s, build a server for 200,000 rubles, and it pays for itself in six months. The author of the test bought exactly that and tested 128 different neural networks, from small LLMs to video generators. Here is what came out.

Why is the V100 so cheap?

The Tesla V100 is a professional data-center card released in 2017. It has 32 GB of memory, 235 TFLOPS in half precision and a 125 W TDP. It shipped in huge volumes, and then the A100 arrived. Cloud farms sold off their old V100s for next to nothing. Individual miners and startups bought up the rest. Now the secondary market is flooded with the cards, and prices have fallen even lower. The chip is outdated in raw capability, but for inference it is ideal.

Where does the V100 stumble?

To be honest: the V100 struggles with TensorFlow (its optimization of CPU-bound computation is weak). Video generation takes five minutes with Flux.1, and the wait would be painful. It generates text decently, but not fast: Llama 2 70B produces 80 tokens per second at int4. For comparison, a cloud A100 delivers 300+. There are memory caveats too: 32 GB is enough for most models, but you cannot run two 70-billion-parameter models at the same time.

Real benchmarks

• Mistral 7B: 200 tokens/sec (int4)
• Llama 2 70B: 80 tokens/sec (int4)
• Stable Diffusion 1.5: 0.8 seconds per image
• Stable Diffusion XL: 2.5 seconds
• Whisper Large: 0.3 of real time (a 60-minute audio file in 18 minutes)
• Flux.1: 300 seconds at 1024×1024

Economics: cloud or bare metal?

Cloud GPUs (Lambda, Vast.ai) cost 0.40-0.80 $/hour for a V100. Monthly rental comes to 300-600 dollars. Our server pays for itself in 4-6 months of operation. The upsides of running locally: electricity is cheaper than cloud traffic, you get full control, and there is no latency. The downsides: responsibility for the hardware, cooling and upgrades falls entirely on you.

What this means

A local AI server is not a replacement for the cloud but a complement. If you run models every day, have already tested your idea and know which neural networks you need, a 200,000-ruble server saves tens of thousands a year and gives you final control over your infrastructure.