LLMs on a GPU-less VPS: 4B models sort support tickets with 90% accuracy in 3–5 seconds
A Habr author tested whether a VPS with 8 GB of RAM and no GPU can handle routine work such as sorting support tickets. The best models in the 2–4B parameter range identified a ticket's topic in 3–5 seconds with about 90% accuracy, while answering from a knowledge base took 0.5–2.5 minutes. That is enough for background tasks, but not for live chat. Pick a 2–4B model and put a shared instruction at the start of the prompt.
AI-processed from Habr AI; edited by Hamidun News
Picture an online store whose support team receives hundreds of emails every day: "where is my order," "I want to return an item," "the promo code doesn't work." They need to be sorted into categories, and you would rather do that on your own inexpensive server than through a cloud API: customer data stays with you, and the bill doesn't grow with the number of requests. Renting a GPU, meanwhile, is expensive.
A Habr author has published part two of a test of lightweight language models on an ordinary VPS without a GPU, taking into account criticism from the comments on part one. The main result: on a server with 8 GB of RAM, the best models in the 2–4 billion parameter range (2–4B) assigned a ticket to the right category in 3–5 seconds, and the choice was correct roughly 9 times out of 10 (about 90% accuracy). Knowledge-base answers and short document summaries took 0.
5 to 2.5 minutes: acceptable for background processing, too slow for a chat with a customer.
The test bench and what changed after the criticism
The tests ran on a JustHost VPS: 5 vCPU Intel Xeon Gold 6148, 8 GB RAM, 4 GB swap, Ubuntu 24.04, Ollama 0.35.0 in Docker. The models used Q4_K_M quantization. To the leaders of part one (QVikhr-3-4B, qwen3.5:2b and 4b, gemma3:4b, granite4:micro-h), the author added five larger models: GigaChat 3.1 Lightning, T-lite-it-2.1, YandexGPT-5-Lite-8B, Qwen3-8B and GLM-4-9B. For classification, d1-2B from Liquid AI was added as well.
Readers had faulted part one for the lack of repeated runs, the lack of no-cache measurements and the tiny samples. Now all models use the same prompt, temperature 0 and seed 42, and the classification on 326 tickets was run twice: all 10 models gave identical answers in every case. The steal metric (how much CPU time neighboring tenants on the host take away) averaged 0.2% over 41 hours of load, so virtualization noise was minimal.
Memory matters more than CPU, and the cache matters more than the model
Moving from 4 to 8 GB eliminated out-of-memory crashes on long texts and almost all swap traffic. For example, QVikhr-3-4B answered from the knowledge base in 73 seconds instead of 94, and swap traffic dropped from 9.1 GB to 43 MB. On this scenario gemma3:4b crashed with 4 GB, but with 8 GB it answered 14 of 15 questions correctly. The fifth vCPU added only 20–25% to prompt-reading speed and about 10% to generation speed — extra memory is more useful than an extra core.
Prompt structure has the biggest effect on speed. If every request begins with the same instruction, the model remembers it and reads only the part that changes — the ticket text. If the instruction has to be re-read each time, latency grows 2–3 times: with the cache, QVikhr-3-4B classified a ticket in 3.8 seconds; without it, in 12.4. The practical rule: shared instruction first, customer text last. With the hybrid qwen3.5 models, the cache behaved unpredictably: qwen3.5:2b started taking the prompt from the cache only after 116 requests, and qwen3.5:4b never did so in the load test.
Accuracy, model size and load
In the first test on 20 tickets, seven models scored 100%, but on 326 tickets 9 of 10 models scored 87–92%. The 2–3 point gap between the leaders is within statistical error, so choose by speed and memory, and validate the model on hundreds of your own examples. The reference labels for the 326 tickets were produced by models; the author blind-labeled 50 random tickets himself, and 45 matched the reference.
The 8–10B models did not outperform the best 4B models and ran slower. The exception on speed is GigaChat 3.1 Lightning: it has 10B parameters, but thanks to its MoE architecture only 1.8B are active for each token. If you need a Russian model, pick this one.
Ollama processes requests one after another, so with four clients each waits about four times longer, and the number of requests per hour does not increase. With a shared prompt, QVikhr and GigaChat handled 750–1000 short requests per hour; without the cache, about 300. On long requests (about 2500 input tokens), qwen3.5:2b was the best at 104 requests per hour versus 48 for QVikhr-3-4B.
d1-2B deserves separate mention: it does not generate text, but outputs the probability of each category in a single pass. Its accuracy matches the generative models, but at a 0.9 threshold it would handle 69% of tickets on its own with 97% accuracy and pass the rest to a human. For support, where an error costs more than an extra manual check, this is a convenient option. It required a separate Ollama 0.40.15 and the num_batch 2048 parameter; otherwise a 946-token ticket returned an error.
One more practical detail: on a server with 8 GB, Ollama's prompt cache can take all the memory by default, and with qwen3.5:4b this caused crashes. Adding LLAMA_ARG_CACHE_RAM=1024 to the settings fixed it. The datasets, scripts and raw responses are in the repository at github.com/alvec7/llm-cpu-bench.
- Nov 18, 2026Qwen on two DGX Spark units runs 34–59% faster in tokens per second and outpaces GLM
- Nov 18, 2026Compressed Bonsai 2 27B hit 17 tokens/s on a 2017 GPU, while Qwen3 8B hit 32
- Nov 18, 2026A $3,000 Mining Card Turned Into an A100 Equivalent: From 8 to 80 GB of Memory
- Nov 12, 2026n8n Releases Nod, an Open-Source Agent That Runs on Your Computer
Key Questions Answered
What are the key details on The test bench and what changed after the criticism?
The tests ran on a JustHost VPS: 5 vCPU Intel Xeon Gold 6148, 8 GB RAM, 4 GB swap, Ubuntu 24.04, Ollama 0.35.0 in Docker. The models used Q4_K_M quantization. To the leaders of part one (QVikhr-3-4B, qwen3.5:2b and 4b, gemma3:4b, granite4:micro-h), the author added five larger models: GigaChat 3.1 Lightning, T-lite-it-2.1, YandexGPT-5-Lite-8B, Qwen3-8B and GLM-4-9B. For classification, d1-2B from Liquid AI was added as well.
What are the key details on Memory matters more than CPU, and the cache matters more than the model?
Moving from 4 to 8 GB eliminated out-of-memory crashes on long texts and almost all swap traffic. For example, QVikhr-3-4B answered from the knowledge base in 73 seconds instead of 94, and swap traffic dropped from 9.1 GB to 43 MB. On this scenario gemma3:4b crashed with 4 GB, but with 8 GB it answered 14 of 15 questions correctly. The fifth vCPU added only 20–25% to prompt-reading speed and about 10% to generation speed — extra memory is more useful than an extra core.
What are the key details on Accuracy, model size and load?
In the first test on 20 tickets, seven models scored 100%, but on 326 tickets 9 of 10 models scored 87–92%. The 2–3 point gap between the leaders is within statistical error, so choose by speed and memory, and validate the model on hundreds of your own examples. The reference labels for the 326 tickets were produced by models; the author blind-labeled 50 random tickets himself, and 45 matched the reference. The 8–10B models did not outperform the best 4B models and ran slower.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.