Ollama and Open WebUI on a GPU-less VPS: An Honest Look at the Limits

Practical experience shows that running a local LLM on a VPS without a GPU is realistic, but you need an honest assessment of the trade-offs.

What you get

Open WebUI is a convenient interface for local models and works without the cloud. Ollama manages model loading and memory. On a basic VPS (2-4 CPU cores, 4-8 GB RAM) you can run small models such as Mistral 7B or Phi 3, but response speed is nothing like what you're used to with GPT.

Real limitations

On a CPU the model thinks more slowly: a single token can take one and a half to two seconds to generate, versus tens of milliseconds on a GPU. That's fine for experiments, but for a production chat you have to choose between speed and cost. RAM and CPU will be maxed out, and competing workloads will slow down.

Minimal stack

• A VPS with at least 4-6 cores, ideally 8 GB of RAM (16 is better)
• Docker and docker-compose for isolation
• Ollama (downloads and caches models)
• Open WebUI (the frontend for Ollama)
• A firewall and a reverse proxy (Nginx) with Basic Auth are mandatory

Local or cloud?

If you run Ollama locally, you pay for hardware once and afterward only for electricity. If you call an API (such as OpenAI or Claude), you pay for every request, but it scales painlessly. For a prototype or experiments, running locally is cheaper. For a production system, it usually ends up costing more because of idle CPU time.

What this means

Local LLMs are getting more accessible, but "just run Ollama" is only realistic if you're ready to accept the speed limits. For small teams that want to control their own data and avoid paying per request, it works.