Ollama and LiteLLM: a Habr guide shows how to run a local LLM chat in Python without the cloud

Habr has published a detailed introductory guide to local LLM development in Python. Instead of starting with cloud APIs, the author recommends pairing Ollama with LiteLLM: install a model on your own computer, set up an environment, and get your first response straight from main.py.

Why local

Most beginner LLM tutorials send newcomers to the cloud on step one: sign up, grab an API key, add a card, watch your limits. For a developer who just wants to understand the basic mechanics, that is extra noise they don't need at all. The new guide takes a different route: run everything locally first, so you can see how the model actually works without billing, external services, or the fear of accidentally spending money on tests.

The approach has another advantage: it makes the full request path transparent. The article breaks the chain down link by link: Python code sends a message to LiteLLM, LiteLLM passes it to Ollama, and Ollama talks to the local model and returns the response to the program. This breakdown isn't only useful for beginners. It helps you quickly figure out where to look when the model doesn't respond, the service isn't running, or the code points to the wrong address.

"This isn't 'AI magic' — it's an ordinary program flow."

What's in the stack

The author separates the tools' roles up front, since they are easy to confuse. Ollama runs the local model and exposes it through a local server. LiteLLM is a Python library that provides a unified interface for calling models. As a result, code that works with a local model today can later be moved to a cloud provider relatively easily, without rewriting the application from scratch. For a first look, it's a practical trade-off between simplicity and future-proofing.

The first part of the series is a step-by-step path with no unnecessary theory. Readers aren't asked to design agents, wire up memory, or build a complex interface right away. The goal is simpler and more useful: confirm that the local model works at all, that Python can reach it, and that the response comes back to your code without external infrastructure. That makes the piece read like a working checklist for your first evening rather than an abstract technology overview.

• install Ollama on Windows, macOS, or Linux;
• download the qwen2.5:3b model and check its response directly in the terminal;
• on weaker hardware, switch to qwen2.5:1.5b;
• create a Python virtual environment and install LiteLLM;
• write a minimal main.py that sends a request to http://localhost:11434.

The choice of starter model is another plus. qwen2.5:3b is presented as a compact and reasonably convenient option for a regular laptop, especially if you need Russian-language support. If resources are tight, the author immediately offers a fallback with a lighter version. That keeps the material grounded rather than abstract: the article promises no miracles, but it genuinely helps you reach a first working response without lengthy config wrangling at the start.

First call from Python

The key part of the text is a minimal Python example. It imports the completion function from LiteLLM, sets the model as ollama_chat/qwen2.5:3b, specifies a local api_base, and passes the user's question in the messages list. This is an important detail: even a single request uses the same structure as a future conversation. In effect, the author isn't just demonstrating a one-off call but laying the groundwork for a console chat with message history and context.

It also helps that the article doesn't stop at the happy path. The end covers common failures: Connection refused if Ollama isn't running; Model not found if the model name in the code doesn't match the installed one; a very slow first response because the model is loading into memory; a ModuleNotFoundError if the package was installed in the wrong environment; and encoding issues in PowerShell. For a beginner, a section like this is often worth more than theory, because these small details are exactly where most first experiments break.

The author has already outlined the next installments: part two will turn the single request into a small console chat, and later parts will add message history and context. So this isn't a standalone snippet but a tidy entry point into a longer path — from running a model locally to a full-fledged application. The format is especially useful for anyone who wants to do more than run a demo and instead gradually make an LLM part of an ordinary Python project.

What it means

Interest in local models is rising again, and materials like this lower the barrier to entry better than any general overview. The Ollama and LiteLLM pairing shows that you can build a first working prototype without the cloud or API keys, then scale the same architecture further if you want. For Russian-speaking developers, it's a good bridge between curiosity about LLMs and real code. Guides like this are what most often turn interest into practice.