Habr author runs Qwen 2.5 14B on a home PC and makes it the resident of his INFERNUM website

A local language model with 14.7 billion parameters, which fits into roughly 9–10 GB of VRAM at 4-bit quantization, now serves as the "mistress" of an entire website. A Habr author set up Qwen 2.5 14B on a home PC and connected it to an interactive project called INFERNUM. The model got a personality, a role and a set of powers inside the page. The author calls it a digital aquarium that holds a character who owns the project.

What we know about the project

The Habr announcement is short, and the technical details are hidden under the cut. The public part shows the main point. The author was not satisfied with getting a model running and ending up with an ordinary "answer bot." He wanted something tangible: a neural network that lives on the site, behaves in character and can act within the boundaries of the platform. The styling nods to a hellish Abode, hence the name INFERNUM.

The author states his motivation without pathos: "because I can." For a hobby project, that is enough. The announcement does not name the hardware configuration, the way the model is launched, the context size or the generation speed, so the performance of this particular build cannot be assessed yet.

Why Qwen 2.5 14B

Alibaba released the model in September 2024, and by autumn 2026 it is no longer new. For a home experiment, it remains a convenient balance of size and quality. The 14B version is distributed under the Apache 2.0 license, so it can be used in projects without separate permissions. It follows instructions and role-play prompts noticeably better than 7B-class models.

General characteristics of the model:

• about 14.7 billion parameters, 48 layers;
• context window of up to 128K tokens, generation of up to 8K tokens at a time;
• trained on a corpus of about 18 trillion tokens;
• multilingual, including Russian;
• about 28–30 GB of memory in FP16, about 9–10 GB at 4-bit quantization plus headroom for the KV-cache.

The last two lines matter most. The quantized version fits on a single GPU with 12–16 GB, which is the level of an ordinary gaming PC. That is what turned such "aquariums" into a mass hobby.

What this kind of architecture looks like

The typical setup for a site with a local model is simple. The web page sends the visitor's message to a backend. The backend assembles a prompt from a system instruction describing the character, the dialogue history and, possibly, the state of the site. It then calls a local inference server, such as Ollama, llama.cpp or vLLM, and returns the answer to the page. The word "powers" in the author's description most likely means function calling: the model returns a structured command, and the backend executes it. We do not know the author's actual implementation; this is a general pattern.

This setup gives full control over the data and does not require paying for tokens. In exchange, it brings the usual problems of home hosting: a single GPU can serve only a limited number of simultaneous dialogues, and the author pays for the bandwidth and the electricity.

What to consider if you want to replicate it

Start with a quantized model in GGUF format and a local server with an OpenAI-compatible API. That makes it easier to swap the model without rewriting the site. Beyond that, you need several safeguards.

• Limit the model's actions to a whitelist of commands and validate each one on the backend. Never execute the model's text directly.
• Treat every visitor message as a potential prompt injection. A character "with powers" is easy to talk out of its role.
• Set limits on response length and on the request rate from a single address, and route GPU requests through a queue.
• Do not put the inference server on an open network. Only a proxy should face the outside.
• Watch the context size: a long dialogue history grows the KV-cache and can push the model out of VRAM.

For implementation details, go to the original article on Habr. The announcement shows the idea, but whether it works depends on the chosen hardware and on how strictly the author separates the model from executable actions.