HOSTKEY wrote a pipe for its bot: Gemma 4 answers in 5–7 seconds, Qwen 3 in 30–40

Hosting provider HOSTKEY has explained why it could not build its Remote Hands Request (RHR, requests for physical server maintenance) support bot from OpenWebUI's standard settings. The team replaced the settings with a pipe: a Python plugin that intercepts the request before it goes to the model and runs it through a fixed chain of steps. The choice of Gemma 4 comes down to response time. With thinking mode disabled, it answers in 5–7 seconds on two 16 GB A4000 GPUs. Qwen 3 on the same hardware thinks for 30–40 seconds, and Gemma 4 trails it in quality only slightly. The write-up's author is Alexander Kazantsev, head of documentation and content at HOSTKEY.

Why model settings were not enough

HOSTKEY initially expected that a custom model built on an open base model, with tools and a knowledge base attached, would be enough for the bot. As an example, the author uses the model hostkeyrurhr, based on gemma4:26b-a4b-it-qat. Its settings enable Function Calling in Legacy mode, set temperature to 0.2, turn off thinking in Ollama, and attach the hostkey_ru knowledge collection built from client documentation. The tools are status_ru_mcp and docs_url_ru: checks of site, Invapi and data center availability in Russia, plus an MCP request for VPS, dedicated server and network traffic configurations.

Asked about a cheap VPS in Germany, the model did not call the get_vps_list tool, which returns a list of VPS with prices by location. The answer sounded plausible, but it was a retelling of instructions from the documentation, not data from the system. Native mode made things worse: even RAG stopped working. When the request goes through the API, for example via CURL, the response is a bare tool_call with empty content, and the client has to call the tool and re-prompt the model. There is no guarantee that the tool will be called, that its data will end up in the answer without mixing with RAG, or that the answer will be in Russian.

In Legacy mode, tool data is mixed into the general context, most often the system prompt. This works with any model, but the model may overcomplicate the call, produce invalid JSON or ignore the tool. In Native mode, the model receives only a list of tools through a dedicated API parameter. Qwen3 works well in both modes, while Gemma 4, even in its it (Instruction-Tuned) version, needs special handling in Native mode. The pipe takes care of that.

Structural complaints piled on top:

• A single model with one large system prompt and one temperature both chooses the action and writes the answer.
• The order of calls is not controlled, yet the team needed this sequence: keyword search, then server availability tools and MCP Invapi, then an answer with RAG.
• RAG cannot be switched off while tools are being called, so the data gets mixed.
• There is no post-processing: no language check, no trimming of service tails, no discarding of placeholder answers, no fallback.
• The user sees a "frozen" interface while several LLM and MCP requests run.

What a pipe is and how hostkeyru_rhr_pipe works

A pipe is a Python class with a pipe method. It receives the request body and returns a string, JSON or a stream. Inside, it can call an LLM any number of times, invoke MCP tools, change the prompt, and turn RAG on or off.

In the OpenWebUI interface, a pipe looks like a separate model: you can assign permissions to it and call it via the API, and external clients only need to swap the custom model's name for the pipe's name. Pipes are created in the Admin Panel, in the Function section. HOSTKEY does not use Valves at all.

These are configurable pipe parameters stored in the database and visible in the admin panel, but the team finds it easier to rewrite prompts and values in code.

The hostkeyru_rhr_pipe pipe serves Russian-language support. Final generation goes to the custom model hostkeyrurhr with RAG enabled, and the pipe does all the preparation before that. It receives body (messages, model, flags, tool_ids, features), tools (a dictionary of available MCP tools) and __event_emitter__, the channel through which statuses are sent to the chat. Instead of a static progress bar, the user sees a feed: "Analyzing the request", "Extracting keywords", "Searching documentation", "Using a tool", "Composing the final answer". Without it, users often leave the chat before the result arrives.

The request text is extracted from the user's last message by code, without neural networks, and it accounts for multimodal messages, which are lists of "text + image" blocks.

Phase −1: a cascade of three tiers

The previous pipe had no "is this an RHR request?" check. As a result, "issue me an invoice" and "ssh isn't working" got the same answer, and the load on the department that services the servers kept growing. The new pipe has a cascade in which each next tier switches on only if the previous one did not give an unambiguous answer.

Tier 1 is the keyword guard (_keyword_prefilter), written in pure Python. It looks for two sets of markers: RHR_KEYWORDS (ssh, перезагруз, raid, сеть, dns, не отвеч) and CLIENT_SOFTWARE_SIGNALS (код сайта, конфиг nginx, wordpress, laravel). If only RHR keys are found, the result is in_scope=True, edge, 0.9. If only software signals are found, it is in_scope=False, client_software, 0.9. If both sets match or neither does, it returns None and the request moves on.

Tier 2 is the LLM classifier (_classify_rhr_scope) on gemma4nothink, a base model with thinking disabled, at temperature 0. The prompt has the RHR_REGULATION rules, five critical rules, eight few-shot examples and a strict JSON response format built in. On any failure it returns in_scope=True, unknown, 0.0, so the client is not turned away.

Tier 3 is the recheck (_recheck_rhr_scope), which guards against false rejections. It runs only if the first pass returned in_scope=False, the category client_software or unknown, and a confidence in the range [0.7; 0.85]. There is no recheck for billing, client_area, legal, sales, abuse or edge.

Examples from the author's table: "ssh isn't working" is settled at the first tier. "I want a refund" is settled at the second as billing with 0.95 and gets a warning. "Something weird with the server" goes to fail-open with a regular answer.

Phase 0: documentation search

The first short request goes to gemma4nothink and turns the user's question into 2–3 keywords for searching the documentation.