Claude Code on GLM-5.3 and Qwen: Five Pitfalls of Swapping the Model via LiteLLM

Swapping the model under Claude Code is trivially easy: set ANTHROPIC_BASE_URL and a token, and the agent sends requests over the Anthropic Messages protocol anywhere you like. The problems start on day two. The team behind the Belarusian gateway ZuBrIQ, where Claude Code officially does not work and there is no way to pay Anthropic, described five pitfalls it stepped on itself.

The setup and the main rule

Between the clients (Claude Code, Codex, OpenCode, Zed, Cursor) and the models sits LiteLLM running on two pods. Clients know only stable names: zubr, zubr-flash, zubr-coder. Behind each is a list of routes with an order and a fallback: GLM-5.3 from a provider, DeepSeek V4 Pro as a backup, and the team's own Qwen3.8-27B on four V100s. If the weights change or a provider goes down, the integrations don't notice. That is convenient, but it is exactly where the silent failures hide.

Three silent failures

First: the quiet fallback. GLM-5.3 was set up as a regular openai/ backend. Chat worked, but Claude Code and Codex were getting answers from the fallback model. LiteLLM translates /v1/messages and /v1/responses into the Responses API, the provider returns 404, and the router silently moves on. There is no error in the logs; the only giveaway is the x-litellm-attempted-fallbacks: 1 header. The fix is LiteLLM's native provider for that vendor, which translates everything into chat/completions.

Second: zero spend. With the native provider, LiteLLM does not take prices from model_info, so spend equals zero, key budgets never trigger, and the margin report lies. Prices must be duplicated directly in litellm_params: input_cost_per_token, output_cost_per_token, cache_read_input_token_cost. One more nuance: an edit via PATCH /model/{id}/update takes a minute or two to reach the second pod, so don't rush to conclude it didn't work.

Third: images. GLM-5.3 is strong at code but does not accept images. The provider's chat endpoint returns 400, while the Anthropic-compatible endpoint silently drops the image, and the model confidently guesses from the text. A user sends a screenshot of an error and gets reasoning about something the model never saw. ZubrCode worked around this: if the model in the catalog has no vision, the image is replaced with a text description from GLM-5.3 Flash, and the result is cached by the image's sha256, because the agent replays the entire history on every turn. There was also a separate catalog bug: supports_vision: null was treated as false, and vision vanished for a model that can actually see.

Claude Code config and your own GPUs

Claude Code itself sends service subtasks, such as titles, context compaction and subagents, to Haiku-, Sonnet- and Opus-class models. If you override only ANTHROPIC_MODEL, these requests will fail with 403 in the middle of work. You need to set ANTHROPIC_DEFAULT_OPUS_MODEL, ANTHROPIC_DEFAULT_SONNET_MODEL and ANTHROPIC_DEFAULT_HAIKU_MODEL. Write ANTHROPIC_BASE_URL without /v1, or you will get /v1/v1/messages. The [1m] suffix signals a 1M window and is stripped before sending; for models without it, the window is set by CLAUDE_CODE_MAX_CONTEXT_TOKENS.

Running your own model on 2017-era cards is a story of its own. V100s have no bf16 or FP8, so the setup uses AWQ INT4 with FP16 compute, vLLM with tensor parallelism across 4 cards, and a fork with SM70 support. The fork does not subtract the weights when calculating the KV-cache budget, so --kv-cache-memory-bytes is set explicitly.

Two instances cannot be started at the same time: the initialization peak exceeds 16 GB, a worker hits OOM, and the main process hangs on the NCCL watchdog even though systemd shows active. Now the second unit waits for the first one's health check. The dense 27B instead of the MoE 35B-A3B is smarter and sees images itself, but the first token arrives in ~1.

3 s instead of ~0.6 s, so it is kept for background tasks and confidential data.

What to do

The limitations are obvious: this is one team's experience, the prices in the examples are illustrative, and LiteLLM's behavior may change between versions. But the principle is general. If you put a gateway between an agent and models, monitor the fallback headers, reconcile spend against the provider's real invoices, check what happens to an image on each route, and define all three slots: Haiku, Sonnet and Opus. The most expensive bugs here don't crash with an error; they answer confidently, just not with the right words. The gateway is part of the authors' commercial platform, so the article is partly promotional, but the technical details are easy to verify on your own test bench.