Local GLM 5.3 Speeds Up AI on ESP32-S3 by 100%, Qwen 3.8 27B by 30%
The author built a home server with two Tesla V100s for 113 105 ₽ and gave local models tasks without a detailed spec. The main test: speed up a small language model on an ESP32-S3 microcontroller, where the engine was written in pure C. GLM 5.3 pushed generation to ~20 tokens per second (+100%), Qwen 3.8 27B to ~13 (+30%). So even a budget build can already handle low-level optimization with local models.
AI-processed from Habr AI; edited by Hamidun News
Local neural networks are usually tested with the same tasks: "make a Minecraft clone" or "draw a Windows 11 interface." The author of an article on Habr decided that was boring. He built a server at home for 113 105 ₽ and gave the models fuzzy tasks close to vibe coding, where instead of a detailed spec the prompt is "make it good."
The most telling example is the ESP32-S3, a cheap microcontroller that runs a small language model. Its engine was written in pure C, and the models had to rewrite it to use the chip's vector instructions. Speed is measured in tokens per second: GLM 5.
3 reached about 20 (+100% over the original), Qwen 3.8 27B about 13 (+30%). For practitioners, this answers two questions: how much such a platform costs and what actually runs on it.
What the server is and how fast it responds
The build includes an AMD EPYC 7452 processor, a HUANANZHI H12D 8D motherboard, 64 GB of DDR4 (4 sticks of 16 GB), two Nvidia V100 16 GB cards in SXM2 format, and a 1 TB Samsung PM9A1 SSD. Water cooling added roughly 50 thousand rubles on top, and the author advises against copying it if you can do without custom water cooling. Prices are as of the purchase six months ago.
The EPYC was chosen for its 128 PCIe 4.0 lanes and eight DDR4 channels. According to the author, this board is the only one for 2nd and 3rd generation EPYC with four x16 slots at a reasonable price.
The Supermicro h11ssl-i has only three slots, and the other options start at 70 thousand. The memory overclocked to 2933 without trouble.
Large MoE models run on this hardware thanks to the Strata and Maya projects, which use RAM sparingly and load weights from the SSD on the fly. Speeds (the first number is prompt processing, the second is generation, both in tokens per second):
- Qwen 3.8 27B: 800–1000 and 50–70;
- Qwen 3.8 Flash Next (125B): 500–1400 and 50–70;
- GLM 5.3 Flash (320B): 200–600 and 15–20;
- DeepSeek v4.1 Flash (763B): 180–250 and 7–15. It technically works but is too slow, so it did not take part in the tests.
Everything ran on Windows 10 through opencode 1.19 with searxng web search. Qwen 27B ran through llama.cpp with a context of 196 608 tokens and MTP speculative decoding, which noticeably speeds up code writing.
How the models handled the tasks
The first task was a console chat reader in ANSI C using the WolfSSL library. GLM, Qwen 27B and Qwen Flash Next produced working code, although 27B had to be told manually that the code did not work on Windows. Muse Spark found the right address, but the project did not build, and it disabled TLS verification. Gemma 4 picked the wrong URL and called curl via popen instead of WolfSSL.
The second task was to speed up the esp32-ai engine on the ESP32-S3 using esp-idf and vector instructions. Flash Next got stuck and made no progress even after several context compressions. Qwen 27B reached ~13 tok/s (+30%) using functions from the ESP-DSP library. GLM 5.3 reached ~20 tok/s (+100%) by writing assembly for the PIE extension. Both versions were verified: the logits match the original engine byte for byte. All the models optimized the head block. GLM ended up with almost the same optimizations as Fable 5, which delivered 22–31 tok/s. The difference is that Fable also sped up the attn block.
Where the results are weaker
The nickname generator was a failure in creativity. Both Qwen models produced pseudo-words, GLM only phrases, and all of them dressed the results up in a 2012 style: "ледянойщит9", "обсидианель", "supernovaRoot". The author expected at least Markov chains, but admits that the spec was vague. There are almost no complaints about the technical execution.
In the contradictory task of animating the Mandelbrot fractal (1080p, 60 fps, ANSI C, GPU on the Blackwell architecture), C89 combines poorly with a GPU, since CUDA is C++. According to the author, 27B at least managed to draw the fractal.
Limitations and takeaway
These are the personal observations of a single author: each task was run once, and the assessments are partly subjective. Model names are given as in the source. The practical takeaway: on a build costing ~113 thousand rubles, local models can solve C and embedded tasks without a detailed spec. For complex low-level optimization, GLM 5.3 showed the best result, while Qwen 3.8 27B is 3–4 times faster at generation and suits quick iterations. The output of any model is still worth verifying.
- Nov 19, 2026MLPerf Training: 10,240-GPU clusters and 49% faster training in six months
- Nov 19, 2026CoreWeave comes in up to 65% cheaper than hyperscalers on a three-year Blackwell cluster
- Nov 19, 2026Open-Source Rust Video Editor FilmCraft Hits 18.5K GitHub Stars and Lets AI Agents Drive the Engine
- Nov 19, 2026Skyflow in AgentCore Gateway hides personal data from AI agents' tools
Key Questions Answered
What the server is and how fast it responds?
The build includes an AMD EPYC 7452 processor, a HUANANZHI H12D 8D motherboard, 64 GB of DDR4 (4 sticks of 16 GB), two Nvidia V100 16 GB cards in SXM2 format, and a 1 TB Samsung PM9A1 SSD. Water cooling added roughly 50 thousand rubles on top, and the author advises against copying it if you can do without custom water cooling. Prices are as of the purchase six months ago. The EPYC was chosen for its 128 PCIe 4.0 lanes and eight DDR4 channels.
How the models handled the tasks?
The first task was a console chat reader in ANSI C using the WolfSSL library. GLM, Qwen 27B and Qwen Flash Next produced working code, although 27B had to be told manually that the code did not work on Windows. Muse Spark found the right address, but the project did not build, and it disabled TLS verification. Gemma 4 picked the wrong URL and called curl via popen instead of WolfSSL. The second task was to speed up the esp32-ai engine on the ESP32-S3 using esp-idf and vector instructions.
Where the results are weaker?
The nickname generator was a failure in creativity. Both Qwen models produced pseudo-words, GLM only phrases, and all of them dressed the results up in a 2012 style: "ледянойщит9", "обсидианель", "supernovaRoot". The author expected at least Markov chains, but admits that the spec was vague. There are almost no complaints about the technical execution.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.
