Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarHabr AI→ original

Local GLM 5.3 Speeds Up AI on ESP32-S3 by 100%, Qwen 3.8 27B by 30%

The author built a home server with two Tesla V100s for 113 105 ₽ and gave local models tasks without a detailed spec. The main test: speed up a small language model on an ESP32-S3 microcontroller, where the engine was written in pure C. GLM 5.3 pushed generation to ~20 tokens per second (+100%), Qwen 3.8 27B to ~13 (+30%). So even a budget build can already handle low-level optimization with local models.

AI-processed from Habr AI; edited by Hamidun News
Local GLM 5.3 Speeds Up AI on ESP32-S3 by 100%, Qwen 3.8 27B by 30%
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
Key metric #1The logits of both versions match the original engine byte for byte
Key metric #2Qwen 3.8 Flash Next got stuck on the ESP32 task and made no progress
Key metric #3DeepSeek v4.1 Flash runs at 7–15 tok/s, so it did not take part in the tests
Local GLM 5.3 Speeds Up AI on ESP32-S3 by 100%, Qwen 3.8 27B by 30%
Featured Video / Demo
Click to load the embedded player (privacy-enhanced youtube-nocookie mode) or read the full breakdown below.Watch on YouTube ↗

Local neural networks are usually tested with the same tasks: "make a Minecraft clone" or "draw a Windows 11 interface." The author of an article on Habr decided that was boring. He built a server at home for 113 105 ₽ and gave the models fuzzy tasks close to vibe coding, where instead of a detailed spec the prompt is "make it good."

The most telling example is the ESP32-S3, a cheap microcontroller that runs a small language model. Its engine was written in pure C, and the models had to rewrite it to use the chip's vector instructions. Speed is measured in tokens per second: GLM 5.

3 reached about 20 (+100% over the original), Qwen 3.8 27B about 13 (+30%). For practitioners, this answers two questions: how much such a platform costs and what actually runs on it.

What the server is and how fast it responds

The build includes an AMD EPYC 7452 processor, a HUANANZHI H12D 8D motherboard, 64 GB of DDR4 (4 sticks of 16 GB), two Nvidia V100 16 GB cards in SXM2 format, and a 1 TB Samsung PM9A1 SSD. Water cooling added roughly 50 thousand rubles on top, and the author advises against copying it if you can do without custom water cooling. Prices are as of the purchase six months ago.

The EPYC was chosen for its 128 PCIe 4.0 lanes and eight DDR4 channels. According to the author, this board is the only one for 2nd and 3rd generation EPYC with four x16 slots at a reasonable price.

The Supermicro h11ssl-i has only three slots, and the other options start at 70 thousand. The memory overclocked to 2933 without trouble.

Large MoE models run on this hardware thanks to the Strata and Maya projects, which use RAM sparingly and load weights from the SSD on the fly. Speeds (the first number is prompt processing, the second is generation, both in tokens per second):

  • Qwen 3.8 27B: 800–1000 and 50–70;
  • Qwen 3.8 Flash Next (125B): 500–1400 and 50–70;
  • GLM 5.3 Flash (320B): 200–600 and 15–20;
  • DeepSeek v4.1 Flash (763B): 180–250 and 7–15. It technically works but is too slow, so it did not take part in the tests.

Everything ran on Windows 10 through opencode 1.19 with searxng web search. Qwen 27B ran through llama.cpp with a context of 196 608 tokens and MTP speculative decoding, which noticeably speeds up code writing.

How the models handled the tasks

The first task was a console chat reader in ANSI C using the WolfSSL library. GLM, Qwen 27B and Qwen Flash Next produced working code, although 27B had to be told manually that the code did not work on Windows. Muse Spark found the right address, but the project did not build, and it disabled TLS verification. Gemma 4 picked the wrong URL and called curl via popen instead of WolfSSL.

The second task was to speed up the esp32-ai engine on the ESP32-S3 using esp-idf and vector instructions. Flash Next got stuck and made no progress even after several context compressions. Qwen 27B reached ~13 tok/s (+30%) using functions from the ESP-DSP library. GLM 5.3 reached ~20 tok/s (+100%) by writing assembly for the PIE extension. Both versions were verified: the logits match the original engine byte for byte. All the models optimized the head block. GLM ended up with almost the same optimizations as Fable 5, which delivered 22–31 tok/s. The difference is that Fable also sped up the attn block.

Where the results are weaker

The nickname generator was a failure in creativity. Both Qwen models produced pseudo-words, GLM only phrases, and all of them dressed the results up in a 2012 style: "ледянойщит9", "обсидианель", "supernovaRoot". The author expected at least Markov chains, but admits that the spec was vague. There are almost no complaints about the technical execution.

In the contradictory task of animating the Mandelbrot fractal (1080p, 60 fps, ANSI C, GPU on the Blackwell architecture), C89 combines poorly with a GPU, since CUDA is C++. According to the author, 27B at least managed to draw the fractal.

Limitations and takeaway

These are the personal observations of a single author: each task was run once, and the assessments are partly subjective. Model names are given as in the source. The practical takeaway: on a build costing ~113 thousand rubles, local models can solve C and embedded tasks without a detailed spec. For complex low-level optimization, GLM 5.3 showed the best result, while Qwen 3.8 27B is 3–4 times faster at generation and suits quick iterations. The output of any model is still worth verifying.

Story

Key Questions Answered

What the server is and how fast it responds?

The build includes an AMD EPYC 7452 processor, a HUANANZHI H12D 8D motherboard, 64 GB of DDR4 (4 sticks of 16 GB), two Nvidia V100 16 GB cards in SXM2 format, and a 1 TB Samsung PM9A1 SSD. Water cooling added roughly 50 thousand rubles on top, and the author advises against copying it if you can do without custom water cooling. Prices are as of the purchase six months ago. The EPYC was chosen for its 128 PCIe 4.0 lanes and eight DDR4 channels.

How the models handled the tasks?

The first task was a console chat reader in ANSI C using the WolfSSL library. GLM, Qwen 27B and Qwen Flash Next produced working code, although 27B had to be told manually that the code did not work on Windows. Muse Spark found the right address, but the project did not build, and it disabled TLS verification. Gemma 4 picked the wrong URL and called curl via popen instead of WolfSSL. The second task was to speed up the esp32-ai engine on the ESP32-S3 using esp-idf and vector instructions.

Where the results are weaker?

The nickname generator was a failure in creativity. Both Qwen models produced pseudo-words, GLM only phrases, and all of them dressed the results up in a 2012 style: "ледянойщит9", "обсидианель", "supernovaRoot". The author expected at least Markov chains, but admits that the spec was vague. There are almost no complaints about the technical execution.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…