TensorFold 1.2.0 boosts Qwen3.8-27B throughput on M3 Ultra by 44.2% at 16 streams

The author of the local inference engine TensorFold has announced the release of version 1.2.0. The headline change for Mac owners is that Qwen3.8-27B with 4-bit MLX weights and DFlash2 mode is noticeably faster than in 1.1.9. The difference is most visible under parallel load: on an M3 Ultra, aggregate throughput at 16 streams rose by 44.2%. The release also adds support for new models on Metal, CUDA and Intel Arc.

Measurements on M3 Ultra and M5 Ultra

The comparison uses the same model throughout: Qwen3.8-27B, 4-bit MLX weights, DFlash2. The baseline is TensorFold 1.1.9.

• M3 Ultra: single-stream decoding is 14.4% faster, and aggregate throughput at 16 streams is 44.2% higher.
• M5 Ultra: a single stream is 5.8% faster, and aggregate throughput at 16 streams is 25.2% higher.

The numbers make the gap between the scenarios clear. A single stream is limited by how fast each token is generated, so the gain there is modest. When several requests run at once, the engine keeps the hardware busier, and the gain is several times larger. For a local server that handles multiple users or parallel agents, this matters more than the speed of a single chat.

Curiously, the relative gain on the newer M5 Ultra chip is smaller than on the M3 Ultra. The author does not explain why. Judging by the numbers, the M5 Ultra has more performance headroom, so the optimizations yield a smaller percentage. But that is a guess, not a statement from the developer. The announcement gives no absolute tokens-per-second figures, so the two chips cannot be compared with each other using this data. We only see the improvement over the previous version on each one.

The author also notes that in the measured scenarios the speedup came with matching tokens. The output text stayed the same as in 1.1.9. That means the gain came without a quality trade-off.

New models and hardware

Besides the speedup, the release expands the list of supported configurations:

• MiniMax-M3.1-Flash on Metal, that is, on Mac.
• An NVFP4 version of Qwen3.6-35B-A3B on CUDA.
• The dense Qwen3.8-27B model in EXL3 format on CUDA.
• Gemma 4 E4B on Intel Arc Pro B70.

The last item is interesting because TensorFold is moving beyond the Apple and Nvidia pairing. Intel Arc Pro GPUs are rarely mentioned in local inference projects, and a working configuration gives an alternative to anyone building a workstation without Nvidia. The announcement does not say how fast Gemma 4 E4B runs on the Arc Pro B70, so this is about support, not about speed results.

What changed in multimodal answers

The release comes with a caveat that anyone using vision models should note. Qwen-VL answers about images may differ from those produced by version 1.1.9. The reason is that image preprocessing now matches the checkpoint's own processor. Previously it apparently diverged from the reference, so the difference more likely corrects the old behavior than introduces a new one. But if you compare results on a fixed set of images or run regression tests, you will have to regenerate the outputs after the update.

This caveat applies specifically to Qwen-VL. The author confirms the token match for the measured speedups, and those relate to the text model Qwen3.8-27B.

Community and practical takeaway

The author thanked nineteen GitHub contributors for pull requests, reviews, testing, bug reports and hardware access: AbrahamCete, akol1, alexderz, Arminova, BadAd84, BlivionIaG, BobClawblaw, fche, gilby, jrb-ai, jschmied, laudney, Mirrdhyn, mrv777, ppardus, simon-lin88, ThomasHalwax, valacroix and wojo. The separate thanks for hardware access explains where support for such different platforms comes from: one person is unlikely to own an M3 Ultra, an M5 Ultra, Nvidia cards and an Intel Arc Pro.

The practical takeaway is simple. For Mac Studio owners with an M3 Ultra who run Qwen3.8-27B in 4-bit MLX, the update has the most noticeable effect with several simultaneous requests. On the M5 Ultra the gain is smaller, but it is there. The caveats: the figures come from the author's announcement, there are no independent measurements, and neither the methodology nor the absolute speeds are disclosed. Before moving production to 1.2.0, run your own workload and, if you use Qwen-VL, recheck the image answers.