Qwen 3.5 on a MacBook Pro: Eight Local Servers Compared for Team Use

Running large models locally on a Mac stopped being an enthusiast's toy a while ago, but the Qwen 3.5 35B story shows there is a wide gap between "it runs" and "it is fit for a team API." The author took a MacBook Pro M2 Max with 64 GB of memory and tested not the model itself but the infrastructure around it: which MLX server holds up under a real workload, does not print flattering numbers in its logs, and does not fall apart the moment two users show up at once.

For the test, a separate Python harness was built and eight local servers were run through it, all positioned as a quick way to stand up an API on top of MLX models on macOS.

The testing did not rely on one convenient question. It used a set of eight prompts of different types and lengths, including AIME-level tasks and long inputs of up to 52 thousand tokens. Each scenario was run five times to smooth out random spikes and give a more honest picture of latency, generation speed and overall behavior under load. The emphasis was on measuring not peak lab speed but behavior close to production use: with streamed responses, network overhead and repeatable measurement conditions.

With a single user there was almost no drama: the top three showed similar results, and in short sessions the difference between them looks more cosmetic than real. That is exactly why the marketing promises in READMEs are so misleading. Looking only at a single request, it seems that almost any modern MLX server is good enough for everyday work. But that conclusion breaks down as soon as a local model goes from a personal tool to a team service, where requests start to overlap in time.

The most revealing stage of the test was the parallel load of two requests. This is where the real gap between solutions appeared. Four of six frameworks effectively slid into a queue and served requests almost sequentially, even though they looked multithreaded from the outside. Another server kept parallelism in name only and dropped to a factor of 0.85x, meaning the second request got in the way more than it helped use the hardware. Only one participant showed a genuine 2.17x speedup, which begins to look like acceptable behavior for a local team API, where the goal is not just to answer one user quickly but to hold several requests without a dramatic slowdown.

Along the way, problems surfaced that matter more than the dry numbers in a table. In one case the author ran into quadratic attention, which in 2026 can still badly degrade behavior on long contexts. In another, phantom 14,000 tokens/sec appeared not because of some magical optimization but because of a single line in an SSE parser that skewed the measurement. The author also mentions a zombie process that left about 20 GB of RAM occupied, a risk READMEs prefer to stay silent about. For anyone planning local production use, these are not trifles: such bugs hurt service predictability, monitoring and support costs more than a difference of a few percent in raw speed.

The practical value of the work is that it shifts the focus from attractive promises to usage scenarios. If one developer needs the model for occasional requests, ease of setup and baseline speed are what matter. But for a team API with parallelism, long contexts and a need to recover quickly from failures, choosing a server by its README is already risky. The benchmark makes a simple point: a local stack for Qwen 3.5 should be evaluated as infrastructure, not as a demo. Otherwise you can end up with a system that looks "fast" in single-request tests but, in live use, turns a powerful MacBook into an expensive queue of requests.