Ollama on Mac Switches to MLX: 1851 Tokens/s Prompt Processing in Version 0.19
Ollama, a tool for running neural networks locally, has moved to MLX in its preview release. MLX is Apple's framework, which makes efficient use of the shared memory pool across the CPU and GPU. For local agents such as Claude Code, that means less waiting: Ollama 0.19 claims 1851 tokens/s for prompt processing and 134 tokens/s for response generation. These numbers come from a single model with int4 quantization, so test your own workload on a Mac with more than 32 GB of memory.
AI-processed from Ollama Blog; edited by Hamidun News
If you have run a coding agent on your Mac, you know the feeling: you send a request and wait while the model "reads" a long context, and only then does it start answering. The team behind Ollama, a popular tool for running neural networks locally, has released a preview version that targets exactly this pause. On machines with Apple Silicon chips, Ollama is now built on MLX, Apple's own machine learning framework designed for unified memory, where the CPU and GPU work from a single shared pool and do not copy data back and forth.
The headline figures of the release apply to Ollama 0.19: 1851 tokens per second for prompt processing (this stage is called prefill) and 134 tokens per second for response generation (decode), using int4 quantization, which compresses the model's weights to 4 bits. A token is a chunk of text, roughly a syllable or a short word. The first number shows how quickly the model takes in your request and context; the second shows how quickly it types out the answer.
What changed under the hood
Previously, Ollama on Macs relied on a different implementation. Now the foundation is MLX. According to the developers, all Apple Silicon devices get the speedup. On the newer M5, M5 Pro, and M5 Max chips, Ollama additionally uses the neural accelerators in the GPU. They reduce time to first token (TTFT), the pause between sending a request and the start of the response, and increase generation speed.
The measurements were taken on March 29, 2026, using Alibaba's Qwen3.5-35B-A3B model. The new implementation with NVFP4 quantization was compared against the previous Ollama 0.18 and the Q4_K_M format. The authors describe the 1851 and 134 tokens/s figures as expected for Ollama 0.19. These are the developers' own data from a single model-and-hardware combination, so results on your Mac will differ.
NVFP4 and caching for agents
The second news item is support for NVIDIA's NVFP4 format. It stores model weights in compressed form while preserving accuracy and reducing the load on memory and disk. The practical implication: more and more inference providers are running models in NVFP4. If you use the same format locally, your responses will be closer to what you would get in production. Ollama will also be able to run models optimized with NVIDIA's tools. Other precision formats are promised for later.
The third improvement is caching, and it matters for agents. Imagine Claude Code branching into several lines of work that share one system prompt and tool description. Previously, that shared "header" had to be processed again each time. Now Ollama reuses the cache across conversations: less memory is occupied and more cache hits occur when branching. In addition, Ollama saves cache snapshots at "smart" points in the prompt, so it does not recompute the text from the beginning. Shared prefixes persist longer, even when old branches are evicted from memory.
How to try it and the limitations
The preview accelerates a single model: Qwen3.5-35B-A3B with sampling parameters tuned for programming. It requires a Mac with more than 32 GB of unified memory. Launch commands:
- ollama launch claude --model qwen3.5:35b-a3b-coding-nvfp4
- ollama launch openclaw --model qwen3.5:35b-a3b-coding-nvfp4
- ollama run qwen3.5:35b-a3b-coding-nvfp4
The limitations are clear: this is a preview, the list of supported models is short, and owners of Macs with 16 or 24 GB of memory will not be able to use this configuration. For those who have fine-tuned their own models on supported architectures, the team promises to simplify importing them. For now, it will keep expanding the list of supported architectures.
What developers should do
If you use local agents such as Claude Code, OpenClaw, or Codex on a Mac with a lot of memory, run your usual workload on the new engine. Measure two metrics separately: time to first token and generation speed. For agents that constantly resend long context, the caching gain may matter more than the increase in response typing speed. Ollama is clearly betting on the combination of MLX, NVFP4, and open Qwen models. In its acknowledgments, the team names the MLX, NVIDIA, GGML, and llama.cpp teams, as well as Alibaba Qwen.
- Nov 21, 2026NAND flash for SSDs drops 19–32% in spot deals — drives could get cheaper
- Nov 21, 2026Unsloth Updates Compressed Qwen3.8-27B: Accuracy Up 5–10% at the Same Size
- Nov 21, 2026AI Writes Vulnerable Code in 44% of Tests: Positive Technologies Gives Agents a Scanner
- Nov 21, 2026Sber Releases a 14,990-Ruble Speaker with GigaChat That Understands Voice Without Internet
Key Questions Answered
What changed under the hood?
Previously, Ollama on Macs relied on a different implementation. Now the foundation is MLX. According to the developers, all Apple Silicon devices get the speedup. On the newer M5, M5 Pro, and M5 Max chips, Ollama additionally uses the neural accelerators in the GPU. They reduce time to first token (TTFT), the pause between sending a request and the start of the response, and increase generation speed. The measurements were taken on March 29, 2026, using Alibaba's Qwen3.5-35B-A3B model.
What are the key details on NVFP4 and caching for agents?
The second news item is support for NVIDIA's NVFP4 format. It stores model weights in compressed form while preserving accuracy and reducing the load on memory and disk. The practical implication: more and more inference providers are running models in NVFP4. If you use the same format locally, your responses will be closer to what you would get in production. Ollama will also be able to run models optimized with NVIDIA's tools. Other precision formats are promised for later.
How to try it and the limitations?
The preview accelerates a single model: Qwen3.5-35B-A3B with sampling parameters tuned for programming. It requires a Mac with more than 32 GB of unified memory. Launch commands: ollama launch claude --model qwen3.5:35b-a3b-coding-nvfp4 ollama launch openclaw --model qwen3.5:35b-a3b-coding-nvfp4 ollama run qwen3.5:35b-a3b-coding-nvfp4 The limitations are clear: this is a preview, the list of supported models is short, and owners of Macs with 16 or 24 GB of memory will not be able to use this configuration.
Build & Ship Production AI Agents Without Boilerplate Code
Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.