Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
InfrastructureInsight · On RadarOllama Blog→ original

Ollama on Mac Switches to MLX: 1851 Tokens/s Prompt Processing in Version 0.19

Ollama, a tool for running neural networks locally, has moved to MLX in its preview release. MLX is Apple's framework, which makes efficient use of the shared memory pool across the CPU and GPU. For local agents such as Claude Code, that means less waiting: Ollama 0.19 claims 1851 tokens/s for prompt processing and 134 tokens/s for response generation. These numbers come from a single model with int4 quantization, so test your own workload on a Mac with more than 32 GB of memory.

AI-processed from Ollama Blog; edited by Hamidun News
Ollama on Mac Switches to MLX: 1851 Tokens/s Prompt Processing in Version 0.19
Source: Ollama Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceOllama Blog ↗
Editor-in-Chief
Key metric #1A token is roughly a syllable; prefill is prompt processing, decode is response generation
Key metric #2The shared system prompt cache is now reused across conversations
Key metric #3NVIDIA's NVFP4 format is supported, bringing responses closer to cloud production

If you have run a coding agent on your Mac, you know the feeling: you send a request and wait while the model "reads" a long context, and only then does it start answering. The team behind Ollama, a popular tool for running neural networks locally, has released a preview version that targets exactly this pause. On machines with Apple Silicon chips, Ollama is now built on MLX, Apple's own machine learning framework designed for unified memory, where the CPU and GPU work from a single shared pool and do not copy data back and forth.

The headline figures of the release apply to Ollama 0.19: 1851 tokens per second for prompt processing (this stage is called prefill) and 134 tokens per second for response generation (decode), using int4 quantization, which compresses the model's weights to 4 bits. A token is a chunk of text, roughly a syllable or a short word. The first number shows how quickly the model takes in your request and context; the second shows how quickly it types out the answer.

What changed under the hood

Previously, Ollama on Macs relied on a different implementation. Now the foundation is MLX. According to the developers, all Apple Silicon devices get the speedup. On the newer M5, M5 Pro, and M5 Max chips, Ollama additionally uses the neural accelerators in the GPU. They reduce time to first token (TTFT), the pause between sending a request and the start of the response, and increase generation speed.

The measurements were taken on March 29, 2026, using Alibaba's Qwen3.5-35B-A3B model. The new implementation with NVFP4 quantization was compared against the previous Ollama 0.18 and the Q4_K_M format. The authors describe the 1851 and 134 tokens/s figures as expected for Ollama 0.19. These are the developers' own data from a single model-and-hardware combination, so results on your Mac will differ.

NVFP4 and caching for agents

The second news item is support for NVIDIA's NVFP4 format. It stores model weights in compressed form while preserving accuracy and reducing the load on memory and disk. The practical implication: more and more inference providers are running models in NVFP4. If you use the same format locally, your responses will be closer to what you would get in production. Ollama will also be able to run models optimized with NVIDIA's tools. Other precision formats are promised for later.

The third improvement is caching, and it matters for agents. Imagine Claude Code branching into several lines of work that share one system prompt and tool description. Previously, that shared "header" had to be processed again each time. Now Ollama reuses the cache across conversations: less memory is occupied and more cache hits occur when branching. In addition, Ollama saves cache snapshots at "smart" points in the prompt, so it does not recompute the text from the beginning. Shared prefixes persist longer, even when old branches are evicted from memory.

How to try it and the limitations

The preview accelerates a single model: Qwen3.5-35B-A3B with sampling parameters tuned for programming. It requires a Mac with more than 32 GB of unified memory. Launch commands:

  • ollama launch claude --model qwen3.5:35b-a3b-coding-nvfp4
  • ollama launch openclaw --model qwen3.5:35b-a3b-coding-nvfp4
  • ollama run qwen3.5:35b-a3b-coding-nvfp4

The limitations are clear: this is a preview, the list of supported models is short, and owners of Macs with 16 or 24 GB of memory will not be able to use this configuration. For those who have fine-tuned their own models on supported architectures, the team promises to simplify importing them. For now, it will keep expanding the list of supported architectures.

What developers should do

If you use local agents such as Claude Code, OpenClaw, or Codex on a Mac with a lot of memory, run your usual workload on the new engine. Measure two metrics separately: time to first token and generation speed. For agents that constantly resend long context, the caching gain may matter more than the increase in response typing speed. Ollama is clearly betting on the combination of MLX, NVFP4, and open Qwen models. In its acknowledgments, the team names the MLX, NVIDIA, GGML, and llama.cpp teams, as well as Alibaba Qwen.

Story

Key Questions Answered

What changed under the hood?

Previously, Ollama on Macs relied on a different implementation. Now the foundation is MLX. According to the developers, all Apple Silicon devices get the speedup. On the newer M5, M5 Pro, and M5 Max chips, Ollama additionally uses the neural accelerators in the GPU. They reduce time to first token (TTFT), the pause between sending a request and the start of the response, and increase generation speed. The measurements were taken on March 29, 2026, using Alibaba's Qwen3.5-35B-A3B model.

What are the key details on NVFP4 and caching for agents?

The second news item is support for NVIDIA's NVFP4 format. It stores model weights in compressed form while preserving accuracy and reducing the load on memory and disk. The practical implication: more and more inference providers are running models in NVFP4. If you use the same format locally, your responses will be closer to what you would get in production. Ollama will also be able to run models optimized with NVIDIA's tools. Other precision formats are promised for later.

How to try it and the limitations?

The preview accelerates a single model: Qwen3.5-35B-A3B with sampling parameters tuned for programming. It requires a Mac with more than 32 GB of unified memory. Launch commands: ollama launch claude --model qwen3.5:35b-a3b-coding-nvfp4 ollama launch openclaw --model qwen3.5:35b-a3b-coding-nvfp4 ollama run qwen3.5:35b-a3b-coding-nvfp4 The limitations are clear: this is a preview, the list of supported models is short, and owners of Macs with 16 or 24 GB of memory will not be able to use this configuration.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Hands-On AI Engineering · Hamidun Academy

Build & Ship Production AI Agents Without Boilerplate Code

Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.

Start 72h Free Trial →Vibe-Coding Intensive
What do you think?
Loading comments…