Ollama speeds up Gemma 4 on Mac by nearly 90% in Aider coding-agent test
Ollama 0.31 has sped up Gemma 4 on Macs with Apple Silicon by nearly 90%. In the Aider polyglot test, where a coding agent solves real programming tasks, the model outputs tokens (pieces of text) faster. A small draft model guesses several tokens ahead, and the large model checks them in a single pass and keeps the correct ones, so the answer does not change. Code is full of brackets and templates, so the guesses often land. For local agents, this means fewer pauses between steps.
AI-processed from Ollama Blog; edited by Hamidun News
A local coding agent slows down not because of a weak model, but because of pauses: it reads files, calls tools, thinks again, and every second of waiting adds up to minutes. The Ollama team (software for running neural networks on your own computer) has released version 0.31, in which Google's Gemma 4 model on Macs with Apple Silicon chips generates tokens, that is, pieces of text, nearly 90% faster. The figure comes from the Aider polyglot test, in which a real coding agent solves real programming tasks rather than synthetic puzzles. The model's output stays the same, and the speedup is enabled by default.
How multi-token prediction works
A standard language model produces text one token at a time: it thinks, writes a fragment, then thinks again. Multi-token prediction (MTP) breaks this queue. Gemma 4 ships with a small, fast draft model. It runs alongside the main model and proposes several next tokens at once.
The main model then checks the whole batch of proposals in a single pass and keeps the tokens it agrees with. This is like a trainee drafting several lines of code while a senior developer accepts the correct ones and strikes out the wrong ones at a glance. If the guesses are right, the model commits several tokens at the cost of one.
Code suits this scheme almost perfectly. It is full of closing brackets, repeated variable names, and boilerplate constructs, so the draft model guesses often. That is why the gain is most noticeable for coding agents that query the model constantly.
Ollama's developers stress separately that the model's output does not change. The main model still decides which tokens to accept, so this is not a simplified version of Gemma 4 but the same result, obtained faster.
Three changes that deliver the speedup
First: automatic draft-length selection. There is no universal number of tokens worth proposing at a time. It depends on the model, the quantization (weight compression), the hardware, and how predictable the text is at that moment. If you propose too few, speed is left on the table. If you propose too many, time goes to verifying rejected guesses, and MTP becomes slower than ordinary generation. Ollama tracks how often guesses are accepted and how long verification takes, and selects the length that yields more tokens per second. When guesses stop landing, the engine falls back to normal one-token-at-a-time mode, so there is no slowdown.
Second: the speculative generation loop itself. The draft model predicts a token, feeds it back to itself, and repeats until it has a short chain. The main model verifies the whole chain. All of this, including token selection, runs on the GPU in one pass without returning to the CPU. By this point, rejected tokens have already been written to the cache, meaning the model's working state, so the previous text does not have to be recomputed. The engine therefore records a rollback point before each proposal, and on rejection it simply reverts to the last accepted token. Nothing that came before is recomputed.
Third: a new kernel for verifying the batch. The main cost lies not in the draft but in verification: the main model runs over a batch of 2–8 tokens. Matrix-multiplication kernels are usually built for either a single token or a large batch, and this intermediate size is awkward for both cases. Ollama added to MLX (Apple's machine-learning framework for Macs) a kernel that reads and unpacks each block of weights once and applies it to the entire batch, instead of rereading the weights for every token. The computation is the same; the speedup comes from avoiding redundant work. The kernel is available in MLX for other models too, not only for Gemma 4 in Ollama.
What matters in practice
The measurements were made on Gemma 4 12B in nvfp4 quantization on the M5 Max chip. The authors themselves caveat that the MTP gain depends heavily on the workload, and that a synthetic test can be tuned to produce any result. So you should not expect exactly 90% on every task: on unpredictable text, guesses will miss more often and the gain will shrink. The automatic length selection, however, should protect against slowdowns.
To try it, you need Ollama 0.31 or newer for macOS. You can run Gemma 4 with an agent using the command ollama launch claude --model gemma4:12b-mlx. If you downloaded Gemma 4 earlier, update the model with ollama pull gemma4:12b-mlx. According to Ollama, the scheme also works with Codex, Droid, OpenCode, Copilot, and other tools.
Gemma 4 is the first model to receive this speedup, and Ollama promises to extend it to other models. For those running coding agents locally on a Mac, this is a free way to cut waiting time without switching models and without losing quality.
- Nov 21, 2026VibeCBS collects project nicknames in advance: the agent finds 2.8 times more documents
- Nov 21, 2026FDE job postings grew 4.6x in a year: what a forward deployed engineer actually does at the client's site
- Nov 21, 2026Sakana AI Opens Applied Research Lab for Market and Economic Modeling
- Nov 21, 2026Meta Is Reportedly Preparing the $300 Muse Charm AI Tamagotchi With a Year of Verizon 5G
Key Questions Answered
How multi-token prediction works?
A standard language model produces text one token at a time: it thinks, writes a fragment, then thinks again. Multi-token prediction (MTP) breaks this queue. Gemma 4 ships with a small, fast draft model. It runs alongside the main model and proposes several next tokens at once. The main model then checks the whole batch of proposals in a single pass and keeps the tokens it agrees with.
What are the key details on Three changes that deliver the speedup?
First: automatic draft-length selection. There is no universal number of tokens worth proposing at a time. It depends on the model, the quantization (weight compression), the hardware, and how predictable the text is at that moment. If you propose too few, speed is left on the table. If you propose too many, time goes to verifying rejected guesses, and MTP becomes slower than ordinary generation.
What matters in practice?
The measurements were made on Gemma 4 12B in nvfp4 quantization on the M5 Max chip. The authors themselves caveat that the MTP gain depends heavily on the workload, and that a synthetic test can be tuned to produce any result. So you should not expect exactly 90% on every task: on unpredictable text, guesses will miss more often and the gain will shrink. The automatic length selection, however, should protect against slowdowns. To try it, you need Ollama 0.31 or newer for macOS.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.