Ollama Speeds Up NVIDIA Model Runs by Up to 20% and Turns On Vulkan for AMD and Intel
Ollama, a program for running neural networks on your own computer, has sped up its handling of the GGUF format. On NVIDIA hardware, the gain reaches 20%, based on a test with Gemma 4 26B on an RTX 5090 using Q4_K_M weight compression. Vulkan, a general-purpose graphics interface, is now enabled by default, so AMD and Intel cards work without vendor-specific libraries. Running local models is now easier on the hardware you already own.
AI-processed from Ollama Blog; edited by Hamidun News
Ollama, a program that runs neural networks directly on your computer rather than in someone else's cloud, has released an update with two notable changes. First, models now run up to 20% faster on NVIDIA graphics cards, according to a test on Gemma 4 26B, an RTX 5090 GPU, and Q4_K_M weight compression (a way to shrink a model so it fits in video memory). Second, Vulkan is enabled by default.
Vulkan is a general-purpose graphics interface that works with cards from different manufacturers. As a result, AMD and Intel owners get GPU acceleration without installing vendor-specific libraries. Ollama also works better with GGUF, the unified file format in which local models are published on Hugging Face.
For developers, this means less fiddling with drivers and more models that launch with a single command.
Running neural networks on your own hardware has long stopped being a hobby for enthusiasts. It is a way to avoid paying for APIs, to keep code and documents from leaving your machine, and to experiment with open models without hassle. But local inference has always had two sore spots: not every graphics card was properly supported, and getting a model from Hugging Face to work with the tool often took a long time. The latest Ollama update addresses both.
Faster on NVIDIA and broader hardware support
The first change is speed. According to Ollama, performance on NVIDIA hardware rose by up to 20%. This is not magic from the developers alone: the optimizations involved teams from NVIDIA and the llama.cpp project, which underpins much of the local model ecosystem. The benchmark ran Gemma 4 26B on an RTX 5090 with Q4_K_M quantization. Quantization compresses model weights to lower precision so the model fits in video memory and runs faster. Q4_K_M is one of the most popular trade-offs between size and quality.
The second change is that Vulkan is now enabled by default. It lets AMD and Intel devices get GPU acceleration without installing the manufacturers' own libraries. Previously, this setup often turned into a quest: finding a compatible driver version, installing extra components, and checking that the hardware was detected. Now, by design, the model should simply run on the graphics card.
In parallel, Ollama expanded its compatibility with the GGUF ecosystem. Support was added for Apple silicon, and now for a wider range of hardware, so more models work out of the box.
How to run a model from Hugging Face
The process is simple. First, download a GGUF file or a folder of such files. Then create a Modelfile, a short text file that specifies the path to the model, for example: FROM ./my-model.Q4_K_M.gguf. Next, run the command ollama create -f Modelfile my-model and launch the resulting model under your chosen name.
This matters in practice: Ollama makes it easier to bring in not only the "official" builds but also fine-tuned versions that the community and third-party teams publish. If you need a model tuned for your task, such as code in a specific language or working with documents, you do not have to wait for it to appear in the catalog.
Agents and assistants on a local model
Another detail is inherited capabilities. If a GGUF model supports tool calling, that capability is preserved in Ollama. It can then be connected to familiar coding agents and personal assistants with one command: ollama launch claude --model my-model, along with similar options for hermes and openclaw. To check whether a specific file supports tool calling, look for the corresponding marker in the model's description.
This is useful for people building agent pipelines who want to run draft workflows locally. Debugging happens without per-token charges, and the data stays on the machine.
What to keep in mind
The "up to 20%" figure applies to one specific configuration: Gemma 4 26B, RTX 5090, and Q4_K_M. Other models, cards, and quantizations may show different gains, so it is better to compare on your own workloads. The announcement gives no performance numbers for Vulkan on AMD and Intel cards: it claims broader coverage, not records. Finally, tool-calling quality depends on the model itself. Not every fine-tuned build handles agentic scenarios reliably, even when the format formally supports them.
The authors also thank Georgi Gerganov and the llama.cpp maintainers, along with hardware partners NVIDIA, AMD, Qualcomm, and Intel, for their work on optimization within the GGML ecosystem. This is a good sign: running models locally is increasingly a shared engineering effort among chip makers, not a niche topic.
The practical takeaway: if you have put off experiments with local models because of finicky hardware or complicated file setup, now is a good time to try. Update Ollama, download a GGUF from Hugging Face, and measure the speed on your own graphics card. It takes an evening, and the result will help you decide which tasks can move from cloud APIs to your own machine.
- Nov 21, 2026NAND flash for SSDs drops 19–32% in spot deals — drives could get cheaper
- Nov 21, 2026Unsloth Updates Compressed Qwen3.8-27B: Accuracy Up 5–10% at the Same Size
- Nov 21, 2026AI Writes Vulnerable Code in 44% of Tests: Positive Technologies Gives Agents a Scanner
- Nov 21, 2026Sber Releases a 14,990-Ruble Speaker with GigaChat That Understands Voice Without Internet
Key Questions Answered
What are the key details on Faster on NVIDIA and broader hardware support?
The first change is speed. According to Ollama, performance on NVIDIA hardware rose by up to 20%. This is not magic from the developers alone: the optimizations involved teams from NVIDIA and the llama.cpp project, which underpins much of the local model ecosystem. The benchmark ran Gemma 4 26B on an RTX 5090 with Q4_K_M quantization. Quantization compresses model weights to lower precision so the model fits in video memory and runs faster. Q4_K_M is one of the most popular trade-offs between size and quality.
How to run a model from Hugging Face?
The process is simple. First, download a GGUF file or a folder of such files. Then create a Modelfile, a short text file that specifies the path to the model, for example: FROM ./my-model.Q4_K_M.gguf. Next, run the command ollama create -f Modelfile my-model and launch the resulting model under your chosen name. This matters in practice: Ollama makes it easier to bring in not only the "official" builds but also fine-tuned versions that the community and third-party teams publish.
What are the key details on Agents and assistants on a local model?
Another detail is inherited capabilities. If a GGUF model supports tool calling, that capability is preserved in Ollama. It can then be connected to familiar coding agents and personal assistants with one command: ollama launch claude --model my-model, along with similar options for hermes and openclaw. To check whether a specific file supports tool calling, look for the corresponding marker in the model's description. This is useful for people building agent pipelines who want to run draft workflows locally.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.