Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarOllama Blog→ original

Ollama Speeds Up NVIDIA Model Runs by Up to 20% and Turns On Vulkan for AMD and Intel

Ollama, a program for running neural networks on your own computer, has sped up its handling of the GGUF format. On NVIDIA hardware, the gain reaches 20%, based on a test with Gemma 4 26B on an RTX 5090 using Q4_K_M weight compression. Vulkan, a general-purpose graphics interface, is now enabled by default, so AMD and Intel cards work without vendor-specific libraries. Running local models is now easier on the hardware you already own.

AI-processed from Ollama Blog; edited by Hamidun News
Ollama Speeds Up NVIDIA Model Runs by Up to 20% and Turns On Vulkan for AMD and Intel
Source: Ollama Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceOllama Blog ↗
Editor-in-Chief
Up to 20% faster on NVIDIAthe optimizations came from NVIDIA and llama.cpp teams
Key metric #2Vulkan is on by default, so AMD and Intel work without extra libraries
Model from Hugging FaceGGUF file, Modelfile, and the ollama create command

Ollama, a program that runs neural networks directly on your computer rather than in someone else's cloud, has released an update with two notable changes. First, models now run up to 20% faster on NVIDIA graphics cards, according to a test on Gemma 4 26B, an RTX 5090 GPU, and Q4_K_M weight compression (a way to shrink a model so it fits in video memory). Second, Vulkan is enabled by default.

Vulkan is a general-purpose graphics interface that works with cards from different manufacturers. As a result, AMD and Intel owners get GPU acceleration without installing vendor-specific libraries. Ollama also works better with GGUF, the unified file format in which local models are published on Hugging Face.

For developers, this means less fiddling with drivers and more models that launch with a single command.

Running neural networks on your own hardware has long stopped being a hobby for enthusiasts. It is a way to avoid paying for APIs, to keep code and documents from leaving your machine, and to experiment with open models without hassle. But local inference has always had two sore spots: not every graphics card was properly supported, and getting a model from Hugging Face to work with the tool often took a long time. The latest Ollama update addresses both.

Faster on NVIDIA and broader hardware support

The first change is speed. According to Ollama, performance on NVIDIA hardware rose by up to 20%. This is not magic from the developers alone: the optimizations involved teams from NVIDIA and the llama.cpp project, which underpins much of the local model ecosystem. The benchmark ran Gemma 4 26B on an RTX 5090 with Q4_K_M quantization. Quantization compresses model weights to lower precision so the model fits in video memory and runs faster. Q4_K_M is one of the most popular trade-offs between size and quality.

The second change is that Vulkan is now enabled by default. It lets AMD and Intel devices get GPU acceleration without installing the manufacturers' own libraries. Previously, this setup often turned into a quest: finding a compatible driver version, installing extra components, and checking that the hardware was detected. Now, by design, the model should simply run on the graphics card.

In parallel, Ollama expanded its compatibility with the GGUF ecosystem. Support was added for Apple silicon, and now for a wider range of hardware, so more models work out of the box.

How to run a model from Hugging Face

The process is simple. First, download a GGUF file or a folder of such files. Then create a Modelfile, a short text file that specifies the path to the model, for example: FROM ./my-model.Q4_K_M.gguf. Next, run the command ollama create -f Modelfile my-model and launch the resulting model under your chosen name.

This matters in practice: Ollama makes it easier to bring in not only the "official" builds but also fine-tuned versions that the community and third-party teams publish. If you need a model tuned for your task, such as code in a specific language or working with documents, you do not have to wait for it to appear in the catalog.

Agents and assistants on a local model

Another detail is inherited capabilities. If a GGUF model supports tool calling, that capability is preserved in Ollama. It can then be connected to familiar coding agents and personal assistants with one command: ollama launch claude --model my-model, along with similar options for hermes and openclaw. To check whether a specific file supports tool calling, look for the corresponding marker in the model's description.

This is useful for people building agent pipelines who want to run draft workflows locally. Debugging happens without per-token charges, and the data stays on the machine.

What to keep in mind

The "up to 20%" figure applies to one specific configuration: Gemma 4 26B, RTX 5090, and Q4_K_M. Other models, cards, and quantizations may show different gains, so it is better to compare on your own workloads. The announcement gives no performance numbers for Vulkan on AMD and Intel cards: it claims broader coverage, not records. Finally, tool-calling quality depends on the model itself. Not every fine-tuned build handles agentic scenarios reliably, even when the format formally supports them.

The authors also thank Georgi Gerganov and the llama.cpp maintainers, along with hardware partners NVIDIA, AMD, Qualcomm, and Intel, for their work on optimization within the GGML ecosystem. This is a good sign: running models locally is increasingly a shared engineering effort among chip makers, not a niche topic.

The practical takeaway: if you have put off experiments with local models because of finicky hardware or complicated file setup, now is a good time to try. Update Ollama, download a GGUF from Hugging Face, and measure the speed on your own graphics card. It takes an evening, and the result will help you decide which tasks can move from cloud APIs to your own machine.

Story

Key Questions Answered

What are the key details on Faster on NVIDIA and broader hardware support?

The first change is speed. According to Ollama, performance on NVIDIA hardware rose by up to 20%. This is not magic from the developers alone: the optimizations involved teams from NVIDIA and the llama.cpp project, which underpins much of the local model ecosystem. The benchmark ran Gemma 4 26B on an RTX 5090 with Q4_K_M quantization. Quantization compresses model weights to lower precision so the model fits in video memory and runs faster. Q4_K_M is one of the most popular trade-offs between size and quality.

How to run a model from Hugging Face?

The process is simple. First, download a GGUF file or a folder of such files. Then create a Modelfile, a short text file that specifies the path to the model, for example: FROM ./my-model.Q4_K_M.gguf. Next, run the command ollama create -f Modelfile my-model and launch the resulting model under your chosen name. This matters in practice: Ollama makes it easier to bring in not only the "official" builds but also fine-tuned versions that the community and third-party teams publish.

What are the key details on Agents and assistants on a local model?

Another detail is inherited capabilities. If a GGUF model supports tool calling, that capability is preserved in Ollama. It can then be connected to familiar coding agents and personal assistants with one command: ollama launch claude --model my-model, along with similar options for hermes and openclaw. To check whether a specific file supports tool calling, look for the corresponding marker in the model's description. This is useful for people building agent pipelines who want to run draft workflows locally.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…