Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableLatent Space (swyx)→ original

llama.cpp 1.0: Stable Interface, 35 Functions Removed, and a Cache for Large MoE Models

llama.cpp, the engine apps use to run neural networks locally, has reached version 1.0. The authors have declared the embedding API stable and removed 35 deprecated functions from it. Third-party programs should no longer break after every update. For MoE models, where only some of the "experts" work on each word and not all of them fit in video memory, they added a cache for the most frequently used ones. If you embed the engine, check your calls to the removed functions.

AI-processed from Latent Space (swyx); edited by Hamidun News
llama.cpp 1.0: Stable Interface, 35 Functions Removed, and a Cache for Large MoE Models
Source: Latent Space (swyx). Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceLatent Space (swyx) ↗
Editor-in-Chief
Key metric #1Hugging Face announced desktop Llama apps built on llama.cpp
Key metric #2SGLang also declared version 1.0 at its summit
Key metric #3The community is asking for more maintainers: there are very many open issues and PRs

If you build an app that runs a language model directly on the user's computer, you most likely depend on llama.cpp. It is an open-source engine that turns a model's weights file into a working chat or assistant without the cloud.

Previously, any change to its internal functions could break someone else's program, and developers had to fix it in a hurry. Now llama.cpp and the ggml library have reached version 1.

0. In the release, the authors declared the public header file stable, meaning the list of functions through which applications connect to the engine, and removed 35 deprecated functions from it. The second piece of news is for those who run large MoE (mixture of experts) models.

Such a model has many separate expert blocks, but only some of them work on each word. If not all experts fit in the graphics card's memory, they have to be kept in ordinary system RAM. Now the most frequently used ones can be assigned a separate cache.

What "1.0" means for developers

Version 1.0 in an open-source project is a promise of stability. Say you have embedded llama.cpp into a notes editor or a corporate assistant. Previously, after updating the engine, you might find that a needed function had been renamed or had disappeared, and the build stopped compiling. Now the public interface is declared stable: you can rely on version 1.x and not expect surprises with every update.

The price of that stability is cleanup. 35 deprecated functions were removed from the public header. If your code calls even one of them, it will need fixing when you move to 1.0. It is a one-time job, but it is better to do it now than under pressure.

According to the release description, the project wants to keep its broad coverage of platforms and model types while remaining "reliable and lightweight" for those who embed it in their products. In parallel, Hugging Face announced desktop Llama apps based on llama.cpp. Their goal is to lower the entry barrier: a local model can be launched in a few clicks, without a console or manual building.

ggml and the SGLang engine also declared version 1.0 around the same time, at the recent SGLang Summit. For the ecosystem of local and self-hosted model deployment, this is a sign of maturing: key tools are announcing stability almost simultaneously.

A cache for "hot" experts

Imagine a large MoE model as a library with hundreds of specialists. For each word, only a few of them are needed, but it is not known in advance which ones. The graphics card holds not all of them; the rest sit in system RAM, and every load costs time. The logic of the new cache is simple, like any cache: the experts that are accessed often stay close at hand in fast memory, while rare ones are loaded as needed.

The project's author, Georgi Gerganov, wrote about this feature, and Daniel Han Chen showed how it works in practice. The source gives no speedup figures, so the real gain depends on the model and the hardware. You will have to test it on your own workloads.

What remains open

The community's main concern is not about technology but about people. Under the announcement on Reddit, which collected around 1800 reactions, users note that llama.cpp has "a ton of open issues and PRs" and the project needs more maintainers. Version 1.0 raises expectations: the more confidently a project calls itself a stable foundation, the more noticeable delays in triaging bugs and merging changes will be.

What to do in practice

  • If you embed llama.cpp in a product, check which of the 35 removed functions you call, and move to the current interface.
  • If you run large MoE models on limited video memory, test the expert cache and compare speed on your own tasks.
  • If you build a product for non-technical users, take a look at Hugging Face's desktop apps based on llama.cpp.

Overall, 1.0 is not a set of flashy features but predictability. For the local model industry, it may matter more than yet another benchmark record.

Story

Key Questions Answered

What "1.0" means for developers?

Version 1.0 in an open-source project is a promise of stability. Say you have embedded llama.cpp into a notes editor or a corporate assistant. Previously, after updating the engine, you might find that a needed function had been renamed or had disappeared, and the build stopped compiling. Now the public interface is declared stable: you can rely on version 1.x and not expect surprises with every update. The price of that stability is cleanup. 35 deprecated functions were removed from the public header.

What are the key details on A cache for "hot" experts?

Imagine a large MoE model as a library with hundreds of specialists. For each word, only a few of them are needed, but it is not known in advance which ones. The graphics card holds not all of them; the rest sit in system RAM, and every load costs time. The logic of the new cache is simple, like any cache: the experts that are accessed often stay close at hand in fast memory, while rare ones are loaded as needed.

What remains open?

The community's main concern is not about technology but about people. Under the announcement on Reddit, which collected around 1800 reactions, users note that llama.cpp has "a ton of open issues and PRs" and the project needs more maintainers. Version 1.0 raises expectations: the more confidently a project calls itself a stable foundation, the more noticeable delays in triaging bugs and merging changes will be.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…