Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarHabr AI→ original

A Breakdown of llama.cpp: Which Thread, Context, and Batch Flags Affect Speed

A Habr author went through llama.cpp's common parameters, which are shared by llama-cli, llama-completion, and llama-server, and commented on each flag. For example, the default batch size is 2048 tokens, the physical micro-batch is 512, the context is taken from the model, and threads are selected automatically. Some flags, such as core pinning and priority, barely change speed, while context and batch sizes directly affect memory. That's why tuning should start with them.

AI-processed from Habr AI; edited by Hamidun News
A Breakdown of llama.cpp: Which Thread, Context, and Batch Flags Affect Speed
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
Key metric #1--threads -1 picks the thread count automatically; extra threads can slow down generation
Key metric #2--keep preserves the beginning of a conversation, such as the system prompt, when the context shifts
Key metric #3--flash-attn is required for V-cache quantization and is set to auto by default

Running models locally with llama.cpp is simple as long as everything works with the defaults. When you need to squeeze out more speed or fit a model into memory, you start reading a help page with about a hundred flags. A Habr author compiled the first part of that help output: the common parameters (Common Params), which work almost identically in llama-cli, llama-completion, and llama-server. For each flag, they quote the original description from the built-in help and add their own commentary. The tests were run on version 0.8.0-dev (build 12014) under Linux x86_64, and the validation date was November 15, 2026.

Service Flags and Cache

The help output starts with utility commands. -h, --help, --usage prints all parameters with their default values and environment variables. --version shows the version and build information, which you can use to check whether the build supports CUDA. --license prints the licenses of llama.cpp and its dependencies, but newer versions no longer have this flag. -cl, --cache-list shows the models in the disk cache. Models only end up there when downloaded via -mu, --model-url; if you load files by direct path, the list will be empty. --completion-bash prints an autocompletion script for the terminal, after which flags can be completed with TAB.

Threads, Cores, and CPU Priorities

The largest block concerns the processor. -t, --threads sets the number of threads used for generation; the default is -1 (automatic selection). The author warns that generation is bound by memory bandwidth, so after a certain number of threads the gains disappear, and extra threads can even slow things down. There is an opinion that Ryzen users should specify physical cores and Intel users logical cores, but this needs to be verified on your own hardware. -tb, --threads-batch controls processing of the incoming prompt (prefill). It scales well with threads, so a higher number is optimal here. If the flag is not set, the value of --threads is used.

Next comes fine-grained placement tuning. -C, --cpu-mask pins work to cores via a hexadecimal mask (for example, 0xF means cores 0–3). -Cr, --cpu-range does the same in a more familiar form, such as 0-7.

--cpu-strict strictly pins threads to cores. In theory, this reduces latency and makes better use of the cache, but in practice the difference is usually small. It can also cause harm: if another process occupies a pinned core, the thread waits instead of moving to a free one.

--prio sets the process priority from low (-1) to realtime (3). For most tasks, the best choice is 0, and realtime can noticeably slow down the system itself. Values 1–3 may require administrator rights; otherwise llama.

cpp will ignore the flag. --poll (0–100, default 50) determines how long a thread actively waits for new work. A zero value reduces load, while a hundred speeds up wake-up, but the difference is on the order of milliseconds per token.

All of these parameters have "batch" versions: --cpu-mask-batch, --cpu-range-batch, --cpu-strict-batch, --prio-batch, and --poll-batch. They work the same way, but only for prefill, and by default they inherit the values of the main flags.

Context, Batches, and Memory

This block matters most in practice. -c, --ctx-size sets the context length. The default is 0 (the value is taken from the model), but for short chats and tool calls, it pays to reduce the size to save memory. -n, --predict limits the number of new tokens; the default is -1 (until the end-of-sequence token). llama-completion also accepts -2, in which case generation continues until the context is filled.

-b, --batch-size (default 2048) sets the logical batch size into which a long prefill is split. -ub, --ubatch-size (default 512) defines the physical micro-batch that actually goes to the CPU or GPU. It must not be larger than the logical batch.

--keep preserves the first N tokens, such as the system prompt, when the context overflows and llama.cpp starts shifting the window. A value of -1 preserves the entire original prompt.

--swa-full enables a full cache for models with a sliding attention window, such as Gemma 2 and 3. Quality and compatibility are theoretically better, but memory requirements increase significantly. -fa, --flash-attn (on/off/auto, default auto) speeds up attention computation and saves memory.

It needs to be enabled when quantizing the V cache, and ideally when quantizing the K cache.

Prompts and Auxiliary Options

-p, --prompt sets the first user message (the system message is set separately via -sys). -f, --file reads the prompt from a file, while -bf, --binary-file reads it byte by byte without any conversion, which is rarely needed. --perf and --no-perf enable internal libllama timings: load time, prefill, and tokens per second. -e, --escape controls the handling of sequences such as \n, while --no-escape passes them through as plain text. The source text cuts off in the description of this flag, so the remaining parameters from the help output are not covered here.

What to Take Away

The author's conclusion is candid: many CPU flags give a small gain, and they should be verified on your own hardware. The real impact usually comes from context, batch sizes, Flash Attention, and thread count. The rest is best left at default, and realtime priority should not be enabled at all.

Story

Key Questions Answered

What are the key details on Service Flags and Cache?

The help output starts with utility commands. -h, --help, --usage prints all parameters with their default values and environment variables. --version shows the version and build information, which you can use to check whether the build supports CUDA. --license prints the licenses of llama.cpp and its dependencies, but newer versions no longer have this flag. -cl, --cache-list shows the models in the disk cache.

What are the key details on Threads, Cores, and CPU Priorities?

The largest block concerns the processor. -t, --threads sets the number of threads used for generation; the default is -1 (automatic selection). The author warns that generation is bound by memory bandwidth, so after a certain number of threads the gains disappear, and extra threads can even slow things down. There is an opinion that Ryzen users should specify physical cores and Intel users logical cores, but this needs to be verified on your own hardware.

What are the key details on Context, Batches, and Memory?

This block matters most in practice. -c, --ctx-size sets the context length. The default is 0 (the value is taken from the model), but for short chats and tool calls, it pays to reduce the size to save memory. -n, --predict limits the number of new tokens; the default is -1 (until the end-of-sequence token). llama-completion also accepts -2, in which case generation continues until the context is filled. -b, --batch-size (default 2048) sets the logical batch size into which a long prefill is split.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…