Starts Nov 23Live Cohort 3 of Vibe Coding & AI Agents: 3 weeks, 9 live hands-on sessionsCohort 3 (Nov 23) →
ReleaseInsight · On RadarQdrant Blog→ original

papr-embed-code-1 finds code 3 pp more accurately than Qwen3, and its vectors compress 31x

Qdrant tested whether code search needs a dedicated embedding model. On unfamiliar code, papr-embed-code-1 retrieves the right results more accurately than the original Qwen3-Embedding: the 4B version is 3.4 pp better on nDCG@10, and the 0.6B version is 2.6 pp better. On ordinary text, both models lose slightly, so keep the base model for prose. Compressing a 4B vector to 1 bit makes it 31 times smaller.

AI-processed from Qdrant Blog; edited by Hamidun News
papr-embed-code-1 finds code 3 pp more accurately than Qwen3, and its vectors compress 31x
Source: Qdrant Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceQdrant Blog ↗
Editor-in-Chief
Key metric #1A 4B vector takes 328 bytes at 1 bit instead of 10 KB in float32
Key metric #24B at 1 bit beats 0.6B in float32 and uses 12 times less RAM
A hybrid with BM25 did not helpat equal weights, quality dropped by 2–17 pp

Say you are building search over a large codebase. A developer types "a function that reads a CSV and returns a dictionary," and the system has to find the right function among hundreds of thousands. To do this, you turn code into embedding vectors, which raises a question: should you use a model fine-tuned specifically on code, or is a strong general-purpose one enough?

The Qdrant team tested this on the open model papr-embed-code-1, which is built on Qwen3-Embedding. The result: on code the model did not see during training, it beats the original by 3.4 pp (4B version) and 2.

6 pp (0.6B version) on nDCG@10. This metric shows how high the correct answers rank within the top ten results; "pp" stands for percentage points, so 3.

4 pp is 0.034 on a scale from 0 to 1. And if you compress each vector to 1 bit, it becomes 31 times smaller: 328 bytes instead of 10 KB.

Why you can't take Papr's headline numbers at face value

papr-embed-code-1 is Qwen3-Embedding with additional training on code search. It comes in two sizes, 0.6B and 4B, and the weights are open. Papr claims record results on the CoIR benchmark, but its own model card says the model trained on almost all of CoIR's training data. A high score there says little.

So Qdrant asked a narrower question: does the model beat its base version on code it has not seen? The tests included matching a Python function to a task description, finding library files for a Stack Overflow question, matching a SQL query to a question about a table, and matching a program to its version in another language. Both models received the same query instructions. Qdrant reproduced Papr's own numbers to within 0.001 nDCG@10. The exception was tasks with many identical documents, where the outcome depends on how ties are resolved. There Qdrant got 62.5% and 65.0% against the claimed 72.0% and 72.3%.

The gain shows up on code, not on prose

On unfamiliar code, both models beat the base model on every task, but the size of the gain varies. On HumanEval, the base 4B model already scored 98%, so there was almost nothing to add. On WikiSQL, the 4B version improved by 7.0 pp.

On ordinary text the picture reverses: 0.6B lost 0.3 pp on average, and 4B lost 1.6 pp. On TREC-COVID (scientific papers about COVID-19), the 0.6B version dropped by 6.5 pp. So if you mostly search prose, stay with the base model.

The main savings come from vector compression

Code turns into a great many vectors: one per function or per chunk of a file. A 4B vector consists of 2560 numbers and takes about 10 KB in float32. With TurboQuant 1-bit quantization, it weighs 328 bytes. On seven code tasks, compression without any re-check kept 99.7% of float32 quality, and with candidate rescoring (rescoring with a 2x oversampling margin) it kept all of it.

This leads to a non-obvious choice: 4B at 1 bit scores higher than 0.6B in float32 and uses 12 times less RAM. The price is speed: 4B computes embeddings roughly four times slower on the same GPU. And on WikiSQL, the smaller model still came out ahead.

Qdrant also tested at large scale: 1,005,474 code fragments in six languages in a single collection, with search restricted to the query's language. Exact brute-force search gives 92.3%. With HNSW ef=128, 1-bit search with rescoring kept 99.5% of that result, against 99.7% for float32, and ran faster: 1.0 ms per query against 1.8 ms. Raising ef to 256 added 0.2–0.4 pp nDCG@10, and 1-bit search took 1.4 ms against 2.9 ms for float32. Without a language filter, quality drops by 0.5–0.7 pp.

BM25 alongside vector search did not help

Code contains many exact names, so BM25 keyword search looked like a natural complement; Qdrant 1.20 even added a tokenizer for code. But BM25 on its own fell far behind: on APPS, where the query is a problem statement and the document is a bare solution, it scored 1%, since the two share almost no words. Fusing it with vector search at equal weights made every task worse by 2–17 pp for 4B. At a BM25 weight of 0.1, the result came out 0.06 pp below vector search alone, and the best gain on any task was 0.2 pp.

What to do in practice

For code search, a fine-tuned model is justified; for prose, it is not. If you can afford 4B, store it at 1 bit with rescoring enabled, and raise ef before you raise the number of bits. Compare the result against an uncompressed collection on your own queries: once without rescoring, once with a 2x margin. If a collection holds several languages, filter by language. It is better not to enable BM25; if your users search for exact identifiers, test it separately and with a low weight. Keep in mind that Qdrant's test is its own set of tasks, and on your corpus the numbers may differ.

Story

Key Questions Answered

Why you can't take Papr's headline numbers at face value?

papr-embed-code-1 is Qwen3-Embedding with additional training on code search. It comes in two sizes, 0.6B and 4B, and the weights are open. Papr claims record results on the CoIR benchmark, but its own model card says the model trained on almost all of CoIR's training data. A high score there says little. So Qdrant asked a narrower question: does the model beat its base version on code it has not seen?

What are the key details on The gain shows up on code, not on prose?

On unfamiliar code, both models beat the base model on every task, but the size of the gain varies. On HumanEval, the base 4B model already scored 98%, so there was almost nothing to add. On WikiSQL, the 4B version improved by 7.0 pp. On ordinary text the picture reverses: 0.6B lost 0.3 pp on average, and 4B lost 1.6 pp. On TREC-COVID (scientific papers about COVID-19), the 0.6B version dropped by 6.5 pp. So if you mostly search prose, stay with the base model.

What are the key details on The main savings come from vector compression?

Code turns into a great many vectors: one per function or per chunk of a file. A 4B vector consists of 2560 numbers and takes about 10 KB in float32. With TurboQuant 1-bit quantization, it weighs 328 bytes. On seven code tasks, compression without any re-check kept 99.7% of float32 quality, and with candidate rescoring (rescoring with a 2x oversampling margin) it kept all of it. This leads to a non-obvious choice: 4B at 1 bit scores higher than 0.6B in float32 and uses 12 times less RAM.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…