Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseResearch Paper · For R&DLanceDB Blog→ original

Cohere Reranker Pushes Hybrid Search Accuracy in RAG Above 90% in LanceDB Test

In RAG, search often finds the right paragraph but ranks it too low, so it never makes it into the prompt. LanceDB tested Cohere Rerank v3, which re-sorts the results in a second pass. In hybrid search (vectors plus keywords), the share of queries where the right chunk made the top results exceeded 90%, 23–49% above the baseline. For that reason, a reranker should be added to hybrid search in RAG right away.

AI-processed from LanceDB Blog; edited by Hamidun News
Cohere Reranker Pushes Hybrid Search Accuracy in RAG Above 90% in LanceDB Test
Source: LanceDB Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceLanceDB Blog ↗
Editor-in-Chief
Cohere v3 barely outperformed v2both versions hit a ceiling of about 90%
Key metric #2For pure vector search, the reranker added +8% (BGE) and +11% (ColBERT)
Key metric #3The test used 800+ question–context pairs from Uber's 2021 annual report

Anyone who has built a RAG system knows the frustrating situation: the search finds the right paragraph but ranks it seventh, while the prompt for the model only includes the first three results. The answer misses the mark even though the data was in the database. Reranking fixes this.

It is a second pass that re-sorts the retrieved results by a different criterion, which does not necessarily match the search's own score. The LanceDB team published a benchmark comparing several rerankers. The main subject of the test is the new Cohere Rerank v3 model.

In hybrid search, which combines vector similarity with keyword search, it raised the hit rate above 90%. Here a "hit" means something simple: the right text chunk ended up in the top K results. Compared with ordinary vector search, that is roughly 23% and 49% more for the two embedding models tested.

How to hook it up

LanceDB has reranking built in for all search types: vector, full-text, and hybrid. To use it, create a reranker object (for example, CohereReranker with an API key) and pass it to the query through the rerank method. The result can then be returned as a pandas table. The code is nearly identical for all three modes, so switching between them and comparing rerankers is easy.

How the test was run

They evaluated only search quality, not the full RAG pipeline. The metric was hit rate: whether the expected chunk appeared in the top K results. This is a simplified version of the familiar recall@k and mAP. The baseline for comparison was ordinary vector search without a reranker.

There is an important detail. When a reranker is applied to vector search, the system first takes twice as many candidates (2K) and then keeps the top K after re-sorting. Without that margin, the reranker would have nothing to reorder, and the effect would be invisible.

Two datasets were used. The first was Uber 10K: more than 800 question–context pairs drawn from Uber's 2021 annual report filed with the SEC. It can be downloaded through LlamaIndex. The second was a set based on a review article about large language models. Testing used two embedding models: BGE and ColBERT. The rerankers compared were ColBERT, Cohere v2, and Cohere v3. The text also mentions cross-encoders and linear score combination, and Cohere outperformed those as well.

What came out

  • Vector search + reranker. Results were similar for both embedding models. ColBERT, serving as both embedder and reranker, performed slightly better than the baseline. Cohere's rerankers showed the highest accuracy: simply re-sorting the expanded result set gave roughly +8% for BGE and +11% for ColBERT.
  • Hybrid search + reranker. Here Cohere again led in both cases, with the hit rate exceeding 90%. That is about +23% and +49% over the baseline for the two embedding functions.
  • Second dataset. The picture was the same: Cohere was best with both ColBERT and BGE, and the ColBERT reranker paired with ColBERT embeddings was slightly better than the baseline. According to the authors, Cohere also came out first on other similar datasets.

Cohere v3 versus v2

The unexpected finding is that on this data the difference between Cohere v3 and v2 is negligible. Both versions land at about 90%, meaning they hit a quality ceiling. The authors believe these datasets simply cannot separate the models.

v3 may stand out more on other tasks. In Cohere's announcement, the rerankers were compared on semi-structured JSON data, with search done via BM25 (which LanceDB treats as full-text search) rather than vectors. Under those conditions v3 is clearly better than v2. LanceDB did not include these measurements in the article, but plans to cover them in upcoming installments, along with more complex cases such as structured and semi-structured search.

Another claimed advantage of v3 is speed: the API should run noticeably faster than v2. However, the authors took this comparison from Cohere's blog and did not measure it themselves.

What to do in practice

If your RAG uses only vector search, a reranker is one of the cheapest ways to improve result quality. Start with hybrid search and the Cohere reranker, then check what v3 delivers on your type of data. For reports and articles, you probably should not expect a jump over v2. For JSON and semi-structured documents, the difference, judging by Cohere's results, may be larger.

There are also limitations. This is a benchmark by the vendor of a vector database, run on two datasets, and the metric is simplified: it shows whether the right text made it into the results, not the quality of the final answers. So before deploying, run the same hit-rate measurement on your own documents.

Story

Key Questions Answered

How to hook it up?

LanceDB has reranking built in for all search types: vector, full-text, and hybrid. To use it, create a reranker object (for example, CohereReranker with an API key) and pass it to the query through the rerank method. The result can then be returned as a pandas table. The code is nearly identical for all three modes, so switching between them and comparing rerankers is easy.

How the test was run?

They evaluated only search quality, not the full RAG pipeline. The metric was hit rate: whether the expected chunk appeared in the top K results. This is a simplified version of the familiar recall@k and mAP. The baseline for comparison was ordinary vector search without a reranker. There is an important detail. When a reranker is applied to vector search, the system first takes twice as many candidates (2K) and then keeps the top K after re-sorting.

What came out?

Vector search + reranker. Results were similar for both embedding models. ColBERT, serving as both embedder and reranker, performed slightly better than the baseline. Cohere's rerankers showed the highest accuracy: simply re-sorting the expanded result set gave roughly +8% for BGE and +11% for ColBERT. Hybrid search + reranker. Here Cohere again led in both cases, with the hit rate exceeding 90%. That is about +23% and +49% over the baseline for the two embedding functions. Second dataset.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…