X5 Tech and T-Bank fine-tune visual retrieval: ColQwen3 reaches 0.84 nDCG@5
In corporate PDFs, the answer is often hidden in a table or chart, while extracted text loses the relationships between numbers and labels. Researchers from X5 Tech and T-Bank fine-tuned visual retrieval using automatically generated and validated page queries. On a private test corpus, ColQwen3-2B scored 0.84 nDCG@5, compared with 0.77 for the best text-based model.
AI-processed from Хабр: Машинное обучение; edited by Hamidun News
If your corporate PDF search finds words from a report but misses the table you need, the problem may lie in document preparation. Text extraction removes cell layouts, the relationships between labels and charts, and other cues that a person picks up from a page in seconds. Roman Makeev of X5 Tech and Egor Machnev of T-Bank described their work on visual retrieval, which treats these documents as images of pages.
The main result from the published announcement: a fine-tuned ColQwen3-2B scored 0.84 nDCG@5 on the private X5-private corpus. The best text-based model in the comparison scored 0.77. For a team building search or RAG over complex PDFs, this is a reason to investigate how many answers are lost before documents even reach the language model.
Why search needs to see the page
Imagine a sales report. A user asks: “Which region saw the largest decline in revenue?” The answer is in a chart: region names appear below the bars, values are shown alongside them, and the period is specified in the title. After text extraction, these elements can become a sequence of words and numbers whose relationships are difficult to reconstruct.
This is a hypothetical example, not a test described by the authors. But it explains the practical purpose of visual retrieval: to use a page’s appearance where text alone is insufficient. This approach is particularly worth testing on presentations, reports, and documents with dense layouts. Still, retrieving the right page does not itself guarantee a correct answer: the next component in the system must read its contents correctly.
The article examines ColPali and ColQwen3, along with their adaptation to a Russian-language domain corpus. The available announcement does not establish that any visual model will outperform any text-based search system. It supports a narrower finding: one specific fine-tuned model achieved the best score in the reported comparison on a single private document set.
Training queries are created automatically
Adapting a retrieval model requires examples of which queries should lead to which documents. Creating these examples manually takes time from employees who understand the contents of the pages and users’ actual tasks.
The authors propose a methodology that uses documents as the basis for automatically generating and validating page queries. Training queries do not need to be written manually. This is the most transferable part of the work: a team gains a way to prepare data for its own domain corpus, rather than simply choosing an off-the-shelf model based on someone else’s ranking.
However, automatic validation itself needs evaluation. The announcement does not explain exactly how it works, which errors it filters out, or how closely the generated queries resemble real user requests. Before deployment, it is worth manually reviewing a sample: each query should have an answer on the page, correspond to a real work task, and avoid giving away the answer through its wording.
What the advantage means and how to test it
nDCG@5 measures the quality of the ranking of the first five results, taking their relevance into account. A score of 0.84 should not be read as “84% correct answers.” The difference from 0.77 is 0.07 metric points; it describes search results, not the accuracy of the entire question-answering system.
The authors explicitly state that X5-private was not used in training. This is a significant detail, but the private corpus limits independent verification. The announcement also provides no data on sample size, the statistical robustness of the advantage, latency, or document-processing costs.
The practical next step is to build a small test set from your own queries and pages with verified answers. Compare your current text-based search with visual retrieval on the same documents, evaluating tables, charts, and ordinary text separately. Then measure indexing time, retrieval speed, and the quality of the final answer. If the improvement is concentrated on complex pages, that provides a basis for choosing an approach for different document types. Deployment decisions should be based on those measurements, rather than a single number from someone else’s corpus.
Source: article announcement on Habr.
- Apr 28, 2026Hugging Face explains how to fine-tune multimodal embedding and reranker models
- Oct 28, 2026T-Bank replaces a swarm of LLMs with a single 32B model and cuts query costs by 2.8–3.9x
- Jul 9, 2026AI in Banking: Axos Brings AI Fintech Arc Technologies In-House for Tech Startups
- Nov 2, 2026The "no gold documents" label lost accuracy, from 0.68 to 0.10, on strong retrievers
Key Questions Answered
Why search needs to see the page?
Imagine a sales report. A user asks: “Which region saw the largest decline in revenue?” The answer is in a chart: region names appear below the bars, values are shown alongside them, and the period is specified in the title. After text extraction, these elements can become a sequence of words and numbers whose relationships are difficult to reconstruct. This is a hypothetical example, not a test described by the authors.
What are the key details on Training queries are created automatically?
Adapting a retrieval model requires examples of which queries should lead to which documents. Creating these examples manually takes time from employees who understand the contents of the pages and users’ actual tasks. The authors propose a methodology that uses documents as the basis for automatically generating and validating page queries. Training queries do not need to be written manually.
What the advantage means and how to test it?
nDCG@5 measures the quality of the ranking of the first five results, taking their relevance into account. A score of 0.84 should not be read as “84% correct answers.” The difference from 0.77 is 0.07 metric points; it describes search results, not the accuracy of the entire question-answering system. The authors explicitly state that X5-private was not used in training. This is a significant detail, but the private corpus limits independent verification.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.