deepvk vs OpenAI text models: a benchmark on court rulings
On a corpus of 858 rulings from the Intellectual Property Rights Court, the local deepvk USER2-base scored almost on par with OpenAI text-embedding-3-large and Voyage. The author tested 7 embedding models and 4 rerankers and concluded that for Russian-language legal RAG, the USER2-base + jina-reranker-v3 combination looks like the most practical option in terms of quality, cost and independence from external APIs.
AI-processed from Habr AI; edited by Hamidun News
The Russian text model deepvk USER2-base delivered results almost comparable to OpenAI and Voyage in a test on a corpus of court documents.
Does legal RAG need an expensive API?
For a narrow legal RAG, an expensive API is not always necessary: a local model can deliver comparable results.
Does a reranker improve search quality on court documents?
The benefit of a reranker depends on the strength of the base model; it is not a universal tool.
Which models were compared in the test?
The test covered deepvk USER2-base, OpenAI and Voyage on a corpus of 858 rulings from the Intellectual Property Rights Court.
What documents was the benchmark run on?
On a corpus of 858 rulings from the Intellectual Property Rights Court and the texts of Part Four of the Civil Code of the Russian Federation.
- Nov 1, 2026Local models lose to Opus and Codex: 0.21 on a MacBook versus 1.00 for Codex
- Nov 1, 2026Kimi scores 96.2% on a 1C database analytics test and emerges as a fallback if Claude access is blocked
- Nov 1, 2026OpenAI Rewrites GPT Behavior Rules: Five Values and a Chain of Command
- Nov 1, 2026Google admits AI Mode cuts search revenue, shares fall 15%
Key Questions Answered
Does legal RAG need an expensive API?
For a narrow legal RAG, an expensive API is not always necessary: a local model can deliver comparable results.
Does a reranker improve search quality on court documents?
The benefit of a reranker depends on the strength of the base model; it is not a universal tool.
Which models were compared in the test?
The test covered deepvk USER2-base, OpenAI and Voyage on a corpus of 858 rulings from the Intellectual Property Rights Court.
What documents was the benchmark run on?
On a corpus of 858 rulings from the Intellectual Property Rights Court and the texts of Part Four of the Civil Code of the Russian Federation.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.