NVIDIA showed how to fine-tune an embedding model for a specific domain in a day

NVIDIA published on Hugging Face a detailed recipe for building a specialized embedding model for RAG in just a few hours without manual labeling. The idea is to take a base model, generate synthetic question-document pairs, fine-tune it on hard negative examples, and immediately check the search improvement.

How the Pipeline Works

At the core is the Llama-Nemotron-Embed-1B-v2 model. The authors propose not collecting a dataset manually, but generating it from your own documents: internal instructions, contracts, logs, and reference articles. To do this, the LLM reads the corpus and creates thousands of pairs from questions and relevant fragments.

Questions are not only factual, but also multi-hop, when you need to connect several pieces of text. This is important for real RAG scenarios, where users rarely ask perfectly localized questions about a single paragraph. Next, the pipeline automatically splits data into train and test sets, prepares a BEIR-compatible benchmark, and launches fine-tuning.

In the article, the entire process is condensed into six CLI commands: from data generation to model deployment via NVIDIA NIM. For a proof of concept, a small corpus of 50-100 documents is sufficient, and for a full run, the authors specify one A100 or H100 class GPU with 80GB of memory. For a corpus of approximately 500 documents, the entire cycle takes about 2-3 hours, although it's formally announced as a "less than a day" process.

Why Hard Negatives Matter

The key step is hard negative mining. If you train the model only on positive pairs, it will quickly learn to separate obviously irrelevant texts, but will confuse similar documents. So the system searches for passages that the base model considers almost correct, but that are not the target answer. A protective threshold is used: anything that scores more than 95% of the minimum score for positive documents is cut off to avoid polluting training with false negatives. The pipeline does several things in sequence:

• embeds all queries and corpus documents
• calculates similarity and excludes already marked positive fragments
• selects top-k hard negatives, by default five per query
• expands multi-hop questions into separate training examples

This approach makes fine-tuning much closer to production search. The model learns to distinguish not between "correct" and "completely off," but between documents that differ in details: contract terms, instruction version, error type, or usage context. It's precisely on such nearly matching fragments that corporate search usually breaks down, and along with it, the quality of RAG answers. That's where expensive errors in LLM-based RAG answers most often hide.

Metrics and Production

Validation is done through BEIR using four standard metrics: nDCG, Recall, Precision, and MAP at different k values. On a synthetic dataset based on NVIDIA's public documentation, the fine-tuned model improved nDCG@10 from 0.555 to 0.

616, and Recall@10 from 0.630 to 0.693, that is, by approximately 10%.

The authors separately reference the Atlassian case: there, the same recipe on a public Jira dataset raised Recall@60 from 0.751 to 0.951.

For corporate search, this is no longer cosmetic, but a noticeable change in relevance. After evaluation, the model is not left in PyTorch format. It can be exported to ONNX or TensorRT, then deployed via NVIDIA NIM as an inference service with an OpenAI-compatible `/v1/embeddings` endpoint.

This removes some integration issues: if you already have a pipeline that can work with embeddings API, there's no need to rewrite the client. The article also includes a separate accuracy check after conversion to catch quality losses due to optimization. That is, we're talking not just about a research recipe, but about a path from raw documents to a production service.

What This Means

The barrier to entry for custom embedding models is noticeably lowered. Instead of weeks of manual labeling, a team can check in one business day whether domain adaptation will provide real search improvements on their data, and quickly decide whether to scale this approach to production.