emb-diversity: инструмент измерения разнообразия данных на основе эмбеддингов

Researchers from the nlpsoc project published a preprint on arXiv in July 2026, along with an open-source tool called emb-diversity — a set of metrics that measures the diversity of text data based on embeddings, not just vocabulary, and works with any embedding model.

What emb-diversity measures

emb-diversity evaluates four types of dataset diversity at once: stylistic, semantic, linguistic, and speaker diversity. Unlike lexical metrics, which count unique words and n-grams, the tool works with numerical vectors (embeddings) — so it applies to any data that can be vectorized, not just text.

The key idea is flexibility. According to the authors, embedding-based measures work with any embedding model and any data that can be vectorized, which makes them suitable for many interpretations of the concept of "diversity."

• The tool is emb-diversity, open source on GitHub (repository nlpsoc/emb-diversity)
• Works with any embedding model and any data that can be vectorized
• Measures 4 types of diversity: stylistic, semantic, linguistic, speaker
• Published as a preprint on arXiv in the cs.CL section, version v1, July 2026
• Covers a wide range of diversity measures in a single package

Why this matters for NLP

emb-diversity closes a specific gap — the fragmentation of existing methods. As the authors note, there are already many tools for measuring the lexical diversity of text, but researchers have not had a standardized way to compute diversity based on embeddings until now.

The diversity of training data directly affects model quality. The preprint's authors cite a growing body of evidence: the more diverse the data, the fairer and more robust NLP models turn out to be. A one-sided or biased dataset leads to biased and fragile systems — and without a common measurement tool, this problem is hard to even detect.

"There is growing evidence that data diversity is critical to building

fair and robust NLP models," — from the abstract of the emb-diversity preprint on arXiv.

How to use the tool

emb-diversity is available as open source in the nlpsoc/emb-diversity repository on GitHub — it can be plugged into your own data preparation pipeline. The authors demonstrate several scenarios: analyzing stylistic, semantic, linguistic, and speaker diversity in specific datasets.

The metrics are built on top of embeddings, so the developer chooses the embedding model for their own task — from multilingual encoders to specialized domain-specific models. This allows measuring not a single fixed notion of "diversity," but the one that matters in a particular project.

What this means

emb-diversity gives teams a unified way to quantitatively assess how diverse their dataset is, even before training a model. For those fighting bias and fragility in systems, this is a cheap diagnostic step before expensive training.

Frequently Asked Questions

What is embedding-based data diversity?

It is a measurement of dataset diversity through numerical vectors (embeddings), rather than through counting unique words. The approach works with any embedding model and any data that can be vectorized, so it covers style, meaning, language, and other properties at once.

Where can I download emb-diversity?

The tool is publicly available on GitHub in the nlpsoc/emb-diversity repository. It is a research package from an arXiv preprint (cs.CL section), available for free.