emb-diversity: An Embedding-Based Tool for Measuring Data Diversity

In July 2026, researchers from the nlpsoc project published a preprint on arXiv along with emb-diversity, an open-source tool. It is a set of metrics that measures the diversity of text data using embeddings rather than vocabulary alone, and it works with any embedding model.

What emb-diversity measures

emb-diversity assesses four types of dataset diversity at once: stylistic, semantic, language, and speaker diversity. Unlike lexical metrics, which count unique words and n-grams, the tool operates on numerical vectors (embeddings). That makes it applicable to any data that can be vectorized, not just text.

The key idea is flexibility. According to the authors, embedding-based measures work with any embedding model and any data that can be vectorized, which makes them suitable for many interpretations of what "diversity" means.

• The tool is emb-diversity, open-source code on GitHub (repository nlpsoc/emb-diversity)
• Works with any embedding model and any data that can be vectorized
• Measures 4 types of diversity: stylistic, semantic, language, and speaker
• Published as a preprint on arXiv in the cs.CL category, version v1, July 2026
• Covers a broad range of diversity measures in a single package

Why it matters for NLP

emb-diversity fills a specific gap: the fragmentation of existing methods. As the authors point out, there are already many tools for measuring the lexical diversity of texts, but researchers have so far lacked a standardized way to compute diversity based on embeddings.

The diversity of training data directly affects model quality. The preprint's authors cite a growing body of evidence that more diverse data produces fairer and more robust NLP models. A one-sided or skewed dataset leads to biased and brittle systems, and without a common measurement tool, such a problem is hard even to detect.

"There is growing evidence that data diversity is critical for

building fair and robust NLP models," from the emb-diversity preprint abstract on arXiv.

How to use the tool

emb-diversity is available as open-source code in the nlpsoc/emb-diversity repository on GitHub, and it can be plugged into your own data-preparation pipeline. The authors demonstrate several use cases: analyzing stylistic, semantic, and language diversity, as well as speaker diversity, in specific datasets.

The metrics are built on top of embeddings, so developers choose the embedding model that fits their task, from multilingual encoders to specialized domain models. This lets them measure not one fixed notion of "diversity" but the one that matters in their particular project.

What this means

emb-diversity gives teams a single way to quantify how diverse their dataset is before they train a model. For those fighting bias and brittleness in their systems, it is a cheap diagnostic step ahead of expensive training.

Frequently asked questions

What is embedding-based data diversity?

It is a way of measuring a dataset's diversity through numerical vectors (embeddings) rather than by counting unique words. The approach works with any embedding model and any data that can be vectorized, so it covers style, meaning, language, and other properties at once.

Where can I download emb-diversity?

The tool is openly available on GitHub in the nlpsoc/emb-diversity repository. It is a research package from an arXiv preprint (cs.CL category) and is free to use.