Static embeddings processed 32.9 million texts on a laptop in 35 minutes

Anyone who has built vector search over a large corpus knows this moment. The data is collected and the vector database is ready, and then it turns out that turning the texts into embeddings takes days. Transformer models capture meaning very well, but they are heavy on a CPU. You can either wait or pay for a GPU. The Qdrant team shows a third path: don't run a neural network at all.

How a model without a neural network works

A static model is almost comically simple. It is a dictionary in which every token in the model's vocabulary is mapped in advance to a vector. Getting a text embedding takes three steps:

1. Split the text into tokens.
2. Pull each token's vector from the table.
3. Average those vectors.

There are no attention layers and no matrix multiplications for each query, so all the work boils down to fast memory operations. That is why the speed is of a different order.

The numbers from the experiment. Using the potion-base-8M model, the authors vectorized 32.9 million text fragments on the CPU of an ordinary laptop in 35 minutes. A transformer model would have needed about 53 days for the same job. Recalculated, the difference is roughly 2,000x. What used to require a cluster or rented GPUs can now be done in the time of a lunch break.

Where the catch is

There is no such thing as a free lunch, and the authors state the price directly: search quality drops. The reason lies in the design itself. In a transformer, every token "looks at" its neighbors, and a word's vector depends on context. In a static model, a word has one vector for every situation.

A simple example. The word "key" in the queries "key to the apartment" and "API key" gets the same vector. A transformer would tell these meanings apart by their surroundings, but averaging cannot. Another problem: averaging loses word order. The phrases "the bank gave the client a loan" and "the client gave the bank a loan" end up almost indistinguishable.

The article gives no specific figures for the quality loss; they are not in the source excerpt. So you can't blindly trust that "it'll be good enough." You need to measure on your own data and your own queries.

What to do with this in practice

Static embeddings do not replace transformers everywhere, but they cover several typical situations well:

• Prototypes and experiments. When you need to quickly test a search idea on millions of documents instead of waiting a week, cheap vectorization saves time and money.
• Huge corpora on a limited budget. If GPUs are unavailable or not economically justified, you can get a working index on ordinary hardware.
• Two-stage search. A fast static model selects candidates, and a heavy reranker or transformer orders the short list. The roughness of the first stage is then compensated by the second.
• Frequent reindexing. If the corpus is updated often, speed becomes a requirement rather than a bonus.

The practical procedure is as follows. Take a representative sample of queries with known correct answers. Run them through both the static and the transformer model and compare the search metrics. If the gap is acceptable for your product, the savings in cost and speed will pay off. If not, use static vectors as the first stage rather than the final one.

The main takeaway: a "smart" model is not always needed at every step of a pipeline. Part of the work can be handed to the simplest possible table, saving days of computation; the key is to measure honestly what you are paying for it.