SynthAVE: 21 Models Label 12,000 Products in Place of Humans

Researchers have published SynthAVE, a method for large-scale synthetic labeling of e-commerce products through voting across multiple LLMs. Rather than annotating millions of data points by hand, the team used a network of 21 LLM configurations (7 models × 3 prompts) whose output matches human expert judgments 95.2% of the time (Cohen's κ = 0.92).

Why manual labeling no longer scales

In e-commerce, every product has dozens of parameters: size, color, material, weight, sleeve length, chest size and so on. A large marketplace handles thousands of product types and hundreds of attributes at once. That means full labeling requires millions of annotations. Hiring people for that volume would cost a monthly budget of tens of millions of rubles.

Companies have long known that LLMs can generate labels automatically. But a question follows: how do you check quality at that scale? Manual verification degenerates into the same expensive work.

The method: an arena of voting models

Instead of a single LLM, the authors call on 7 different models, each with 3 prompt variants. All 21 configurations evaluate every product sample independently, and the final result is chosen by majority vote.

Key figures for the dataset:

• 12,726 products
• 229 product types
• 792 distinct attributes
• 4 languages: Spanish, French, Italian, German

The result: human-level quality at machine speed

Majority voting reached Cohen's κ = 0.92 against human labeling (95.2% direct agreement). Individual models agree with one another at only κ = 0.76.

The paradox: every model has its own blind spots. One handles fabrics better, another sizes, a third materials. But when many models vote, their errors offset one another, as in a good jury.

What this means

E-commerce companies gain a way to scale data labeling to human-level quality at a cost 10–100 times lower than manual work. This opens the door to automating thousands of microtasks that used to require people. The method is general: wherever mass labeling is needed (medicine, finance, logistics), you can assemble an arena of judges and use voting.