Live Nov 18 · Cohort 3Nov 18, 19:00 MSK — free hands-on AI agents live workshop · Live Cohort 3 starts Nov 23Join Nov 18 Live →Cohort 3 (Nov 23)
ReleaseInsight · On RadarHabr AI→ original

RWB Didn't Replace Its OCR Pipeline with a Single VLM: 400M Images a Day and Seven Stages

The RWB moderation team runs about 400M images a day through OCR, drawn from 35M product listings. One VLM could, in theory, replace the chain of models, but in production it breaks down on speed, memory, text geometry and the addition of new languages. So a multi-stage pipeline reads the text, and Qwen3-VL serves only as an auditor on a fraction of a percent of traffic.

AI-processed from Habr AI; edited by Hamidun News
RWB Didn't Replace Its OCR Pipeline with a Single VLM: 400M Images a Day and Seven Stages
Source: Habr AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceHabr AI ↗
Editor-in-Chief
Key metric #1A script classifier isolates each new language in its own branch
Key metric #2A segmenter and TPS straighten curved text before recognition
A VLM drifts on boxeswith 20+ boxes, the errors are already obvious

Everyone has gotten used to the idea that a transformer can do it all: feed it an image, get text back. But the Moderation Platform team at Wildberries (RWB) sees it differently. More than 35M product listings pass through its moderation every day, and roughly 400M images go through OCR. More than 90% of that flow is product-listing photos. And they are read not by one large model but by a pipeline of several stages.

Why moderation needs OCR, and why "one model" doesn't work

Text in an image is not an end in itself here. It is the input for checks. It is used to find contact details that route around the platform, banned words, and drugs listed in a product's composition. The listings are chaotic: busy backgrounds, diagonal and arched captions, vertical text, several scripts in a single photo. RWB operates in more than 10 countries, so multilingual support is mandatory.

The team seriously considered two candidates for the "one model" role. A large VLM such as Qwen3-VL reads characters well, but at this volume it runs into memory and speed limits. A compact end-to-end model such as GLM-OCR (under a billion parameters) is closer to the pipeline in speed. The question is different: can it be fixed and developed as precisely, one piece at a time?

The VLM ended up with a job elsewhere. It runs in parallel on a tiny fraction of traffic, computes metrics and hunts for corner cases for the main pipeline. It is an internal auditor, not a workhorse.

Where end-to-end breaks down in operation

Geometry. A rectangular detector box is not enough for arcs and diagonals, so RWB uses a segmenter. It outputs a text mask, and the mask's shape determines the crop type: horizontal text is simply cut out, slanted text is rotated into place, vertical text is turned upright, and curved text is straightened. For curved lines, the pipeline estimates a midline and builds a TPS (thin plate spline) transform with an adaptive number of control points. If text on an arc reads poorly, the team fixes only that block and leaves the rest alone. In a single model all of these steps happen implicitly, and you can only check the final answer.

Languages. Russian accounts for about 80% of traffic, and the other languages are far smaller. If you fine-tune a shared model on a new language, quality on the old ones, Russian included, may drop. The average metric will barely show it, but in production it turns into thousands of errors. Mixing in old data and using adapters reduce the risk but do not guarantee isolation, and before every release you still have to re-check all languages.

In the pipeline, a script classifier solves this problem. It does not read the text; it only picks a recognition branch for each crop. There are now more than five such branches, and similar languages are merged into groups. A new language gets its own recognizer, its own golden dataset and a separate rollout. The weights of the Russian model stay the same, so there is no need to prove the absence of regression on the main traffic. Crops from one branch can be batched and processed in parallel, and resources can be allocated according to real traffic. An end-to-end model, by contrast, generates all the text token by token.

Reading order and coordinates. A product listing is not a book page: text sits in islands, banners and badges. Read left to right and top to bottom, and you get "3 pcs, A LOT, WITHOUT, MINIMUM, PROTEIN, GLUTEN, CARBS". Clustering nearby boxes gives the proper "3 pcs / A LOT OF PROTEIN / GLUTEN-FREE / MINIMUM CARBS". In the pipeline this is a separate algorithm with clear rules. If the order is wrong, it is to blame, not the recognizers.

Coordinates are also part of the contract. They are needed to identify the source of a signal (the product itself or a marketing banner) and to blur personal data in photos. A VLM generates the box together with the text, and on images with more than twenty boxes, according to the team's checks, the errors are already obvious. The model may name the text correctly, but its geometry can't be trusted as evidence for banning a listing.

What to take away from this

The main conclusion is that the question is not "VLM or pipeline" but "where do you need observability and isolation of changes." If you have a large flow, uneven language distribution and decisions with consequences, modularity gives you three things:

  • an error can be localized to a specific stage;
  • new functionality can be rolled out without re-checking everything else;
  • resources scale with real traffic, not with the heaviest scenario.

A universal model remains useful here as an independent quality judge on a small share of traffic.

There are limitations too. A pipeline is harder to maintain and needs several models plus a classifier that may confuse similar scripts. For such cases the authors advise merging the languages into a group when the cost of a routing error is higher than the loss in quality. If you have a couple of thousand scans a day and a single language, a large VLM is most likely simpler and cheaper. But when it comes to hundreds of millions of images and moderation, transparency and control matter more than the elegance of "one model."

Story

Key Questions Answered

Why moderation needs OCR, and why "one model" doesn't work?

Text in an image is not an end in itself here. It is the input for checks. It is used to find contact details that route around the platform, banned words, and drugs listed in a product's composition. The listings are chaotic: busy backgrounds, diagonal and arched captions, vertical text, several scripts in a single photo. RWB operates in more than 10 countries, so multilingual support is mandatory. The team seriously considered two candidates for the "one model" role.

Where end-to-end breaks down in operation?

Geometry. A rectangular detector box is not enough for arcs and diagonals, so RWB uses a segmenter. It outputs a text mask, and the mask's shape determines the crop type: horizontal text is simply cut out, slanted text is rotated into place, vertical text is turned upright, and curved text is straightened. For curved lines, the pipeline estimates a midline and builds a TPS (thin plate spline) transform with an adaptive number of control points.

What to take away from this?

The main conclusion is that the question is not "VLM or pipeline" but "where do you need observability and isolation of changes." If you have a large flow, uneven language distribution and decisions with consequences, modularity gives you three things: an error can be localized to a specific stage; new functionality can be rolled out without re-checking everything else; resources scale with real traffic, not with the heaviest scenario.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Private AI Builders Community · Vibe Club

Ship AI Faster with Founders, Engineers & Weekly Architecture Reviews

Inside Vibe Club (hamidun.com/club) — weekly live masterminds with Zhemal Khamidun, Tech Radar, full LMS access, and 370+ production Claude Code skills.

Join Vibe Club →Explore Hamidun Academy
What do you think?
Loading comments…