WeRide Uses LanceDB to Cut Search for Rare Dangerous Driving Footage from a Week to an Hour
Rare situations, such as a person lying on the road, are essential for training a self-driving car, but plain SQL cannot find them among billions of frames. WeRide adopted the LanceDB vector database: an engineer types a phrase or uploads an image and gets 1000 similar frames in milliseconds. Collecting these dangerous cases dropped from 1 week to 1 hour. If you have a lot of unlabeled images, semantic search will pay for itself quickly.
AI-processed from LanceDB Blog; edited by Hamidun News
For a self-driving car, the hardest part is not ordinary road driving but rare, dangerous situations, such as a person lying on the roadway. Such episodes are vanishingly rare in recordings from cameras, radar and lidar, and models learn worst from exactly these cases. Previously, WeRide engineers needed a full week to collect such cases for training.
Since adopting the LanceDB vector database, which searches not by exact values in columns but by the meaning of an image or a phrase, the process takes about an hour. For any team with large volumes of images, video or other "heavy" data without manual labels, this is a clear example of how faster search speeds up the whole cycle: find the problem, collect data, retrain.
Why plain SQL didn't work
WeRide, a Nasdaq-listed company, develops autonomous driving technology from Level 2 to Level 4. It conducts research, testing and operations in more than 30 cities across seven countries, and its autonomous fleet has been running for more than 1700 days. That produces a constant stream of multimodal data. In 2023, Fei Chen took over the data infrastructure team, which was tasked with quickly finding data that matches a specific criterion, above all rare examples from the "long tail" of scenarios.
SQL is powerless here. A query like "find frames where a person is lying on the ground" cannot be written as a condition on columns if the table has no such label. So images and text are converted into vectors (embeddings), and the database looks for the nearest ones by meaning.
The platform requirements were strict:
- P99 latency in milliseconds (that is, 99% of queries fall within that limit) when returning the top 1000 results for a text or image query across billions of records;
- vector search combined with metadata filters;
- minimal bandwidth consumption, so as not to interfere with the fast storage that feeds model training;
- reasonable cost and simple maintenance as data grows.
How the pipeline works
Images and video from the vehicles are converted into vectors. They are loaded together with metadata into LanceDB, which runs on top of a custom storage cluster built by the same infrastructure team. LanceDB indexes the data for fast search. New data arrives continuously, so reindexing runs on a schedule: once a week and once a month. This is a practical trade-off: the index is not updated instantly, but the system is simpler and cheaper.
The search itself works like this. An engineer enters text such as "person lying on the road" or supplies a sample image. The query is converted into an embedding, and the database finds similar vectors while applying filters on metadata fields at the same time. The top 1000 results go through feature engineering and end up in the training set.
What the company got, and the caveats
The headline number of the case study: mining data on rare, safety-critical cases dropped from 1 week to 1 hour. WeRide also reports higher productivity for ML developers in data exploration and debugging, as well as shorter model training times thanks to more efficient data I/O. The case study gives no specific percentages for these two improvements, so their scale cannot be assessed.
The caveat is obvious: the piece was published on the vendor's own blog together with the customer. There is no independent comparison with other vector databases, and the cluster cost and search accuracy are not stated. Claims that LanceDB will become an "indispensable tool" should be read as marketing. The team also plans to use the database for analyzing driving style and personalizing rides.
What to take away
First, define your vector database requirements in terms of P99 latency, metadata filters and load on storage, not impressive benchmarks on other people's datasets. In production, almost nobody searches on a "pure" vector; filters are always needed.
Second, if the data goes into training rather than a real-time answer for a user, scheduled reindexing (once a week or once a month) is usually enough.
Third, measure the value of search not in milliseconds but in the length of your team's work cycle. Shrinking the collection of rare examples from a week to an hour delivers more than speeding up training itself. This approach also applies to robots, medical imaging, video surveillance and any product whose quality depends on rare cases.
- Nov 21, 2026VibeCBS collects project nicknames in advance: the agent finds 2.8 times more documents
- Nov 21, 2026FDE job postings grew 4.6x in a year: what a forward deployed engineer actually does at the client's site
- Nov 21, 2026Sakana AI Opens Applied Research Lab for Market and Economic Modeling
- Nov 21, 2026Meta Is Reportedly Preparing the $300 Muse Charm AI Tamagotchi With a Year of Verizon 5G
Key Questions Answered
Why plain SQL didn't work?
WeRide, a Nasdaq-listed company, develops autonomous driving technology from Level 2 to Level 4. It conducts research, testing and operations in more than 30 cities across seven countries, and its autonomous fleet has been running for more than 1700 days. That produces a constant stream of multimodal data. In 2023, Fei Chen took over the data infrastructure team, which was tasked with quickly finding data that matches a specific criterion, above all rare examples from the "long tail" of scenarios.
How the pipeline works?
Images and video from the vehicles are converted into vectors. They are loaded together with metadata into LanceDB, which runs on top of a custom storage cluster built by the same infrastructure team. LanceDB indexes the data for fast search. New data arrives continuously, so reindexing runs on a schedule: once a week and once a month. This is a practical trade-off: the index is not updated instantly, but the system is simpler and cheaper. The search itself works like this.
What the company got, and the caveats?
The headline number of the case study: mining data on rare, safety-critical cases dropped from 1 week to 1 hour. WeRide also reports higher productivity for ML developers in data exploration and debugging, as well as shorter model training times thanks to more efficient data I/O. The case study gives no specific percentages for these two improvements, so their scale cannot be assessed. The caveat is obvious: the piece was published on the vendor's own blog together with the customer.
Ship AI Faster with Founders, Engineers & Weekly Architecture Reviews
Inside Vibe Club (hamidun.com/club) — weekly live masterminds with Zhemal Khamidun, Tech Radar, full LMS access, and 370+ production Claude Code skills.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.