Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseInsight · On RadarLanceDB Blog→ original

WeRide Uses LanceDB to Cut Search for Rare Dangerous Driving Footage from a Week to an Hour

Rare situations, such as a person lying on the road, are essential for training a self-driving car, but plain SQL cannot find them among billions of frames. WeRide adopted the LanceDB vector database: an engineer types a phrase or uploads an image and gets 1000 similar frames in milliseconds. Collecting these dangerous cases dropped from 1 week to 1 hour. If you have a lot of unlabeled images, semantic search will pay for itself quickly.

AI-processed from LanceDB Blog; edited by Hamidun News
WeRide Uses LanceDB to Cut Search for Rare Dangerous Driving Footage from a Week to an Hour
Source: LanceDB Blog. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceLanceDB Blog ↗
Editor-in-Chief
P99 in milliseconds99% of queries across billions of records stay within the limit
Key metric #2The index is rebuilt once a week and once a month, so freshness is not critical
Key metric #3Embeddings and metadata are stored in WeRide's own storage

For a self-driving car, the hardest part is not ordinary road driving but rare, dangerous situations, such as a person lying on the roadway. Such episodes are vanishingly rare in recordings from cameras, radar and lidar, and models learn worst from exactly these cases. Previously, WeRide engineers needed a full week to collect such cases for training.

Since adopting the LanceDB vector database, which searches not by exact values in columns but by the meaning of an image or a phrase, the process takes about an hour. For any team with large volumes of images, video or other "heavy" data without manual labels, this is a clear example of how faster search speeds up the whole cycle: find the problem, collect data, retrain.

Why plain SQL didn't work

WeRide, a Nasdaq-listed company, develops autonomous driving technology from Level 2 to Level 4. It conducts research, testing and operations in more than 30 cities across seven countries, and its autonomous fleet has been running for more than 1700 days. That produces a constant stream of multimodal data. In 2023, Fei Chen took over the data infrastructure team, which was tasked with quickly finding data that matches a specific criterion, above all rare examples from the "long tail" of scenarios.

SQL is powerless here. A query like "find frames where a person is lying on the ground" cannot be written as a condition on columns if the table has no such label. So images and text are converted into vectors (embeddings), and the database looks for the nearest ones by meaning.

The platform requirements were strict:

  • P99 latency in milliseconds (that is, 99% of queries fall within that limit) when returning the top 1000 results for a text or image query across billions of records;
  • vector search combined with metadata filters;
  • minimal bandwidth consumption, so as not to interfere with the fast storage that feeds model training;
  • reasonable cost and simple maintenance as data grows.

How the pipeline works

Images and video from the vehicles are converted into vectors. They are loaded together with metadata into LanceDB, which runs on top of a custom storage cluster built by the same infrastructure team. LanceDB indexes the data for fast search. New data arrives continuously, so reindexing runs on a schedule: once a week and once a month. This is a practical trade-off: the index is not updated instantly, but the system is simpler and cheaper.

The search itself works like this. An engineer enters text such as "person lying on the road" or supplies a sample image. The query is converted into an embedding, and the database finds similar vectors while applying filters on metadata fields at the same time. The top 1000 results go through feature engineering and end up in the training set.

What the company got, and the caveats

The headline number of the case study: mining data on rare, safety-critical cases dropped from 1 week to 1 hour. WeRide also reports higher productivity for ML developers in data exploration and debugging, as well as shorter model training times thanks to more efficient data I/O. The case study gives no specific percentages for these two improvements, so their scale cannot be assessed.

The caveat is obvious: the piece was published on the vendor's own blog together with the customer. There is no independent comparison with other vector databases, and the cluster cost and search accuracy are not stated. Claims that LanceDB will become an "indispensable tool" should be read as marketing. The team also plans to use the database for analyzing driving style and personalizing rides.

What to take away

First, define your vector database requirements in terms of P99 latency, metadata filters and load on storage, not impressive benchmarks on other people's datasets. In production, almost nobody searches on a "pure" vector; filters are always needed.

Second, if the data goes into training rather than a real-time answer for a user, scheduled reindexing (once a week or once a month) is usually enough.

Third, measure the value of search not in milliseconds but in the length of your team's work cycle. Shrinking the collection of rare examples from a week to an hour delivers more than speeding up training itself. This approach also applies to robots, medical imaging, video surveillance and any product whose quality depends on rare cases.

Story

Key Questions Answered

Why plain SQL didn't work?

WeRide, a Nasdaq-listed company, develops autonomous driving technology from Level 2 to Level 4. It conducts research, testing and operations in more than 30 cities across seven countries, and its autonomous fleet has been running for more than 1700 days. That produces a constant stream of multimodal data. In 2023, Fei Chen took over the data infrastructure team, which was tasked with quickly finding data that matches a specific criterion, above all rare examples from the "long tail" of scenarios.

How the pipeline works?

Images and video from the vehicles are converted into vectors. They are loaded together with metadata into LanceDB, which runs on top of a custom storage cluster built by the same infrastructure team. LanceDB indexes the data for fast search. New data arrives continuously, so reindexing runs on a schedule: once a week and once a month. This is a practical trade-off: the index is not updated instantly, but the system is simpler and cheaper. The search itself works like this.

What the company got, and the caveats?

The headline number of the case study: mining data on rare, safety-critical cases dropped from 1 week to 1 hour. WeRide also reports higher productivity for ML developers in data exploration and debugging, as well as shorter model training times thanks to more efficient data I/O. The case study gives no specific percentages for these two improvements, so their scale cannot be assessed. The caveat is obvious: the piece was published on the vendor's own blog together with the customer.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Private AI Builders Community · Vibe Club

Ship AI Faster with Founders, Engineers & Weekly Architecture Reviews

Inside Vibe Club (hamidun.com/club) — weekly live masterminds with Zhemal Khamidun, Tech Radar, full LMS access, and 370+ production Claude Code skills.

Join Vibe Club →Explore Hamidun Academy
What do you think?
Loading comments…