Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseAPI / Product · ActionableVoyage AI→ original

Voyage AI taught search to understand video: 4.65% more accurate than Google and 6 times cheaper

Voyage AI has released voyage-multimodal-3.5, a model that turns text, screenshots, PDFs, and now video into numerical vectors, so a single text query can find the right fragment, such as the moment in a call where the deadline was discussed. On three video test sets, it finds the right clip 4.65% more accurately than Google Multimodal Embedding 001, and a 512×512 video costs roughly 6 times less. For searching lectures and recordings, this replaces a chain of speech recognition plus a frame index with one model.

AI-processed from Voyage AI; edited by Hamidun News
Voyage AI taught search to understand video: 4.65% more accurate than Google and 6 times cheaper
Source: Voyage AI. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceVoyage AI ↗
Editor-in-Chief
Key metric #1The vector can be shortened to 1024, 512, or 256 numbers instead of 2048 (Matryoshka).
Key metric #2On documents, it leads Cohere Embed v4 by 4.56% across 15 datasets.
Key metric #3The first 200 million tokens and 150 billion pixels are free.

Imagine you have a hundred recordings of calls with slides, and you want to find the moment when the deadline was moved. Ordinary text search won't help here: half the information is in the frames, not in the words. Voyage AI has released voyage-multimodal-3.5, a model that turns text, images, and video into vectors (sets of numbers used to find content that is similar in meaning). On three video test sets, it finds the right content 4.65% more accurately than Google's Multimodal Embedding 001 embedding model, measured by NDCG@10, and one 512×512 pixel video costs roughly 6 times less. NDCG@10 shows how high the genuinely relevant results rank within the top ten.

What the model does

The previous version already handled mixed text and images: screenshots, PDFs, tables, charts, and slides. The new version adds video frames. Video is fed in as an ordered sequence of frames, with every 1120 pixels counted as one token, and the maximum context is 32 thousand tokens. Voyage calls the model the first production video embedding model with Matryoshka support: the vector can be truncated from 2048 to 1024, 512, or 256 dimensions to save index space.

There is also an architectural detail. CLIP-style models, which include Cohere's earlier multimodal models, pass text and images through two separate "towers." This creates a known imbalance: a text query tends to match other text more readily than a relevant image, because text vectors sit closer to one another. Voyage passes all data types through one shared transformer. According to the company, similarity in the shared space then reflects meaning rather than file format.

Test results

Voyage evaluated the model on 18 multimodal datasets: 15 covering document search in image form and 3 covering video search. The query was always text, and the target could be a figure, a photo, text, a document screenshot, or a combination of these. For each task, the company compared the best models available at the time. For each query, it took the top 10 results by cosine similarity.

  • On video: 4.65% more accurate than Google Multimodal Embedding 001 and roughly 6 times cheaper per 512×512 video.
  • On documents: 4.56% more accurate than Cohere Embed v4 across 15 datasets.
  • On plain text (38 datasets across six domains: law, finance, correspondence, code, web, and technical documentation), it trails the flagship text model voyage-3-large by only 0.29%, but costs $0.06 less per million tokens.

There is a caveat. The announcement summary gives the lead over Cohere as 4.56%, while the document section cites 2.26%. These are probably different aggregation methods, but the conclusion is the same: the model does lead Cohere, though by a modest margin. The gap over Google on documents is far larger, 30.57%. However, Voyage notes that Google's average result there is 51.78%, and the chart rounds it to 65% for visual clarity. For video, the only comparison is with Google. Voyage ran all the measurements itself.

Practical use

For video, the company offers three guidelines. First, split long recordings into scenes so that each vector describes one coherent episode. Second, align chunk boundaries with pauses in the speech transcript so that each fragment contains a complete thought. Third, if a scene does not fit within 32 thousand tokens, lower the resolution or the frame rate. The voyageai package includes a Video.from_path utility for preparing clips.

Quantization offers another lever for cutting costs: 32-bit numbers, signed and unsigned 8-bit integers, and a binary format are available, with minimal quality loss. For an index covering millions of video scenes, this translates directly into memory savings.

Who stands to benefit: teams searching training courses, product demos, recordings of calls with slides, or scans of contracts. Previously, you had to transcribe speech, index frames separately, and merge the results. Now you can try a single index. The first 200 million tokens and 150 billion pixels are free, so you can build a prototype on your own data at no cost. Before migrating, run the model on your own set of queries: the vendor's tests do not replace validation on real data.

Story

Key Questions Answered

What the model does?

The previous version already handled mixed text and images: screenshots, PDFs, tables, charts, and slides. The new version adds video frames. Video is fed in as an ordered sequence of frames, with every 1120 pixels counted as one token, and the maximum context is 32 thousand tokens. Voyage calls the model the first production video embedding model with Matryoshka support: the vector can be truncated from 2048 to 1024, 512, or 256 dimensions to save index space. There is also an architectural detail.

What are the key details on Test results?

Voyage evaluated the model on 18 multimodal datasets: 15 covering document search in image form and 3 covering video search. The query was always text, and the target could be a figure, a photo, text, a document screenshot, or a combination of these. For each task, the company compared the best models available at the time. For each query, it took the top 10 results by cosine similarity. On video: 4.65% more accurate than Google Multimodal Embedding 001 and roughly 6 times cheaper per 512×512 video.

What are the key details on Practical use?

For video, the company offers three guidelines. First, split long recordings into scenes so that each vector describes one coherent episode. Second, align chunk boundaries with pauses in the speech transcript so that each fragment contains a complete thought. Third, if a scene does not fit within 32 thousand tokens, lower the resolution or the frame rate. The voyageai package includes a Video.from_path utility for preparing clips.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Unified AI Workspace · Nomad

Test Frontier LLMs & Autonomous Agents in One Workspace

Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.

Try Nomad AI →Launch Nomad Agent
What do you think?
Loading comments…