Voyage AI taught search to understand video: 4.65% more accurate than Google and 6 times cheaper
Voyage AI has released voyage-multimodal-3.5, a model that turns text, screenshots, PDFs, and now video into numerical vectors, so a single text query can find the right fragment, such as the moment in a call where the deadline was discussed. On three video test sets, it finds the right clip 4.65% more accurately than Google Multimodal Embedding 001, and a 512×512 video costs roughly 6 times less. For searching lectures and recordings, this replaces a chain of speech recognition plus a frame index with one model.
AI-processed from Voyage AI; edited by Hamidun News
Imagine you have a hundred recordings of calls with slides, and you want to find the moment when the deadline was moved. Ordinary text search won't help here: half the information is in the frames, not in the words. Voyage AI has released voyage-multimodal-3.5, a model that turns text, images, and video into vectors (sets of numbers used to find content that is similar in meaning). On three video test sets, it finds the right content 4.65% more accurately than Google's Multimodal Embedding 001 embedding model, measured by NDCG@10, and one 512×512 pixel video costs roughly 6 times less. NDCG@10 shows how high the genuinely relevant results rank within the top ten.
What the model does
The previous version already handled mixed text and images: screenshots, PDFs, tables, charts, and slides. The new version adds video frames. Video is fed in as an ordered sequence of frames, with every 1120 pixels counted as one token, and the maximum context is 32 thousand tokens. Voyage calls the model the first production video embedding model with Matryoshka support: the vector can be truncated from 2048 to 1024, 512, or 256 dimensions to save index space.
There is also an architectural detail. CLIP-style models, which include Cohere's earlier multimodal models, pass text and images through two separate "towers." This creates a known imbalance: a text query tends to match other text more readily than a relevant image, because text vectors sit closer to one another. Voyage passes all data types through one shared transformer. According to the company, similarity in the shared space then reflects meaning rather than file format.
Test results
Voyage evaluated the model on 18 multimodal datasets: 15 covering document search in image form and 3 covering video search. The query was always text, and the target could be a figure, a photo, text, a document screenshot, or a combination of these. For each task, the company compared the best models available at the time. For each query, it took the top 10 results by cosine similarity.
- On video: 4.65% more accurate than Google Multimodal Embedding 001 and roughly 6 times cheaper per 512×512 video.
- On documents: 4.56% more accurate than Cohere Embed v4 across 15 datasets.
- On plain text (38 datasets across six domains: law, finance, correspondence, code, web, and technical documentation), it trails the flagship text model voyage-3-large by only 0.29%, but costs $0.06 less per million tokens.
There is a caveat. The announcement summary gives the lead over Cohere as 4.56%, while the document section cites 2.26%. These are probably different aggregation methods, but the conclusion is the same: the model does lead Cohere, though by a modest margin. The gap over Google on documents is far larger, 30.57%. However, Voyage notes that Google's average result there is 51.78%, and the chart rounds it to 65% for visual clarity. For video, the only comparison is with Google. Voyage ran all the measurements itself.
Practical use
For video, the company offers three guidelines. First, split long recordings into scenes so that each vector describes one coherent episode. Second, align chunk boundaries with pauses in the speech transcript so that each fragment contains a complete thought. Third, if a scene does not fit within 32 thousand tokens, lower the resolution or the frame rate. The voyageai package includes a Video.from_path utility for preparing clips.
Quantization offers another lever for cutting costs: 32-bit numbers, signed and unsigned 8-bit integers, and a binary format are available, with minimal quality loss. For an index covering millions of video scenes, this translates directly into memory savings.
Who stands to benefit: teams searching training courses, product demos, recordings of calls with slides, or scans of contracts. Previously, you had to transcribe speech, index frames separately, and merge the results. Now you can try a single index. The first 200 million tokens and 150 billion pixels are free, so you can build a prototype on your own data at no cost. Before migrating, run the model on your own set of queries: the vendor's tests do not replace validation on real data.
- Nov 21, 2026Apple Is Building an AI Agent and Making It Harder for Outsiders to Get Access: Permission Now Takes 14 Steps, Not 4
- Nov 21, 2026Gemini 4 Neon shrank the TS Rust binary from 47.44 to 7.97 MB in under 10 hours
- Nov 21, 2026A swimming team captain handed ChatGPT three scheduled tasks and cut her routine work by a multiple
- Nov 21, 2026MIRA Neural Network Renders Rocket League at 20 FPS, but Ping Hits 460 ms
Key Questions Answered
What the model does?
The previous version already handled mixed text and images: screenshots, PDFs, tables, charts, and slides. The new version adds video frames. Video is fed in as an ordered sequence of frames, with every 1120 pixels counted as one token, and the maximum context is 32 thousand tokens. Voyage calls the model the first production video embedding model with Matryoshka support: the vector can be truncated from 2048 to 1024, 512, or 256 dimensions to save index space. There is also an architectural detail.
What are the key details on Test results?
Voyage evaluated the model on 18 multimodal datasets: 15 covering document search in image form and 3 covering video search. The query was always text, and the target could be a figure, a photo, text, a document screenshot, or a combination of these. For each task, the company compared the best models available at the time. For each query, it took the top 10 results by cosine similarity. On video: 4.65% more accurate than Google Multimodal Embedding 001 and roughly 6 times cheaper per 512×512 video.
What are the key details on Practical use?
For video, the company offers three guidelines. First, split long recordings into scenes so that each vector describes one coherent episode. Second, align chunk boundaries with pauses in the speech transcript so that each fragment contains a complete thought. Third, if a scene does not fit within 32 thousand tokens, lower the resolution or the frame rate. The voyageai package includes a Video.from_path utility for preparing clips.
Test Frontier LLMs & Autonomous Agents in One Workspace
Run Claude Opus 5.5, GPT-6, Gemini 3.1, and DeepSeek side-by-side in Nomad AI (nomadnet.ai) — unified model routing, deep research, and multimodal workflows under one subscription.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.