Amazon shows natural-language search over large video archives built on Nova

Amazon has shown how to build search over large video archives without manual tagging or strict dependence on keywords. Instead of scripted tagging, the system builds multimodal embeddings for audio and image, then finds clips by meaning through OpenSearch.

How the search works

The solution is built on Amazon Nova Multimodal Embeddings and Amazon OpenSearch Service. Videos are uploaded to S3, after which Nova's asynchronous API automatically cuts them into 15-second segments and builds 1024-dimensional vectors in AUDIO_VIDEO_COMBINED mode. This matters: the model accounts for sound as well as picture, so search understands not isolated words in a caption but the context of the scene itself: who is speaking, what is happening in the frame and what the mood of the fragment is.

Separately, AWS suggests running clips through Nova Pro or Nova 2 Lite to generate 10–15 descriptive tags based on a given taxonomy. As a result, the system keeps two indexes: a vector index for semantic search and a text index for keyword search. This setup means you don't have to choose between "smart" search and metadata filtering; you can combine both in a single interface. In effect, the same archive can be browsed with queries like "a person walking along the beach at sunset" as well as with strict text filters.

• Text-to-video search: a natural-language query is turned into an embedding and compared with video segments.
• Similar-video search: the system takes the vector of an already known clip and finds fragments with similar content.
• Hybrid search: k-NN and BM25 results are merged, by default weighted 70% to semantics and 30% to text.

Scale and economics

AWS tested the setup not on a demo set of a couple of files but on a corpus of roughly 792,000 videos from the Multimedia Commons and MEVA datasets. In total that is about 8,480 hours of content, or 30.5 million seconds. Full processing took 41 hours on four c7i.48xlarge instances with 600 parallel workers. Bedrock has a limit of 30 concurrent async jobs per account, so the example uses a job queue that polls status and loads new clips as slots free up.

The cost picture is also fairly transparent. AWS estimates the first year of such a system at roughly $23.6–27.3 thousand, depending on the chosen OpenSearch payment model. Of that, about $18.1 thousand goes to the one-time upload and embedding generation, and the rest to a year of running the search layer. The main expense is not EC2 compute but the embeddings themselves, because Nova is billed by video duration. The breakdown looks like this:

• about $17,096 for generating multimodal embeddings in Amazon Bedrock
• about $571 for auto-tagging with Nova Pro
• about $421 for EC2 compute for batch processing
• $5,544 to $9,240 per year for storage and search in OpenSearch, depending on the payment model

AWS also explains why the example uses 1024-dimensional vectors rather than 3072-dimensional ones: generation cost does not change, but storage becomes roughly three times cheaper with minimal loss of accuracy.

On the search side, the numbers already look production-grade: semantic k-NN responds in about 76 ms, BM25 in 30 ms and hybrid mode in 106 ms. The indexes for the whole corpus take up about 29.8 GB, so even a large video archive does not require exotic infrastructure.

Practical caveats

This material matters not as yet another model announcement but as a ready-made engineering template. AWS effectively shows how to move from manual video tagging to an AI data lake, where search is built around embeddings rather than human descriptions. For teams at media and entertainment companies, this can cover several tasks at once: finding duplicates, navigating the archive, quickly picking b-roll and building internal tools for editors, producers and archivists.

But there are limits. Running it requires Bedrock in the us-east-1 region, OpenSearch 2.11 or newer, S3 and configured IAM permissions. Speed and cost depend directly on clip length: in the test, a 45-second video took about 70 seconds to process. If your metadata is good, AWS advises raising the share of text search, up to 50/50. And if the library keeps growing, the processing logic can be moved to AWS Batch and scaled in parts.

What this means

Amazon is showing that multimodal video search can now be treated not as a research toy but as a well-understood infrastructure pattern. For media teams, it is a chance to stop living in manual tags and finally search the archive the way people actually phrase queries: in plain language.