Meta released Tribe v2 — a model that predicts the brain's response to video, audio, and text

Meta released TRIBE v2 — a model that predicts brain responses to video, audio, and text based on fMRI data. The project promises to accelerate neuroscience: instead of scanning new people, researchers can first test hypotheses in simulation.

What Meta released

TRIBE v2 is a trimodal model designed for in-silico brain research. It takes video, audio, and language as input, converts them into representations from pre-trained models, and then predicts the pattern of neural activity that fMRI would detect across the entire brain. For Meta, this is an attempt to move away from the old logic, where separate narrow models were built for each cognitive function: movement was studied separately, faces separately, speech separately.

TRIBE v2 should connect these pieces into one system that works simultaneously across different types of stimuli and tasks. According to Meta, the new version provides 70 times higher spatial resolution than comparable solutions, and consistently outperforms classical linear encoding models. The main distinction from many previous works is zero-shot generalization: the model can predict responses for new people, new tasks, and even new languages without separate retraining for each scenario.

In their blog, Meta directly calls TRIBE v2 a tool that should work as a "digital twin" of neural activity and allow part of experiments to be conducted without recruiting new volunteers.

Meta calls TRIBE v2 a "digital twin" of human neural activity.

What it was trained on

The foundation of TRIBE v2 is a unified corpus of more than 1,000 hours of fMRI and 720 participants. Training combined both "deep" datasets with many recordings per person and "broad" samples with hundreds of people and short sessions. Subjects watched movies, listened to podcasts and audio, viewed images and texts, and participated in more controlled laboratory paradigms. This is important: the model learns not from a single genre of stimuli, but from a fairly broad slice of what humans see, hear, and read.

• films and video clips
• podcasts and other audio stimuli
• texts and individual sentences
• experimental tasks like showing objects and words

Meta also released the paper, code, model weights, and a demo. This is not just a press release: researchers can run their own video, audio, or text stimulus and see the predicted neural response across the cortex. In their repository, the company notes that basic inference provides an averaged response of an "average" participant on a cortical grid of approximately 20,000 vertices — meaning we're talking about a working tool, not just a beautiful concept.

What the tests showed

In experiments, TRIBE v2 predicted cortical and some subcortical responses above chance level across different tasks. The authors show a fairly expected but important picture: when listening to podcasts, temporal regions are more prominent; when watching videos, visual cortex; and multimodal stimuli produce significant responses across much of the cortex. Against this backdrop, the comparison with a strong baseline linear method on the same features is particularly telling: the advantage is explained not by better input, but by the architecture itself, which nonlinearly combines video, audio, and language.

Meta separately tested how the model performs on new people and new studies. On some test sets, TRIBE v2's predictions were closer to the averaged group response than actual recordings of most individual participants. On the Human Connectome Project dataset, the authors report a correlation of around **0.

4**, which is roughly twice the median performance of a single participant. At the same time, the authors honestly acknowledge the system's limits: fMRI itself is a slow and indirect measure of brain activity, so the model doesn't see millisecond-level neuronal dynamics, doesn't cover smell, balance, and touch, and currently describes the brain as a passive observer rather than an active agent.

What it means

TRIBE v2 doesn't read minds and doesn't replace the laboratory, but it sets a new scale for computational neuroscience. If Meta's approach holds up to external scrutiny, researchers will be able to test hypotheses more cheaply, design experiments, and faster translate ideas from neuroscience to AI models and back.