AV-FABLE: Researchers Assemble 998 Hours of Video to Hunt Pinpoint Deepfakes

A team of researchers affiliated with Tsinghua University has published AV-FABLE, a benchmark for forensic analysis of long audio-visual videos. It includes 16 650 pairs of a "clean clip and its edited twin," 998 hours of material in total, and 27 editing configurations. Alongside it, the authors propose a method called NFSR. It learns to find manipulations not in the text answers of a multimodal model, but in its frozen internal features.

Why a "fake or not" verdict is not enough

Real-world forgeries rarely span an entire clip. More often they are a short episode inside an authentic recording: a swapped phrase, an inserted fragment, an altered scene. Video-level classification is useless here. If a clip is 90% genuine, it is the other 10% that must be found and shown to an expert.

Existing audio-visual deepfake benchmarks mostly rely on short talking-head clips. Work on temporal localization in long videos is mostly visual. AV-FABLE closes this gap: the scenes are open-domain, the videos are long, and the manipulations are sparse.

An important design detail is the two independent timelines, one for audio and one for video. A voice swap with untouched visuals and, conversely, frame edits with original sound are scored separately. According to the authors, evaluation on a shared timeline hides some errors: a model gets credit for hitting the interval even though it identified the wrong modality.

The language model stays silent, even though the features know

The authors tested audio-visual large language models (AV-LLMs) in zero-shot mode, that is, through an ordinary prompt. The results are weak: detection is close to random and temporal localization is poor. Ask such models "where is the fake here," and they fail to give a meaningful answer.

The researchers then took the frozen perceptual features of the same models and trained supervised "readout" heads on top of them. A strong forensic signal turned up. The conclusion: evidence of manipulation is encoded in the AV-LLM representations, but the language output does not reveal it. The knowledge is there; the interface to it is not.

Two decoders instead of one

NFSR (Native Forensics via Separated Readouts) is built on this observation. The design works like this:

• the backbone is a frozen AV-LLM, and its weights are not fine-tuned;
• an audio decoder reads the features and predicts time intervals of manipulation in the sound;
• a visual decoder does the same for the video track;
• the decoders are separate, so each modality gets its own temporal annotation.

The authors report that NFSR delivers the best audio-visual localization in-domain and in all three out-of-distribution scenarios. One of them is transfer to held-out videos from an external benchmark for long-video forensics.

Benchmark and method at a glance

• Name: AV-FABLE, a dual-timeline benchmark for sparse manipulations in open-domain long audio-visual videos.
• Scale: 16 650 original–manipulated pairs, 998 hours.
• Edits: 27 editing configurations.
• Annotation: separate timelines for audio and video.
• Baseline: zero-shot AV-LLMs produce near-random detection and weak localization.
• Method: NFSR with frozen AV-LLM features and separate audio and visual temporal decoders.
• Evaluation: leads in-domain and in three OOD scenarios, including an external long-video benchmark.
• Release: the dataset, code and model weights are promised to be made open. The preprint is arXiv 2610.33413, and the paper has 25 upvotes on Hugging Face Daily Papers.

For platforms and newsrooms that verify video content, the approach is pragmatic. There is no need to train a huge multimodal model from scratch: you can take an off-the-shelf AV-LLM, freeze it and add lightweight decoders. Separate timelines give the reviewer a concrete result, "audio swap at such-and-such minute," instead of a vague "suspicious." If the weights and data are released openly, as promised, the benchmark will likely become a working standard for comparing detectors of long-form fakes. It will also show how quickly defenses are falling behind generative editing tools.