AV-FABLE: Researchers Assemble 998 Hours of Video to Hunt for Pinpoint Deepfakes

A team of researchers affiliated with Tsinghua University has published AV-FABLE, a forensic benchmark for long audio-visual videos. It comprises 16,650 pairs of "clean clip — edited twin," totaling 998 hours of footage and 27 editing configurations. Alongside it, the authors propose the NFSR method, which learns to spot manipulations not in the text output of a multimodal model's responses, but in its frozen internal features.

Why a Simple "Fake or Not" Verdict Isn't Enough

Real-world fakes rarely span an entire video. More often, it's a short episode embedded in an otherwise genuine recording: a swapped phrase, an inserted segment, an altered scene. Whole-video classification is useless here. A clip might be 90% authentic, and it's precisely that other 10% that needs to be located and flagged for a human expert.

Existing audio-visual deepfake benchmarks are mostly built on short talking-head clips. Work on temporal localization in long videos has largely focused on the visual channel alone. AV-FABLE closes that gap: the scenes are open-domain, the clips are long, and the manipulations are sparse.

An important design detail is the use of two independent timelines — one for audio, one for video. A voice swap over an untouched picture, and conversely a frame edit over original audio, are scored separately. According to the authors, evaluating against a single merged timeline hides certain errors: a model can get credit for hitting the right time interval even though it identified the wrong modality.

The

Language Model Stays Silent, Even Though the Features Know

The authors tested audio-visual large language models (AV-LLMs) in zero-shot mode — that is, via a plain prompt. The result was weak: detection was close to random, and temporal localization was poor. Asked "where's the fake here," these models don't give a substantive answer.

The researchers then took the frozen perceptual features of those same models and trained supervised "readout" heads on top of them. A strong forensic signal emerged. The conclusion: evidence of manipulation is encoded in AV-LLM representations, but the language output doesn't surface it. The knowledge is there; there's just no interface to it.

Two Decoders Instead of One

That observation is the basis for NFSR (Native Forensics via Separated Readouts). The scheme works like this:

• backbone — a frozen AV-LLM, its weights are not fine-tuned;
• audio decoder reads the features and predicts temporal intervals of manipulation in the audio;
• visual decoder does the same for the video stream;
• the decoders are separate, so each modality gets its own temporal annotation.

According to the authors, NFSR delivers the best audio-visual localization both within domain and across all three out-of-distribution scenarios. One of those scenarios is transfer to held-out videos from an external benchmark for long-video forensics.

Benchmark and Method Parameters

• Name: AV-FABLE, a dual-timeline benchmark for sparse manipulations in open-domain long audio-visual videos.
• Scale: 16,650 "original vs. manipulated" pairs, 998 hours.
• Edits: 27 editing configurations.
• Annotation: separate timelines for audio and video.
• Baseline: zero-shot AV-LLMs show near-random detection and weak localization.
• Method: NFSR, using frozen AV-LLM features with separate audio and visual temporal decoders.
• Evaluation: leads in-domain and across three OOD scenarios, including an external long-video benchmark.
• Publication: the dataset, code, and model weights are promised to be open-sourced. Preprint — arXiv 2610.33413, with 25 upvotes on Hugging Face Daily Papers.

For platforms and newsrooms that vet video content, the approach is pragmatic. There's no need to train a massive multimodal model from scratch: you can take an existing AV-LLM, freeze it, and bolt on lightweight decoders. Separate timelines give a reviewer a concrete result — "audio swap at such-and-such minute" instead of a vague "suspicious." If the weights and data are released as promised, the benchmark will likely become a working standard for comparing long-form deepfake detectors. It will also reveal just how fast the gap between defenses and generative editing tools is widening.