CVPR 2023 - Self-Supervised Video Forensics by Audio-Visual Anomaly Detection

In this episode we discuss Self-Supervised Video Forensics by Audio-Visual Anomaly Detection by Chao Feng, Ziyang Chen, Andrew Owens. The paper proposes a method for detecting inconsistencies between the visual and audio signals in manipulated videos using anomaly detection. The method trains an autoregressive model on real, unlabeled data to generate audio-visual feature sequences capturing temporal synchronization. The model flags videos with a low probability of being genuine at test time and achieves strong performance in detecting manipulated speech videos, despite being trained solely on real videos.