CVPR 2023 - Detecting and Grounding Multi-Modal Media Manipulation

In this episode we discuss Detecting and Grounding Multi-Modal Media Manipulation by Rui Shao, Tianxing Wu, Ziwei Liu. This paper discusses a new research problem for detecting and grounding multi-modal media manipulation, which requires deeper reasoning across different modalities. The authors propose a new dataset and a novel model called HierArchical Multi-modal Manipulation rEasoning tRansformer (HAMMER) to fully capture the fine-grained interaction between different modalities. Dedicated manipulation detection and grounding heads are integrated from shallow to deep levels based on the interacted multi-modal information. The authors conduct comprehensive experiments and set up rigorous evaluation metrics, demonstrating the superiority of their model and revealing valuable observations to facilitate future research in multi-modal media manipulation.