Arxiv paper - Towards Understanding Camera Motions in Any Video

In this episode, we discuss Towards Understanding Camera Motions in Any Video by Zhiqiu Lin, Siyuan Cen, Daniel Jiang, Jay Karhade, Hewei Wang, Chancharik Mitra, Tiffany Ling, Yuhan Huang, Sifan Liu, Mingyu Chen, Rushikesh Zawar, Xue Bai, Yilun Du, Chuang Gan, Deva Ramanan. The paper presents CameraBench, a large-scale, expertly annotated video dataset and benchmark for analyzing camera motion using a novel taxonomy developed with cinematographers. It reveals that existing models struggle with either semantic or geometric aspects of camera motion, but fine-tuning generative video-language models on CameraBench improves performance across tasks. The work aims to advance automatic understanding of camera motions, supported by human studies, tutorials, and diverse video applications.