arxiv preprint - DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos

In this episode, we discuss DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videos by Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, Ying Shan. DepthCrafter is a novel method for estimating temporally consistent depth in open-world videos without needing additional data like camera poses or optical flow. It generalizes to diverse video content by utilizing a three-stage training strategy rooted in a pre-trained image-to-video diffusion model, enabling it to handle up to 110-frame sequences. Evaluations show DepthCrafter's state-of-the-art performance, bolstering applications like depth-based visual effects and conditional video generation.