arxiv Preprint - Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation

In this episode we discuss Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation by Yingqing He, Menghan Xia, Haoxin Chen, Xiaodong Cun, Yuan Gong, Jinbo Xing, Yong Zhang, Xintao Wang, Chao Weng, Ying Shan, Qifeng Chen. This paper presents Animate-A-Story, a framework for generating storytelling videos by customizing existing video clips. The framework includes two modules: Motion Structure Retrieval and Structure-Guided Text-to-Video Synthesis. The first module retrieves relevant video clips based on query texts, while the second module generates coherent videos guided by motion structure and text prompts. The approach proposed in the paper surpasses existing baselines in terms of visual consistency and performance.