CVPR 2023, highlight paper - Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion

In this episode we discuss Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion by Tengfei Wang, Bo Zhang, Ting Zhang, Shuyang Gu, Jianmin Bao, Tadas Baltrusaitis, Jingjing Shen, Dong Chen, Fang Wen, Qifeng Chen, Baining Guo. The paper proposes a 3D generative model called "Rodin" that uses diffusion models to create 3D digital avatars represented as neural radiance fields in an efficient way. The model utilizes 3D-aware convolution to attend to projected features in the 2D feature plane, preserving the integrity of diffusion in 3D while achieving computational efficiency. The paper also includes latent conditioning for global coherence and hierarchical synthesis to enhance avatar details. The results show that Rodin can generate highly detailed avatars with realistic hairstyles and facial hair.