CVPR 2023 - DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation

In this episode we discuss DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation by Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, Kfir Aberman. The paper discusses a new approach to personalize text-to-image diffusion models by fine-tuning the pre-trained model with a few images of a particular subject, allowing the model to learn a unique identifier associated with that subject. The unique identifier enables the synthesis of novel photorealistic images of the subject in different scenes. Through a new autogenous class-specific prior preservation loss, the technique facilitates subject synthesis in diverse poses, lighting conditions, and views, providing impressive results for various applications, including subject recontextualization, text-guided view synthesis, and artistic rendering.