CVPR 2023 - Regularized Vector Quantization for Tokenized Image Synthesis

In this episode we discuss Regularized Vector Quantization for Tokenized Image Synthesis by Jiahui Zhang, Fangneng Zhan, Christian Theobalt, Shijian Lu. The paper proposes a regularized vector quantization framework for quantizing images into discrete representations, which has been a fundamental problem in generative modeling. Existing approaches either learn the discrete representation deterministically or stochastically, but suffer from various drawbacks, such as severe codebook collapse, low codebook utilization, and perturbed reconstruction objective. The proposed framework mitigates these issues effectively by applying regularization from two perspectives and introducing a probabilistic contrastive loss. Experiments show that the framework consistently outperforms prevailing vector quantization methods across various generative models.