CVPR 2023 - Towards Bridging the Performance Gaps of Joint Energy-based Models

In this episode we discuss Towards Bridging the Performance Gaps of Joint Energy-based Models by Xiulong Yang, Qing Su, Shihao Ji. The paper introduces a variety of training techniques to improve the performance of the Joint Energy-based Model (JEM), which combines a discriminative and a generative model in a single network. The proposed techniques aim to bridge the accuracy gap in classification and the generation quality gap compared to state-of-the-art generative models. The authors incorporate a sharpness-aware minimization framework and exclude data augmentation from the maximum likelihood estimate pipeline to achieve state-of-the-art performance in image classification, generation, calibration, out-of-distribution detection, and adversarial robustness on multiple datasets.