CVPR 2023 - Foundation Model Drives Weakly Incremental Learning for Semantic Segmentation

In this episode we discuss Foundation Model Drives Weakly Incremental Learning for Semantic Segmentation by Chaohui Yu, Qiang Zhou, Jingliang Li, Jianlong Yuan, Zhibin Wang, Fan Wang. The paper proposes a novel and data-efficient framework for weakly incremental learning for semantic segmentation (WILSS) called FMWISS. WILSS aims to learn to segment new classes from cheap and readily available image-level labels. The proposed framework uses pre-training based co-segmentation to generate dense pseudo labels and a teacher-student architecture to optimize noisy pseudo masks with a dense contrastive loss. Additionally, memory-based copy-paste augmentation is introduced to address the catastrophic forgetting problem of old classes. The framework achieves superior performance on Pascal VOC and COCO datasets compared to state-of-the-art methods.