CVPR 2023 - FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentation

In this episode we discuss FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentation by Jie Qin, Jie Wu, Pengxiang Yan, Ming Li, Ren Yuxi, Xuefeng Xiao, Yitong Wang, Rui Wang, Shilei Wen, Xin Pan, Xingang Wang. The paper proposes FreeSeg, a generic framework for unified, universal, and open-vocabulary image segmentation. Existing methods use specialized architectures or parameters to tackle specific segmentation tasks, leading to fragmentation and hindered uniformity. FreeSeg optimizes an all-in-one network through one-shot training and uses the same architecture and parameters for diverse segmentation tasks. Adaptive prompt learning improves model robustness in multi-task scenarios, and experimental results show that FreeSeg outperforms task-specific architectures by a large margin. The project page is https://FreeSeg.github.io.