CVPR 2023 - PolyFormer: Referring Image Segmentation as Sequential Polygon Generation

In this episode we discuss PolyFormer: Referring Image Segmentation as Sequential Polygon Generation by Jiang Liu, Hui Ding, Zhaowei Cai, Yuting Zhang, Ravi Kumar Satzoda, Vijay Mahadevan, R. Manmatha. The paper presents a new approach to referring image segmentation that uses sequential polygon generation instead of directly predicting pixel-level masks. The method, called Polygon Transformer (PolyFormer), takes a sequence of image patches and text query tokens as input and outputs a sequence of polygon vertices. A regression-based decoder is also proposed for more accurate geometric localization. In experiments, PolyFormer outperforms prior methods on challenging datasets and shows strong generalization ability on referring video segmentation without fine-tuning.