CVPR 2023, highlight paper - Beyond mAP: Towards better evaluation of instance segmentation

In this episode we discuss Beyond mAP: Towards better evaluation of instance segmentation by Rohit Jena, Lukas Zhornyak, Nehal Doiphode, Pratik Chaudhari, Vivek Buch, James Gee, Jianbo Shi. The paper proposes new measures to account for duplicate predictions in instance segmentation, which the commonly used Average Precision metric does not penalize. The authors suggest a Semantic Sorting and NMS module to remove duplicates based on a pixel occupancy matching scheme. They argue that this approach can mitigate hedged predictions and preserve AP, allowing for a better trade-off between false positives and high recall. The experiments show that modern segmentation networks have a considerable amount of duplicates, which can be reduced with the proposed method.