CVPR 2023 - What Can Human Sketches Do for Object Detection?

In this episode we discuss What Can Human Sketches Do for Object Detection? by Authors: - Pinaki Nath Chowdhury - Ayan Kumar Bhunia - Aneeshan Sain - Subhadeep Koley - Tao Xiang - Yi-Zhe Song Affiliation: SketchX, CVSSP, University of Surrey, United Kingdom. The paper proposes a new object detection framework that utilizes sketches to detect objects. It is the first attempt to cultivate the expressiveness of sketches for the task of object detection, with instance-aware and part-aware detection capabilities. The model is designed to work without knowing the category of objects beforehand and without requiring bounding boxes or class labels. The framework combines an existing sketch-based image retrieval (SBIR) model with the generalization ability of CLIP to build highly generalizable sketch and photo encoders that can be adapted for object detection. The proposed framework outperforms both supervised and weakly-supervised object detectors on zero-shot setups in standard object detection datasets like PASCAL-VOC and MS-COCO.