CVPR 2023 - Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable Style

In this episode we discuss Zero-Shot Everything Sketch-Based Image Retrieval, and in Explainable Style by Authors: 1. Fengyin Lin 2. Mingkang Li 3. Da Li 4. Timothy Hospedales 5. Yi-Zhe Song Affiliations: 1. Beijing University of Posts and Telecommunications 2. Samsung AI Centre, Cambridge 3. University of Edinburgh 4. SketchX, CVSSP, University of Surrey. The paper presents a novel approach to zero-shot sketch-based image retrieval (ZS-SBIR) that tackles all variants of the problem using just one network. The authors aim to make the matching process more explainable, and achieve this through a transformer-based cross-modal network that compares groups of key local patches. The network includes three novel components: a self-attention module, a cross-attention module, and a kernel-based relation network. Experiment results show superior performance across all ZS-SBIR settings, and the explainable goal is achieved through visualizing cross-modal token correspondences and sketch to photo synthesis. Code and models are available for reproducibility.