AI Breakdown

CVPR 2023 - CrOC: Cross-View Online Clustering for Dense Visual Representation Learning

In this episode we discuss CrOC: Cross-View Online Clustering for Dense Visual Representation Learning
by Thomas Stegmüller, Tim Lebailly, Behzad Bozorgtabar, Tinne Tuytelaars, Jean-Philippe Thiran. The paper proposes a method called Cross-view consistency objective with an Online Clustering (CrOC) to learn dense visual representations without labels in scene-centric data. The method uses an online clustering algorithm that operates on both views' features, avoiding issues with content not represented in both views and ambiguous object matching. The proposed method shows excellent performance in linear and unsupervised segmentation transfer tasks on various datasets and video object segmentation. Pre-trained models and code are publicly available.

CVPR 2023 - Efficient Map Sparsification Based on 2D and 3D Discretized Grids

In this episode we discuss Efficient Map Sparsification Based on 2D and 3D Discretized Grids
by Xiaoyu Zhang, Yun-Hui Liu. The paper proposes an efficient linear approach for map sparsification, which involves selecting a subset of landmarks from a larger map for robot navigation. Existing methods require heavy computation and memory capacity, especially for large-scale environments. The proposed approach uses a 2D discretized grid for landmark selection and introduces a space constraint term based on 3D grids to address the impact of different spatial distributions. The experiments demonstrate that the proposed method outperforms previous methods in both efficiency and performance. Relevant codes will be released on GitHub.

CVPR 2023 - Learning Generative Structure Prior for Blind Text Image Super-resolution

In this episode we discuss Learning Generative Structure Prior for Blind Text Image Super-resolution
by Xiaoming Li, Wangmeng Zuo, Chen Change Loy. This paper proposes a novel prior for blind text image super-resolution (SR), focusing on character structure, which can deal with diverse font styles and unknown degradation. The authors store discrete features for each character in a codebook to drive a StyleGAN to generate high-resolution structural details that aid text SR. The proposed structure prior exerts stronger character-specific guidance than previous methods based on character recognition, resulting in compelling performance on synthetic and real datasets. The code for the proposed approach is available on GitHub.

CVPR 2023 - Re-thinking Federated Active Learning based on Inter-class Diversity

In this episode we discuss Re-thinking Federated Active Learning based on Inter-class Diversity
by SangMook Kim, Sangmin Bae, Hwanjun Song, Se-Young Yun. The paper discusses the use of federated active learning (FAL) frameworks in situations where a significant amount of unlabeled data is present. The authors demonstrate that the effectiveness of available query selector models depends on the global and local inter-class diversity. They propose LoGo, a FAL sampling strategy that integrates both "global" and "local-only" models and consistently outperforms six other active learning strategies in various experimental settings. The code for LoGo is available on GitHub.

CVPR 2023 - Super-Resolution Neural Operator

In this episode we discuss Super-Resolution Neural Operator
by Min Wei, Xuesong Zhang. The paper proposes a deep learning framework called Super-resolution Neural Operator (SRNO) that can generate high-resolution images from their low-resolution counterparts. It works by learning the mapping between the function spaces of the LR and HR image pairs, embedding the LR input into a higher-dimensional latent representation space, iteratively approximating the implicit image function with kernel integral mechanisms, and generating the RGB representation at the target coordinates. The SRNO outperforms existing continuous SR methods in terms of both accuracy and running time.

CVPR 2023 - Unsupervised Contour Tracking of Live Cells by Mechanical and Cycle Consistency Losses

In this episode we discuss Unsupervised Contour Tracking of Live Cells by Mechanical and Cycle Consistency Losses
by Junbong Jang, Kwonmoo Lee, Tae-Kyun Kim. The paper proposes a deep learning-based method for tracking the dynamic changes of cellular morphology in live cell videos. The proposed method includes point correspondence and considering local shapes and textures on the contour, which previous methods did not. Unsupervised learning is used, consisting of mechanical and cyclical consistency losses, to train the contour tracker. The proposed method outperforms existing methods and is publicly available.

CVPR 2023 - Probabilistic Prompt Learning for Dense Prediction

In this episode we discuss Probabilistic Prompt Learning for Dense Prediction
by Hyeongjun Kwon, Taeyong Song, Somi Jeong, Jin Kim, Jinhyun Jang, Kwanghoon Sohn. This paper proposes a new approach called "probabilistic prompt learning" to improve the performance of dense prediction tasks. The authors introduce learnable class-agnostic attribute prompts to describe universal attributes across object classes, which are combined with class information and visual-context knowledge to create a class-specific textual distribution. Text representations are then sampled and used to guide the dense prediction task using a probabilistic pixel-text matching loss, resulting in improved stability and generalization capabilities. The effectiveness of the proposed method is demonstrated through extensive experiments and ablation studies.

CVPR 2023 - SunStage: Portrait Reconstruction and Relighting using the Sun as a Light Stage

In this episode we discuss SunStage: Portrait Reconstruction and Relighting using the Sun as a Light Stage
by Yifan Wang, Aleksander Holynski, Xiuming Zhang, Xuaner Zhang. The paper presents SunStage, a lightweight alternative to a light stage that captures facial appearance and relighting data using only a smartphone camera and the sun. The method requires the user to capture a selfie video outdoors and uses the varying angles between the sun and face for joint reconstruction of facial geometry, reflectance, camera pose, and lighting parameters. The approach is able to reconstruct detailed facial appearance and geometry, enabling compelling effects such as relighting, novel view synthesis, and reflectance editing.

CVPR 2023 - Feature Separation and Recalibration for Adversarial Robustness

In this episode we discuss Feature Separation and Recalibration for Adversarial Robustness
by Woo Jae Kim, Yoonki Cho, Junsik Jung, Sung-Eui Yoon. The paper proposes a novel approach called Feature Separation and Recalibration (FSR) to improve the robustness of deep neural networks against adversarial attacks. The FSR method recalibrates the non-robust feature activations, which are responsible for model mispredictions under adversarial attacks, by disentangling them from the robust feature activations and adjusting them to restore potentially useful cues for correct model predictions. The results of extensive experiments show that FSR outperforms traditional deactivation techniques and improves the robustness of existing adversarial training methods by up to 8.57% with minimal computational overhead.

CVPR 2023 - DETR with Additional Global Aggregation for Cross-domain Weakly Supervised Object Detection

In this episode we discuss DETR with Additional Global Aggregation for Cross-domain Weakly Supervised Object Detection
by Zongheng Tang, Yifan Sun, Si Liu, Yi Yang. The paper proposes a method for cross-domain weakly supervised object detection (CDWSOD) by adapting the detector from source to target domain through weak supervision using DETR (transformers-based object detection model). The proposed method, DETR-GA, simultaneously makes "instance-level + image-level" predictions and utilizes "strong + weak" supervisions. The method uses query-based aggregation that helps in locating corresponding positions, excluding distractions from non-relevant regions, and making strong and weak supervision mutually benefit each other for domain alignment. Extensive experiments show that DETR-GA significantly improves cross-domain detection accuracy and advances the state-of-the-art.