AI Breakdown

CVPR 2023 - Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures

In this episode we discuss Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures
by Eugenia Iofinova, Alexandra Peste, Dan Alistarh. The paper investigates the relationship between neural network pruning and induced bias in Convolutional Neural Networks (CNNs) for computer vision. The authors show that highly-sparse models (with less than 10% remaining weights) can maintain accuracy without increasing bias when compared to dense models. However, at higher sparsities, pruned models exhibit higher uncertainties in their outputs, as well as increased correlations, which are linked to increased bias. The authors propose easy-to-use criteria to establish whether pruning will increase bias and identify samples most susceptible to biased predictions.

CVPR 2023 - Practical Network Acceleration with Tiny Sets

In this episode we discuss Practical Network Acceleration with Tiny Sets
by Guo-Hua Wang, Jianxin Wu. The paper proposes a new method called PRACTISE for accelerating networks using only small training sets. It suggests dropping blocks as a better approach than filter-level pruning for achieving higher acceleration ratio and improved latency-accuracy performance under few-shot settings. The paper introduces a new concept called "recoverability" to measure the difficulty of recovering the compressed network and proposes an algorithm using it to select which blocks to drop. PRACTISE outperforms previous methods by a significant margin and also shows high generalization ability under data-free or out-of-domain data settings.

CVPR 2023 - CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP

In this episode we discuss CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP
by Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, Wenping Wang. The paper explores how Contrastive Language-Image Pre-training (CLIP) knowledge can benefit 3D scene understanding, which has yet to be explored. The authors propose a framework called CLIP2Scene that can transfer CLIP knowledge from 2D image-text pre-trained models to a 3D point cloud network. Experiments conducted on SemanticKITTI, nuScenes, and ScanNet show that the pre-trained 3D network achieves impressive performance on various downstream tasks, including annotation-free and fine-tuning with labeled data for semantic segmentation, outperforming other self-supervised methods.

CVPR 2023 - Zero-Shot Noise2Noise: Efficient Image Denoising without any Data

In this episode we discuss Zero-Shot Noise2Noise: Efficient Image Denoising without any Data
by Youssef Mansour, Reinhard Heckel. The paper proposes a new method for image denoising that does not rely on any training data or knowledge of the noise distribution and is computationally efficient. The proposed method utilizes a simple 2-layer network that can denoise pixel-wise independent noise and outperforms existing dataset-free methods at a reduced cost. The method is motivated by Noise2Noise and Neighbour2Neighbour and achieves a better trade-off between denoising quality, generalization, and computational resources.

CVPR 2023 - Single Image Backdoor Inversion via Robust Smoothed Classifiers

In this episode we discuss Single Image Backdoor Inversion via Robust Smoothed Classifiers
by Mingjie Sun, Zico Kolter. The paper proposes a new method called SmoothInv for identifying backdoor triggers in machine learning models. Previous methods used an optimization process to flip a support set of clean images into the target class. However, the paper demonstrates that SmoothInv can reliably recover the trigger with as few as one image, without requiring an explicit modeling of the trigger or complex regularization schemes. The proposed method is shown to be effective in identifying backdoors in existing models and remains robust against adaptive attackers.

CVPR 2023 - Fake it till you make it: Learning transferable representations from synthetic ImageNet clones

In this episode we discuss Fake it till you make it: Learning transferable representations from synthetic ImageNet clones
by Mert Bulent Sariyildiz, Karteek Alahari, Diane Larlus, Yannis Kalantidis. The paper investigates the ability of synthetic images, generated using Stable Diffusion, to replace real images for training models for ImageNet classification. Using only class names to build the dataset, the study explores the usefulness of synthetic clones of ImageNet for training classification models from scratch. The results show that models trained on synthetic images exhibit strong generalization properties and perform on par with models trained on real data for transfer.

CVPR 2023 - Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations

In this episode we discuss Learning Procedure-aware Video Representation from Instructional Videos and Their Narrations by Yiwu Zhong, Licheng Yu, Yang Bai, Shangwen Li, Xueting Yan, Yin Li. The paper proposes a method to learn a video representation that encodes both action steps and their temporal ordering from a large-scale dataset of web instructional videos without human annotations. The method involves jointly learning a video representation for individual step concepts and a deep probabilistic model to capture temporal dependencies and individual variations in the step ordering. The model achieves significant improvements in step classification and forecasting as well as promising results in zero-shot inference and predicting diverse and plausible steps for incomplete procedures. The code is available on GitHub.

CVPR 2023 - Focused and Collaborative Feedback Integration

In this episode we discuss Focused and Collaborative Feedback Integration
by Qiaoqiao Wei, Hui Zhang, Jun-Hai Yong. The paper proposes Focused and Collaborative Feedback Integration (FCFI), an approach for click-based interactive image segmentation. FCFI fully exploits feedback by focusing on a local area around the new click and correcting the feedback based on high-level feature similarities. It updates the feedback and deep features collaboratively, achieving state-of-the-art performance with less computational overhead than previous methods on four benchmarks. The source code is available on GitHub.

CVPR 2023 - Seeing What You Said: Talking Face Generation Guided

In this episode we discuss Seeing What You Said: Talking Face Generation Guided
by Jiadong Wang, Xinyuan Qian, Malu Zhang, Robby T. Tan, Haizhou Li. The paper discusses the generation of talking faces, also known as speech-to-lip generation, which reconstructs facial motions concerning lips based on speech input. The authors propose using a lip-reading expert to improve the intelligibility of the generated lip regions by penalizing incorrect generation results. They also introduce contrastive learning and a transformer in their approach to enhance lip-speech synchronization and audio-video encoding. The proposal achieved superior performance in reading intelligibility and lip-speech synchronization compared to other state-of-the-art methods.

CVPR 2023 - Instance-Aware Domain Generalization for Face Anti-Spoofing

In this episode we discuss Instance-Aware Domain Generalization for Face Anti-Spoofing
by Qianyu Zhou, Ke-Yue Zhang, Taiping Yao, Xuequan Lu, Ran Yi, Shouhong Ding, Lizhuang Ma. The paper discusses the development of a Face Anti-Spoofing (FAS) system based on Domain Generalization (DG) which aligns features on the instance level without relying on domain labels. This is in contrast to previous methods that focused on domain-level alignment and used artificial domain labels that did not accurately reflect real domain distributions. The proposed Instance-Aware Domain Generalization framework utilizes Asymmetric Instance Adaptive Whitening, Dynamic Kernel Generator, and Categorical Style Assembly to improve generalization and eliminate style-sensitive feature correlation. The paper concludes that their method outperforms state-of-the-art competitors.