AI Breakdown

CVPR 2023 - Neural Residual Radiance Fields for Streamably Free-Viewpoint Videos

In this episode we discuss Neural Residual Radiance Fields for Streamably Free-Viewpoint Videos
by Liao Wang, Qiang Hu, Qihan He, Ziyu Wang, Jingyi Yu, Tinne Tuytelaars, Lan Xu, Minye Wu. The paper introduces a new technique called Residual Radiance Field (ReRF), a compact neural representation for achieving real-time free-view rendering on long-duration dynamic scenes. ReRF explicitly models residual information between adjacent timestamps in the spatial-temporal feature space using a global coordinate-based tiny MLP as the feature decoder. The paper also presents a special free-view video (FVV) codec based on ReRF that achieves three orders of magnitude compression rate and provides a companion ReRF player to support online streaming of long-duration FVVs of dynamic scenes. Extensive experiments demonstrate the effectiveness of ReRF for compactly representing dynamic radiance fields, enabling an unprecedented free-viewpoint viewing experience in speed and quality.

CVPR 2023 - DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering

In this episode we discuss DANI-Net: Uncalibrated Photometric Stereo by Differentiable Shadow Handling, Anisotropic Reflectance Modeling, and Neural Inverse Rendering
by Zongrui Li, Qian Zheng, Boxin Shi, Gang Pan, Xudong Jiang. The paper proposes a deep learning approach, called DANI-Net, to solve the challenging problem of uncalibrated photometric stereo (UPS) which is complicated by unknown lighting. UPS is particularly difficult for non-Lambertian objects with complex shapes and irregular shadows, and for general materials with complex reflectance such as anisotropic reflectance. Unlike previous methods that use non-differentiable shadow maps and assume isotropic material, DANI-Net benefits from cues of shadow and anisotropic reflectance through two differentiable paths, resulting in superior and robust performance on multiple real-world datasets.

CVPR 2023 - Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-shot Learning with Hyperspherical Embeddings

In this episode we discuss Hubs and Hyperspheres: Reducing Hubness and Improving Transductive Few-shot Learning with Hyperspherical Embeddings
by Daniel J. Trosten, Rwiddhi Chakraborty, Sigurd Løkse, Kristoffer Knutsen Wickstrøm, Robert Jenssen, Michael C. Kampffmeyer. This paper proposes two approaches to address the hubness problem in distance-based classification in transductive few-shot learning. The authors prove that uniform distribution of representations on the hypersphere can eliminate hubness and the proposed approaches optimize a tradeoff between uniformity and local similarity preservation, reducing hubness while retaining class structure. Experiment results show that the proposed methods significantly improve transductive few-shot learning accuracy for a variety of classifiers.

CVPR 2023 - TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation

In this episode we discuss TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation
by Taeyeop Lee, Jonathan Tremblay, Valts Blukis, Bowen Wen, Byeong-Uk Lee, Inkyu Shin, Stan Birchfield, In So Kweon, Kuk-Jin Yoon. In this paper, the authors propose a method called "Test-Time Adaptation for Category-level Object Pose Estimation" or TTA-COPE, for addressing source-to-target domain gaps. They design a pose ensemble approach using pose-aware confidence and a self-training loss. Unlike previous methods, TTA-COPE processes test data in a sequential, online manner and does not require access to the source domain at runtime. Experimental results show improved category-level object pose performance under semi-supervised and unsupervised settings. The project page for TTA-COPE is available at https://taeyeop.com/ttacope.

CVPR 2023 - Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures

In this episode we discuss Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures
by Eugenia Iofinova, Alexandra Peste, Dan Alistarh. The paper investigates the relationship between neural network pruning and induced bias in Convolutional Neural Networks (CNNs) for computer vision. The authors show that highly-sparse models (with less than 10% remaining weights) can maintain accuracy without increasing bias when compared to dense models. However, at higher sparsities, pruned models exhibit higher uncertainties in their outputs, as well as increased correlations, which are linked to increased bias. The authors propose easy-to-use criteria to establish whether pruning will increase bias and identify samples most susceptible to biased predictions.

CVPR 2023 - Practical Network Acceleration with Tiny Sets

In this episode we discuss Practical Network Acceleration with Tiny Sets
by Guo-Hua Wang, Jianxin Wu. The paper proposes a new method called PRACTISE for accelerating networks using only small training sets. It suggests dropping blocks as a better approach than filter-level pruning for achieving higher acceleration ratio and improved latency-accuracy performance under few-shot settings. The paper introduces a new concept called "recoverability" to measure the difficulty of recovering the compressed network and proposes an algorithm using it to select which blocks to drop. PRACTISE outperforms previous methods by a significant margin and also shows high generalization ability under data-free or out-of-domain data settings.

CVPR 2023 - CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP

In this episode we discuss CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP
by Runnan Chen, Youquan Liu, Lingdong Kong, Xinge Zhu, Yuexin Ma, Yikang Li, Yuenan Hou, Yu Qiao, Wenping Wang. The paper explores how Contrastive Language-Image Pre-training (CLIP) knowledge can benefit 3D scene understanding, which has yet to be explored. The authors propose a framework called CLIP2Scene that can transfer CLIP knowledge from 2D image-text pre-trained models to a 3D point cloud network. Experiments conducted on SemanticKITTI, nuScenes, and ScanNet show that the pre-trained 3D network achieves impressive performance on various downstream tasks, including annotation-free and fine-tuning with labeled data for semantic segmentation, outperforming other self-supervised methods.

CVPR 2023 - Zero-Shot Noise2Noise: Efficient Image Denoising without any Data

In this episode we discuss Zero-Shot Noise2Noise: Efficient Image Denoising without any Data
by Youssef Mansour, Reinhard Heckel. The paper proposes a new method for image denoising that does not rely on any training data or knowledge of the noise distribution and is computationally efficient. The proposed method utilizes a simple 2-layer network that can denoise pixel-wise independent noise and outperforms existing dataset-free methods at a reduced cost. The method is motivated by Noise2Noise and Neighbour2Neighbour and achieves a better trade-off between denoising quality, generalization, and computational resources.

CVPR 2023 - Single Image Backdoor Inversion via Robust Smoothed Classifiers

In this episode we discuss Single Image Backdoor Inversion via Robust Smoothed Classifiers
by Mingjie Sun, Zico Kolter. The paper proposes a new method called SmoothInv for identifying backdoor triggers in machine learning models. Previous methods used an optimization process to flip a support set of clean images into the target class. However, the paper demonstrates that SmoothInv can reliably recover the trigger with as few as one image, without requiring an explicit modeling of the trigger or complex regularization schemes. The proposed method is shown to be effective in identifying backdoors in existing models and remains robust against adaptive attackers.

CVPR 2023 - Fake it till you make it: Learning transferable representations from synthetic ImageNet clones

In this episode we discuss Fake it till you make it: Learning transferable representations from synthetic ImageNet clones
by Mert Bulent Sariyildiz, Karteek Alahari, Diane Larlus, Yannis Kalantidis. The paper investigates the ability of synthetic images, generated using Stable Diffusion, to replace real images for training models for ImageNet classification. Using only class names to build the dataset, the study explores the usefulness of synthetic clones of ImageNet for training classification models from scratch. The results show that models trained on synthetic images exhibit strong generalization properties and perform on par with models trained on real data for transfer.