AI Breakdown

CVPR 2023 - Learning Anchor Transformations for 3D Garment Animation

In this episode we discuss Learning Anchor Transformations for 3D Garment Animation
by Fang Zhao, Zekun Li, Shaoli Huang, Junwu Weng, Tianfei Zhou, Guo-Sen Xie, Jue Wang, Ying Shan. The paper presents a new anchor-based deformation model called AnchorDEF, which predicts 3D garment animation from a body motion sequence. The model deforms a garment mesh template using a mixture of rigid transformations and extra nonlinear displacements, guided by a set of anchors around the mesh surface. The transformed anchors are constrained to satisfy position, normal, and direction consistencies, ensuring better generalization. The model achieves state-of-the-art performance on 3D garment deformation prediction, especially for loose-fitting garments.

CVPR 2023 - OrienterNet: Visual Localization in 2D Public Maps with Neural Matching

In this episode we discuss OrienterNet: Visual Localization in 2D Public Maps with Neural Matching
by Paul-Edouard Sarlin, Daniel DeTone, Tsun-Yi Yang, Armen Avetisyan, Julian Straub, Tomasz Malisiewicz, Samuel Rota Bulo, Richard Newcombe, Peter Kontschieder, Vasileios Balntas. The paper introduces OrienterNet, a deep neural network that can localize an image with sub-meter accuracy using 2D semantic maps, enabling anyone to localize anywhere such maps are available. OrienterNet estimates the location and orientation of a query image by matching a neural Bird's-Eye View with open and globally available maps from OpenStreetMap. The network is supervised only by camera poses but learns to perform semantic matching with a wide range of map elements in an end-to-end manner. The paper also introduces a large crowd-sourced dataset of images captured across 12 cities from the viewpoints of cars, bikes, and pedestrians to enable the network's training.

CVPR 2023 - NAR-Former: Neural Architecture Representation Learning towards Holistic Attributes Prediction

In this episode we discuss NAR-Former: Neural Architecture Representation Learning towards Holistic Attributes Prediction
by Yun Yi, Haokui Zhang, Wenze Hu, Nannan Wang, Xiaoyu Wang. The paper proposes a neural architecture representation model that can be used to estimate attributes of different neural network architectures such as accuracy and latency without running actual training or inference tasks. The proposed model first uses a simple and effective tokenizer to encode operation and topology information into a single sequence, then uses a multi-stage fusion transformer to build a compact vector representation. An information flow consistency augmentation is proposed for efficient model training, which achieves promising results in predicting both cell architectures and whole deep neural networks. Code is available on Github.

CVPR 2023 - Boundary Unlearning

In this episode we discuss Boundary Unlearning
by Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, Chen Wang. The paper proposes "Boundary Unlearning" as an efficient machine unlearning technique to enable deep neural networks (DNNs) to unlearn, or forget, a fraction of training data and its lineage. The proposed method focuses on the decision space of the model rather than the parameter space, and involves shifting the decision boundary of the original DNN model to imitate the decision behavior of the model retrained from scratch. The proposed technique is evaluated on image classification and face recognition tasks, with expected speed-up compared to retraining from scratch.

CVPR 2023 - FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentation

In this episode we discuss FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentation
by Jie Qin, Jie Wu, Pengxiang Yan, Ming Li, Ren Yuxi, Xuefeng Xiao, Yitong Wang, Rui Wang, Shilei Wen, Xin Pan, Xingang Wang. The paper proposes FreeSeg, a generic framework for unified, universal, and open-vocabulary image segmentation. Existing methods use specialized architectures or parameters to tackle specific segmentation tasks, leading to fragmentation and hindered uniformity. FreeSeg optimizes an all-in-one network through one-shot training and uses the same architecture and parameters for diverse segmentation tasks. Adaptive prompt learning improves model robustness in multi-task scenarios, and experimental results show that FreeSeg outperforms task-specific architectures by a large margin. The project page is https://FreeSeg.github.io.

CVPR 2023 - Equiangular Basis Vectors

In this episode we discuss Equiangular Basis Vectors
by Yang Shen, Xuhao Sun, Xiu-Shen Wei. This paper proposes a new approach for classification tasks, called Equiangular Basis Vectors (EBVs), which generate normalized vector embeddings as "predefined classifiers". These vectors are required to be equal in status and as orthogonal as possible. By minimizing the spherical distance between the embedding of an input and its categorical EBV during training, predictions are made by identifying the EBV with the smallest distance during inference. The method outperforms fully connected classifiers on the ImageNet-1K dataset and other tasks, and does not significantly increase computation compared to classical metric learning methods.

CVPR 2023 - Learning to Retain while Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation

In this episode we discuss Learning to Retain while Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation
by Gaurav Patel, Konda Reddy Mopuri, Qiang Qiu. The paper introduces a framework called Learning to Retain while Acquiring, which addresses the issue of non-stationary distribution of pseudo-samples in the Adversarial Data-free Knowledge Distillation (DFKD) framework. The proposed method treats the tasks of learning from newly generated samples and retaining knowledge on previously met samples as meta-train and meta-test, respectively. The authors also identify an implicit aligning factor between the two tasks, showing that the student update strategy enforces a common gradient direction for both objectives. The effectiveness of the proposed method is demonstrated through extensive evaluation and comparison on multiple datasets.

CVPR 2023 - Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation

In this episode we discuss Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation
by Mingjie Li, Bingqian Lin, Zicong Chen, Haokun Lin, Xiaodan Liang, Xiaojun Chang. The paper proposes a knowledge graph with dynamic structure and nodes to enhance automatic radiology reporting. Existing models that use medical knowledge graphs have limited effectiveness because they have fixed structures that don't update during training. The proposed model, named DCL, allows for the addition of specific knowledge extracted from retrieved reports in a bottom-up manner, integrating each image feature with an updated graph. The model also introduces image-report contrastive and image-report matching losses to better represent visual features and textual information. Evaluation on two datasets shows that DCL outperforms previous state-of-the-art models.

CVPR 2023 - NeuDA: Neural Deformable Anchor for High-Fidelity Implicit Surface Reconstruction

In this episode we discuss NeuDA: Neural Deformable Anchor for High-Fidelity Implicit Surface Reconstruction
by Bowen Cai, Jinchi Huang, Rongfei Jia, Chengfei Lv, Huan Fu. The paper proposes a new approach called Neural Deformable Anchor (NeuDA) for implicit surface reconstruction using differentiable ray casting. Unlike previous methods, NeuDA leverages hierarchical voxel grids to capture sharp local topologies and maintain anchor grids where each vertex stores a 3D position instead of direct embedding. The paper also introduces a hierarchical positional encoding method for the anchor structure to exploit the properties of high-frequency and low-frequency geometry and appearance. Experiments on two datasets demonstrate NeuDA's ability to produce promising mesh surfaces.

CVPR 2023 - NeuFace: Realistic 3D Neural Face Rendering from Multi-view Images

In this episode we discuss NeuFace: Realistic 3D Neural Face Rendering from Multi-view Images
by Mingwu Zheng, Haiyu Zhang, Hongyu Yang, Di Huang. The paper presents a new 3D face rendering model, called Neu-Face, that uses neural rendering techniques to learn accurate and physically-meaningful underlying 3D representations. It incorporates the neural BRDFs (bidirectional reflectance distribution function) into physically based rendering to capture sophisticated facial geometry and appearance clues. The model uses approximated BRDF integration and a new low-rank prior to effectively lower ambiguities and boost performance. The experiments show the superiority of Neu-Face in human face rendering, with decent generalization ability to common objects. The code is available at NeuFace.