AI Breakdown

CVPR 2023 - 3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification

In this episode we discuss 3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification
by Jiazhao Zhang, Liu Dai, Fanpeng Meng, Qingnan Fan, Xuelin Chen, Kai Xu, He Wang. The paper proposes a framework for object goal navigation in 3D environments using two sub-policies - corner-guided exploration policy and category-aware identification policy. Unlike other approaches that use 2D maps, scene graphs, or image sequences, this framework leverages fine-grained spatial information to improve ObjectNav capability. Through extensive experiments, the proposed framework outperforms other modular-based methods on Matterport3D and Gibson datasets while requiring significantly less computational cost for training. The code for the framework will be released to the community.

CVPR 2023 - GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learning

In this episode we discuss GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learning
by Zhenyu Xie, Zaiyu Huang, Xin Dong, Fuwei Zhao, Haoye Dong, Xijin Zhang, Feida Zhu, Xiaodan Liang. The paper proposes a General-Purpose Virtual Try-ON framework, named GP-VTON, for transferring a garment onto a specific person. The proposed framework addresses the limitations of existing methods which fail to preserve semantic information of the garment parts, result in texture distortion and limit the scalability of the system. It introduces a Local-Flow Global-Parsing (LFGP) warping module and a Dynamic Gradient Truncation (DGT) training strategy, resulting in better warping of different garment parts and avoiding texture squeezing. The proposed framework outperforms existing state-of-the-art methods on two high-resolution benchmarks.

CVPR 2023 - StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning

In this episode we discuss StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning
by Yuqian Fu, Yu Xie, Yanwei Fu, Yu-Gang Jiang. The paper proposes a novel model-agnostic meta Style Adversarial training (StyleAdv) method for Cross-Domain Few-Shot Learning (CD-FSL), a task that aims to transfer prior knowledge learned on a source dataset to novel target datasets. This is achieved by using a style adversarial attack method that synthesizes "virtual" and "hard" adversarial styles for model training, gradually making the model robust to visual styles and boosting its generalization ability. The proposed method achieves state-of-the-art results on eight various target datasets, whether built upon ResNet or ViT. Code is available on GitHub.

CVPR 2023 - Learning Anchor Transformations for 3D Garment Animation

In this episode we discuss Learning Anchor Transformations for 3D Garment Animation
by Fang Zhao, Zekun Li, Shaoli Huang, Junwu Weng, Tianfei Zhou, Guo-Sen Xie, Jue Wang, Ying Shan. The paper presents a new anchor-based deformation model called AnchorDEF, which predicts 3D garment animation from a body motion sequence. The model deforms a garment mesh template using a mixture of rigid transformations and extra nonlinear displacements, guided by a set of anchors around the mesh surface. The transformed anchors are constrained to satisfy position, normal, and direction consistencies, ensuring better generalization. The model achieves state-of-the-art performance on 3D garment deformation prediction, especially for loose-fitting garments.

CVPR 2023 - OrienterNet: Visual Localization in 2D Public Maps with Neural Matching

In this episode we discuss OrienterNet: Visual Localization in 2D Public Maps with Neural Matching
by Paul-Edouard Sarlin, Daniel DeTone, Tsun-Yi Yang, Armen Avetisyan, Julian Straub, Tomasz Malisiewicz, Samuel Rota Bulo, Richard Newcombe, Peter Kontschieder, Vasileios Balntas. The paper introduces OrienterNet, a deep neural network that can localize an image with sub-meter accuracy using 2D semantic maps, enabling anyone to localize anywhere such maps are available. OrienterNet estimates the location and orientation of a query image by matching a neural Bird's-Eye View with open and globally available maps from OpenStreetMap. The network is supervised only by camera poses but learns to perform semantic matching with a wide range of map elements in an end-to-end manner. The paper also introduces a large crowd-sourced dataset of images captured across 12 cities from the viewpoints of cars, bikes, and pedestrians to enable the network's training.

CVPR 2023 - NAR-Former: Neural Architecture Representation Learning towards Holistic Attributes Prediction

In this episode we discuss NAR-Former: Neural Architecture Representation Learning towards Holistic Attributes Prediction
by Yun Yi, Haokui Zhang, Wenze Hu, Nannan Wang, Xiaoyu Wang. The paper proposes a neural architecture representation model that can be used to estimate attributes of different neural network architectures such as accuracy and latency without running actual training or inference tasks. The proposed model first uses a simple and effective tokenizer to encode operation and topology information into a single sequence, then uses a multi-stage fusion transformer to build a compact vector representation. An information flow consistency augmentation is proposed for efficient model training, which achieves promising results in predicting both cell architectures and whole deep neural networks. Code is available on Github.

CVPR 2023 - Boundary Unlearning

In this episode we discuss Boundary Unlearning
by Min Chen, Weizhuo Gao, Gaoyang Liu, Kai Peng, Chen Wang. The paper proposes "Boundary Unlearning" as an efficient machine unlearning technique to enable deep neural networks (DNNs) to unlearn, or forget, a fraction of training data and its lineage. The proposed method focuses on the decision space of the model rather than the parameter space, and involves shifting the decision boundary of the original DNN model to imitate the decision behavior of the model retrained from scratch. The proposed technique is evaluated on image classification and face recognition tasks, with expected speed-up compared to retraining from scratch.

CVPR 2023 - FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentation

In this episode we discuss FreeSeg: Unified, Universal and Open-Vocabulary Image Segmentation
by Jie Qin, Jie Wu, Pengxiang Yan, Ming Li, Ren Yuxi, Xuefeng Xiao, Yitong Wang, Rui Wang, Shilei Wen, Xin Pan, Xingang Wang. The paper proposes FreeSeg, a generic framework for unified, universal, and open-vocabulary image segmentation. Existing methods use specialized architectures or parameters to tackle specific segmentation tasks, leading to fragmentation and hindered uniformity. FreeSeg optimizes an all-in-one network through one-shot training and uses the same architecture and parameters for diverse segmentation tasks. Adaptive prompt learning improves model robustness in multi-task scenarios, and experimental results show that FreeSeg outperforms task-specific architectures by a large margin. The project page is https://FreeSeg.github.io.

CVPR 2023 - Equiangular Basis Vectors

In this episode we discuss Equiangular Basis Vectors
by Yang Shen, Xuhao Sun, Xiu-Shen Wei. This paper proposes a new approach for classification tasks, called Equiangular Basis Vectors (EBVs), which generate normalized vector embeddings as "predefined classifiers". These vectors are required to be equal in status and as orthogonal as possible. By minimizing the spherical distance between the embedding of an input and its categorical EBV during training, predictions are made by identifying the EBV with the smallest distance during inference. The method outperforms fully connected classifiers on the ImageNet-1K dataset and other tasks, and does not significantly increase computation compared to classical metric learning methods.

CVPR 2023 - Learning to Retain while Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation

In this episode we discuss Learning to Retain while Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation
by Gaurav Patel, Konda Reddy Mopuri, Qiang Qiu. The paper introduces a framework called Learning to Retain while Acquiring, which addresses the issue of non-stationary distribution of pseudo-samples in the Adversarial Data-free Knowledge Distillation (DFKD) framework. The proposed method treats the tasks of learning from newly generated samples and retaining knowledge on previously met samples as meta-train and meta-test, respectively. The authors also identify an implicit aligning factor between the two tasks, showing that the student update strategy enforces a common gradient direction for both objectives. The effectiveness of the proposed method is demonstrated through extensive evaluation and comparison on multiple datasets.