AI Breakdown

ICCV 2023 - UnLoc: A Unified Framework for Video Localization Tasks

In this episode we discuss UnLoc: A Unified Framework for Video Localization Tasks
by Shen Yan, Xuehan Xiong, Arsha Nagrani, Anurag Arnab, Zhonghao Wang, Weina Ge, David Ross, Cordelia Schmid. The paper introduces UnLoc, a unified framework for video localization using large-scale image-text pretrained models. UnLoc eliminates the need for action proposals, motion-based features, and representation masking by combining moment retrieval, temporal localization, and action segmentation in a single stage model. Experimental results show that UnLoc outperforms previous methods and achieves state-of-the-art results in all three localization tasks.

arxiv Preprint - Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies

In this episode we discuss Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies
by Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, William Yang Wang. The paper provides a comprehensive review of self-correction strategies for large language models (LLMs). It examines recent work on self-correction techniques, categorizing them into training-time, generation-time, and post-hoc correction methods. The authors also discuss the applications of self-correction and highlight future directions and challenges in this area.

ICLR 2023 - Rethinking the Expressive Power of GNNs via Graph Biconnectivity

In this episode we discuss Rethinking the Expressive Power of GNNs via Graph Biconnectivity
by Bohang Zhang, Shengjie Luo, Liwei Wang, Di He. This paper introduces a new approach called Generalized Distance Weisfeiler-Lehman (GD-WL) to study the expressive power of Graph Neural Networks (GNNs). The authors show that most existing GNN architectures are not expressive for certain metrics related to graph biconnectivity, except for the ESAN framework. They demonstrate that GD-WL is provably expressive for all biconnectivity metrics and outperforms previous GNN architectures in practical experiments.

ICLR 2023 - Conditional Antibody Design as 3D Equivariant Graph Translation

In this episode we discuss Conditional Antibody Design as 3D Equivariant Graph Translation
by Xiangzhe Kong, Wenbing Huang, Yang Liu. The paper introduces a method called Multi-channel Equivariant Attention Network (MEAN) for antibody design. MEAN addresses challenges faced by existing deep-learning-based methods by formulating antibody design as a conditional graph translation problem and incorporating additional components. The MEAN model utilizes a proposed attention mechanism and generates both 1D CDR sequences and 3D structures, outperforming state-of-the-art models in sequence and structure modeling, antigen-binding CDR design, and binding affinity optimization.

arxiv Preprint - ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation

In this episode we discuss ReCLIP: Refine Contrastive Language Image Pre-Training with Source Free Domain Adaptation
by Xuefeng Hu, Ke Zhang, Lu Xia, Albert Chen, Jiajia Luo, Yuyin Sun, Ken Wang, Nan Qiao, Xiao Zeng, Min Sun, Cheng-Hao Kuo, Ram Nevatia. The paper discusses ReCLIP, a source-free domain adaptation method for large-scale pre-training vision-language models like CLIP. ReCLIP addresses the challenges of domain gaps and misalignment by learning a projection space and utilizing cross-modality self-training with pseudo labels. Experimental results show that ReCLIP reduces the average error rate of CLIP across 22 image classification benchmarks.

arxiv Preprint - LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition

In this episode we discuss LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
by Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, Min Lin. The paper presents LoraHub, a framework for combining Low-rank adaptations (LoRA) to improve cross-task generalization in fine-tuning large language models (LLMs). LoraHub allows the assembly of LoRA modules trained on different tasks, enabling adaptable performance on unseen tasks with just a few examples. Experimental results demonstrate that LoraHub achieves similar performance to in-context learning in few-shot scenarios without the need for in-context examples for each inference input. Additionally, the paper highlights the importance of creating a community for sharing trained LoRA modules to advance general intelligence and LLMs in production.

ICLR 2023 - Emergence of Maps in the Memories of Blind Navigation Agents

In this episode we discuss Emergence of Maps in the Memories of Blind Navigation Agents
by Erik Wijmans, Manolis Savva, Irfan Essa, Stefan Lee, Ari S. Morcos, Dhruv Batra. The paper explores whether blind artificial intelligence agents can develop implicit maps of their environment. The study involves training these agents in navigation tasks and finding that they display intelligent behavior and utilize memory over long time periods. The representations created by the blind agents include maps and collision detection neurons, supporting the idea that mapping is a vital mechanism for navigation in both biological and artificial agents.

ICLR 2023 - On the duality between contrastive and non-contrastive self-supervised learning

In this episode we discuss On the duality between contrastive and non-contrastive self-supervised learning
by Quentin Garrido, Yubei Chen, Adrien Bardes, Laurent Najman, Yann Lecun. This paper discusses the duality between contrastive and non-contrastive self-supervised learning methods for image representations. It highlights the theoretical similarities between these approaches and introduces algebraically related contrastive and covariance-based non-contrastive criteria. The authors demonstrate through analysis and experiments that the performance gaps between the two methods can be closed with improved network design and hyperparameter tuning, challenging the assumption that non-contrastive methods require large output dimensions.

arxiv Preprint - LISA: Reasoning Segmentation via Large Language Model

In this episode we discuss LISA: Reasoning Segmentation via Large Language Model
by Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, Jiaya Jia. The paper introduces a new segmentation task called reasoning segmentation and presents a benchmark dataset for evaluating models. They propose LISA, a model that combines language generation with the ability to produce segmentation masks. LISA demonstrates robust zero-shot capability and performs well on both reasoning segmentation and referring segmentation tasks.

arxiv Preprint - Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP

In this episode we discuss Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP
by Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, Liang-Chieh Chen. The paper proposes a single-stage framework for open-vocabulary segmentation using a shared Frozen Convolutional CLIP (FC-CLIP) backbone. FC-CLIP simplifies the pipeline and achieves a better accuracy-cost trade-off compared to existing two-stage approaches. It outperforms previous methods on various benchmarks, sets a new state-of-the-art performance on open-vocabulary semantic segmentation datasets, and is significantly faster and uses fewer parameters.