AI Breakdown

ICLR 2023 - On the duality between contrastive and non-contrastive self-supervised learning

In this episode we discuss On the duality between contrastive and non-contrastive self-supervised learning
by Quentin Garrido, Yubei Chen, Adrien Bardes, Laurent Najman, Yann Lecun. This paper discusses the duality between contrastive and non-contrastive self-supervised learning methods for image representations. It highlights the theoretical similarities between these approaches and introduces algebraically related contrastive and covariance-based non-contrastive criteria. The authors demonstrate through analysis and experiments that the performance gaps between the two methods can be closed with improved network design and hyperparameter tuning, challenging the assumption that non-contrastive methods require large output dimensions.

arxiv Preprint - LISA: Reasoning Segmentation via Large Language Model

In this episode we discuss LISA: Reasoning Segmentation via Large Language Model
by Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, Jiaya Jia. The paper introduces a new segmentation task called reasoning segmentation and presents a benchmark dataset for evaluating models. They propose LISA, a model that combines language generation with the ability to produce segmentation masks. LISA demonstrates robust zero-shot capability and performs well on both reasoning segmentation and referring segmentation tasks.

arxiv Preprint - Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP

In this episode we discuss Convolutions Die Hard: Open-Vocabulary Segmentation with Single Frozen Convolutional CLIP
by Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, Liang-Chieh Chen. The paper proposes a single-stage framework for open-vocabulary segmentation using a shared Frozen Convolutional CLIP (FC-CLIP) backbone. FC-CLIP simplifies the pipeline and achieves a better accuracy-cost trade-off compared to existing two-stage approaches. It outperforms previous methods on various benchmarks, sets a new state-of-the-art performance on open-vocabulary semantic segmentation datasets, and is significantly faster and uses fewer parameters.

ICCV 2023 - PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalization

In this episode we discuss PromptStyler: Prompt-driven Style Generation for Source-free Domain Generalization
by Junhyeong Cho, Gilhyun Nam, Sungyeon Kim, Hunmin Yang, Suha Kwak. The paper introduces a method called PromptStyler for domain generalization in a joint vision-language space. It achieves this by synthesizing diverse styles using prompts without using any images. The method learns to generate different style features using learnable style word vectors and ensures that content information is preserved by keeping style-content features close to their corresponding content features. The results show that PromptStyler outperforms existing methods on multiple benchmark datasets while requiring no images and only a short training time.

arxiv Preprint - Extrapolating Large Language Models to Non-English by Aligning Languages

In this episode we discuss Extrapolating Large Language Models to Non-English by Aligning Languages
by Wenhao Zhu, Yunzhe Lv, Qingxiu Dong, Fei Yuan, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, Lei Li. The paper proposes a method to improve the language abilities of large language models (LLMs) in non-English languages. They achieve this by creating semantic alignment between English and non-English languages. The authors demonstrate through experiments that the cross-lingual models outperform their English counterparts by a significant margin, particularly in Chinese humanities tasks. They also find that incorporating non-English text in the translation task data is highly effective in enhancing non-English ability.

ICLR 2023 - Universal Few-shot Learning of Dense Prediction Tasks with Visual Token Matching

In this episode we discuss Universal Few-shot Learning of Dense Prediction Tasks with Visual Token Matching
by Donggyun Kim, Jinwoo Kim, Seongwoong Cho, Chong Luo, Seunghoon Hong. The paper proposes Visual Token Matching (VTM), a few-shot learning solution for arbitrary dense prediction tasks in computer vision. VTM uses non-parametric matching on patch-level embedded tokens of images and labels to handle different tasks. It incorporates a hierarchical encoder-decoder architecture with ViT backbones and performs token matching at multiple feature hierarchies. Experimental results demonstrate that VTM successfully learns various unseen dense prediction tasks, surpassing fully supervised baselines with just 10 labeled examples.

ICML 2023 - Generalization on the Unseen, Logic Reasoning and Degree Curriculum

In this episode we discuss Generalization on the Unseen, Logic Reasoning and Degree Curriculum
by Emmanuel Abbe, Samy Bengio, Aryo Lotfi, Kevin Rizk. This paper examines the performance of different network architectures trained by stochastic gradient descent (SGD) in the generalization on the unseen (GOTU) setting. The authors find that certain network models, such as Transformers, random features models, and diagonal linear networks, can learn a min-degree-interpolator on unseen data. They also introduce a curriculum learning algorithm called Degree-Curriculum to address the challenges of learning in combinatorial reasoning tasks.

arxiv Preprint - Gorilla: Large Language Model Connected with Massive APIs

In this episode we discuss Gorilla: Large Language Model Connected with Massive APIs
by Shishir G. Patil, Tianjun Zhang, Xin Wang, Joseph E. Gonzalez. The paper introduces Gorilla, a fine-tuned Large Language Model (LLM) that excels in generating accurate API calls. By combining Gorilla with a document retriever, the model exhibits the ability to adapt to changes in test-time documents, addressing the issue of hallucination commonly observed in LLMs. The authors introduce APIBench, a dataset containing HuggingFace, TorchHub, and TensorHub APIs, to evaluate Gorilla's performance and demonstrate the potential for LLMs to utilize tools more accurately and enhance the reliability of their outputs.

ICML 2023 - Learning-Rate-Free Learning by D-Adaptation

In this episode we discuss Learning-Rate-Free Learning by D-Adaptation
by Aaron Defazio, Konstantin Mishchenko. The paper introduces D-Adaptation, a learning-rate-free approach for setting the learning rate in convex minimization problems. It achieves the optimal rate of convergence without additional evaluations per step. The method is shown to match hand-tuned learning rates in diverse machine learning problems.

arxiv Preprint - Shepherd: A Critic for Language Model Generation

In this episode we discuss Shepherd: A Critic for Language Model Generation
by Tianlu Wang, Ping Yu, Xiaoqing Ellen Tan, Sean O'Brien, Ramakanth Pasunuru, Jane Dwivedi-Yu, Olga Golovneva, Luke Zettlemoyer, Maryam Fazel-Zarandi, Asli Celikyilmaz. The paper introduces Shepherd, a language model trained to critique responses generated by large language models (LLMs) and offer suggestions for improvement. Despite its smaller size, Shepherd's critiques are on par or preferred over established models like ChatGPT. Evaluation results demonstrate Shepherd's superior performance and highlight its potential in enhancing the reliability and coherence of LLM outputs.