AI Breakdown

arxiv Preprint - Language Modeling Is Compression

In this episode we discuss Language Modeling Is Compression
by Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, Marcus Hutter, Joel Veness. The authors argue that large language models can be seen as powerful compressors due to their predictive capabilities. They demonstrate that these models outperform specific compressors like PNG and FLAC. The paper explores the implications of the prediction-compression equivalence and discusses the use of any compressor to build a conditional generative model.

arxiv Preprint - From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting

In this episode we discuss From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting
by Griffin Adams, Alexander Fabbri, Faisal Ladhak, Eric Lehman, Noémie Elhadad. The paper introduces a method called "Chain of Density" (CoD) for generating summaries with varying levels of information density. Using GPT-4, the authors generate entity-sparse summaries and then iteratively add missing salient entities without increasing the length. CoD summaries are found to be more abstractive, exhibit more fusion, and have less lead bias compared to GPT-4 summaries generated by a vanilla prompt, with human preference favoring denser GPT-4 summaries.

ICCV 2023 - Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing

In this episode we discuss Multimodal Garment Designer: Human-Centric Latent Diffusion Models for Fashion Image Editing
by Alberto Baldrati, Davide Morelli, Giuseppe Cartella, Marcella Cornia, Marco Bertini, Rita Cucchiara. This paper introduces a framework for generating fashion images using multimodal prompts such as text, body poses, and garment sketches. The proposed architecture utilizes latent diffusion models, a novel approach in the fashion domain. The effectiveness of the framework is demonstrated through experiments using extended fashion datasets, showing realistic and coherent results.

arxiv Preprint - GPT Can Solve Mathematical Problems Without a Calculator

In this episode we discuss GPT Can Solve Mathematical Problems Without a Calculator
by Zhen Yang, Ming Ding, Qingsong Lv, Zhihuan Jiang, Zehai He, Yuyi Guo, Jinfeng Bai, Jie Tang. The paper challenges the belief that large language models cannot perform arithmetic operations accurately without calculator tools. The researchers present MathGLM, a 2 billion-parameter language model, which achieves nearly 100% accuracy in multi-digit arithmetic operations, surpassing GPT-4. They demonstrate the model's capability by training it on a dataset containing multi-step arithmetic operations and math problems described in text, and it performs similarly to GPT-4 on a Chinese math problem test set. The results suggest that language models can excel in mathematical problem-solving without the need for calculators, given sufficient training data.

ICCV 2023 - Adding Conditional Control to Text-to-Image Diffusion Models

In this episode we discuss Adding Conditional Control to Text-to-Image Diffusion Models
by Lvmin Zhang, Anyi Rao, Maneesh Agrawala. The paper introduces ControlNet, a neural network architecture that enhances spatial control in text-to-image diffusion models. It incorporates additional images, such as edge maps and human pose skeletons, as conditioning factors to specify desired image composition. ControlNet utilizes pretrained encoding layers and gradually adjusts parameters for improved spatial control. Experimental results with different conditioning controls and datasets demonstrate the effectiveness of ControlNet, suggesting its potential to broaden the applications of image diffusion models.

arxiv Preprint - Neurons in Large Language Models: Dead, N-gram, Positional

In this episode we discuss Neurons in Large Language Models: Dead, N-gram, Positional
by Elena Voita, Javier Ferrando, Christoforos Nalmpantis. In this paper, the authors analyze a family of language models called OPT models and focus on the activation of neurons in the feedforward blocks. They find that there are many inactive "dead" neurons in the early part of the network and that active neurons in this region primarily act as token and n-gram detectors. The authors also identify positional neurons that are activated based on position rather than textual data.

arxiv Preprint - eDKM: An Efficient and Accurate Train-time Weight Clustering for Large Language Models

In this episode we discuss eDKM: An Efficient and Accurate Train-time Weight Clustering for Large Language Models
by Minsik Cho, Keivan A. Vahid, Qichen Fu, Saurabh Adya, Carlo C Del Mundo, Mohammad Rastegari, Devang Naik, Peter Zatloukal. The paper proposes eDKM, a memory-efficient implementation of weight clustering for large language models (LLMs). LLMs have high performance but require compression for storage-limited devices. The eDKM technique reduces the memory footprint of Differentiable KMeans Clustering (DKM) by orders of magnitudes, allowing for efficient LLM compression with good accuracy.

arxiv Preprint - Link-Context Learning for Multimodal LLMs

In this episode we discuss Link-Context Learning for Multimodal LLMs
by Yan Tai, Weichen Fan, Zhao Zhang, Feng Zhu, Rui Zhao, Ziwei Liu. The paper presents a method called link-context learning (LCL) that enhances the learning abilities of Multimodal Large Language Models (MLLMs). LCL aims to enable MLLMs to recognize new images and understand unfamiliar concepts without the need for training. It focuses on strengthening the causal relationship between the support set and the query set to help MLLMs discern analogies and causal associations between data points. Experimental results demonstrate that the proposed LCL-MLLM performs better in link-context learning compared to traditional MLLMs.

arxiv Preprint - ProPainter: Improving Propagation and Transformer for Video Inpainting

In this episode we discuss ProPainter: Improving Propagation and Transformer for Video Inpainting
by Shangchen Zhou, Chongyi Li, Kelvin C. K. Chan, Chen Change Loy. The paper discusses the limitations of existing approaches to video inpainting, specifically flow-based propagation and spatiotemporal Transformer methods, due to spatial misalignment and limited temporal range. To address these challenges, the authors propose ProPainter, a framework that combines dual-domain propagation with image and feature warping for reliable global correspondences. They also introduce a mask-guided sparse video Transformer to enhance efficiency. ProPainter achieves superior results with a 1.46 dB improvement in PSNR while maintaining efficiency, making it a valuable tool for video inpainting applications.

arxiv Preprint - Large Language Models as Optimizers

In this episode we discuss Large Language Models as Optimizers
by Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, Xinyun Chen. The paper introduces Optimization by PROmpting (OPRO), a method that uses large language models as optimizers in the absence of gradients. OPRO utilizes natural language descriptions of the optimization task to generate new solutions in each step, which are evaluated and added to the prompt for subsequent steps. Experimental results demonstrate that prompts optimized by OPRO outperform human-designed prompts on various tasks, with performance improvements of up to 8% on GSM8K and up to 50% on Big-Bench Hard tasks.