AI Breakdown

ArXiv Preprint - Learning From Mistakes Makes LLM Better Reasoner

In this episode we discuss Learning From Mistakes Makes LLM Better Reasoner
by Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, Weizhu Chen. The paper introduces LEarning from MistAkes (LEMA), a method that improves large language models' (LLMs) ability to solve math problems by fine-tuning them using GPT-4-generated mistake-correction data pairs. LEMA involves identifying an LLM's errors in reasoning, explaining why the mistake occurred, and providing the correct solution. LEMA showed significant performance enhancements on mathematical reasoning tasks, surpassing state-of-the-art performances of open-source models, with the intention to release the code, data, and models publicly.

ArXiv Preprint - The Generative AI Paradox: ”What It Can Create, It May Not Understand”

In this episode we discuss The Generative AI Paradox: "What It Can Create, It May Not Understand"
by Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian Fisher, Abhilasha Ravichander, Khyathi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi. The paper examines the paradox in generative AI models where they excel in output generation but struggle with comprehension. The authors propose the Generative AI Paradox hypothesis, stating that the models acquire superior generative abilities without corresponding understanding abilities. They compare the performance of humans and models in language and image tasks and find that while models outperform humans in generation, they consistently lag behind in understanding, cautioning against comparing AI to human intelligence.

ArXiv Preprint - TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise

In this episode we discuss TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise
by Nan He, Hanyu Lai, Chenyang Zhao, Zirui Cheng, Junting Pan, Ruoyu Qin, Ruofan Lu, Rui Lu, Yunchen Zhang, Gangming Zhao, Zhaohui Hou, Zhiyuan Huang, Shaoqing Lu, Ding Liang, Mingjie Zhan. The paper introduces TeacherLM, a series of language models designed to teach other models. The TeacherLM-7.1B model achieved a high score on MMLU and outperformed models with more parameters. It also has data augmentation abilities and has been used to teach multiple student models.

ArXiv Preprint - MM-VID: Advancing Video Understanding with GPT-4V(ision)

In this episode we discuss MM-VID: Advancing Video Understanding with GPT-4V(ision)
by Kevin Lin, Faisal Ahmed, Linjie Li, Chung-Ching Lin, Ehsan Azarnasab, Zhengyuan Yang, Jianfeng Wang, Lin Liang, Zicheng Liu, Yumao Lu, Ce Liu, Lijuan Wang. The paper introduces MM-VID, a system that incorporates GPT-4V with vision, audio, and speech experts to enhance video understanding. It focuses on handling complex tasks like tracking character storylines across multiple episodes. The paper showcases the capabilities of MM-VID through detailed responses and demonstrations in various figures.

ArXiv Preprint - Zephyr: Direct Distillation of LM Alignment

In this episode we discuss Zephyr: Direct Distillation of LM Alignment
by Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, Thomas Wolf. The paper introduces ZEPHYR, a language model that focuses on aligning with user intent to improve task accuracy. The authors employ distilled supervised fine-tuning (dSFT) on larger models but note the lack of alignment with natural prompts. To address this, the authors experiment with preference data from AI Feedback (AIF) and use distilled direct preference optimization (dDPO) to enhance intent alignment. Their approach, requiring only a few hours of training, achieves state-of-the-art performance on chat benchmarks without human annotation, surpassing the performance of the best RLHF-based model LLAMA2-CHAT-70B on MT-Bench.

ArXiv Preprint - ControlLLM: Augment Language Models with Tools by Searching on Graphs

In this episode we discuss ControlLLM: Augment Language Models with Tools by Searching on Graphs
by Zhaoyang Liu, Zeqiang Lai, Zhangwei Gao, Erfei Cui, Xizhou Zhu, Lewei Lu, Qifeng Chen, Yu Qiao, Jifeng Dai, Wenhai Wang. The paper introduces a framework called ControlLLM that enhances large language models (LLMs) by allowing them to use multi-modal tools for complex tasks. ControlLLM addresses challenges such as ambiguous prompts, inaccurate tool selection, parameterization, and inefficient tool scheduling. It consists of three components: a task decomposer, a Thoughts-on-Graph paradigm, and an execution engine. The framework is evaluated on tasks involving image, audio, and video processing, and it outperforms existing methods in terms of accuracy, efficiency, and versatility.

ArXiv Preprint - Talk like a Graph: Encoding Graphs for Large Language Models

In this episode we discuss Talk like a Graph: Encoding Graphs for Large Language Models
by Bahare Fatemi, Jonathan Halcrow, Bryan Perozzi. The paper discusses the encoding of graph-structured data for use in large language models (LLMs). It investigates different graph encoding methods, the nature of graph tasks, and the structure of the graph, and their impact on LLM performance in graph reasoning tasks. The study highlights the importance of choosing appropriate graph encoding methods and prompts to enhance LLM performance.

arxiv Preprint - AgentTuning: Enabling Generalized Agent Abilities for LLMs

In this episode we discuss AgentTuning: Enabling Generalized Agent Abilities for LLMs
by Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, Jie Tang. AgentTuning is a method that enhances the agent abilities of large language models (LLMs) while maintaining their general capabilities. It introduces AgentInstruct, a lightweight instruction-tuning dataset, and combines it with open-source instructions from general domains. The resulting model, AgentLM, demonstrates generalized agent capabilities comparable to commercial LLMs.

ArXiv Preprint - Jailbreaking Black Box Large Language Models in Twenty Queries

In this episode we discuss Jailbreaking Black Box Large Language Models in Twenty Queries
by Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, Eric Wong. The paper introduces an algorithm called Prompt Automatic Iterative Refinement (PAIR) that generates "jailbreaks" for large language models (LLMs) using only black-box access. PAIR leverages an attacker LLM to automatically create vulnerabilities for a targeted LLM without human intervention. The algorithm requires fewer than twenty queries to create a jailbreak and achieves competitive success rates on different LLMs, including GPT-3.5/4. The research aims to identify weaknesses in LLMs to enhance their safety and prevent potential misuse.

ArXiv Preprint - Matryoshka Diffusion Models

In this episode we discuss Matryoshka Diffusion Models
by Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Josh Susskind, Navdeep Jaitly. The paper introduces Matryoshka Diffusion Models (MDM) for high-resolution image and video synthesis. The authors propose a diffusion process that denoises inputs at multiple resolutions simultaneously. They also present a NestedUNet architecture that combines features and parameters for small-scale inputs with larger scales, allowing for improved optimization for high-resolution generation. The approach is demonstrated to be effective on various benchmarks, achieving strong zero-shot generalization using a dataset of only 12 million images.