AI Breakdown

arxiv Preprint - LLM-Rec: Personalized Recommendation via Prompting Large Language Models

In this episode we discuss LLM-Rec: Personalized Recommendation via Prompting Large Language Models
by Hanjia Lyu, Song Jiang, Hanqing Zeng, Qifan Wang, Si Zhang, Ren Chen, Chris Leung, Jiajie Tang, Yinglong Xia, Jiebo Luo. The paper examines different prompting strategies for improving personalized recommendation performance using large language models (LLMs) through input augmentation. The proposed approach, LLM-Rec, incorporates four prompting strategies, demonstrating that using LLM-generated augmented input text enhances recommendation performance. The recommendation-driven and engagement-guided prompting strategies highlight the LLM's comprehension of global and local item characteristics, underscoring the necessity of diverse prompts and input augmentation techniques to enhance recommendation capabilities with LLMs.

ICCV 2023 - Robust Monocular Depth Estimation under Challenging Conditions

In this episode we discuss Robust Monocular Depth Estimation under Challenging Conditions
by Stefano Gasperini, Nils Morbitzer, HyunJun Jung, Nassir Navab, Federico Tombari. The paper addresses the limitations of existing monocular depth estimation methods in challenging lighting and weather conditions. The authors propose md4all, a simple and reliable solution that can handle diverse conditions without modification at inference time. The approach involves generating complex training samples, training the model using self- or full-supervision, and computing standard losses on the original images. Extensive experiments on public datasets demonstrate the effectiveness of the approach, surpassing previous works in both standard and challenging conditions.

arxiv Preprint - LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models

In this episode we discuss LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models
by Chi Han, Qifan Wang, Wenhan Xiong, Yu Chen, Heng Ji, Sinong Wang. The paper introduces LM-Infinite as a solution to the challenge of length generalization in Large Language Models (LLMs). Existing methods for handling longer sequences are resource-intensive and time-consuming, prompting the need for a simpler approach. LM-Infinite employs a Λ-shaped attention mask and a distance limit, requiring no parameter updates or learning. The technique consistently generates fluent texts and performs well on downstream tasks even with input sequences exceeding training lengths.

arxiv Preprint - Llama 2: Open Foundation and Fine-Tuned Chat Models

In this episode we discuss Llama 2: Open Foundation and Fine-Tuned Chat Models
by Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, Thomas Scialom. The paper introduces Llama 2, a collection of pretrained and fine-tuned large language models optimized for dialogue purposes. Ranging from 7 billion to 70 billion parameters, the Llama 2-Chat models surpass existing open-source chat models in performance across different benchmarks. The authors also conduct human evaluations, indicating that their models could be viable alternatives to closed-source models, and provide detailed insights into their fine-tuning process and safety enhancements.

arxiv Preprint - Nougat: Neural Optical Understanding for Academic Documents

In this episode we discuss Nougat: Neural Optical Understanding for Academic Documents
by Lukas Blecher, Guillem Cucurull, Thomas Scialom, Robert Stojnic. The paper introduces Nougat, a neural optical understanding model for academic documents. Nougat utilizes a Visual Transformer model and Optical Character Recognition (OCR) to convert scientific documents into a markup language, bridging the gap between human-readable and machine-readable text. The method is versatile, capable of processing scanned papers and books, and includes a pre-trained model and code on GitHub, as well as a pipeline for creating datasets.

arxiv Preprint - Graph of Thoughts: Solving Elaborate Problems with Large Language Models

In this episode we discuss Graph of Thoughts: Solving Elaborate Problems with Large Language Models
by Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Michal Podstawski, Hubert Niewiadomski, Piotr Nyczyk, Torsten Hoefler. The paper introduces a framework called Graph of Thoughts (GoT) that enhances the prompting capabilities of large language models (LLMs). GoT models the information generated by an LLM as an arbitrary graph, where LLM thoughts are vertices and edges represent dependencies between these thoughts. The paper demonstrates that GoT outperforms state-of-the-art methods on different tasks and can be used to spearhead new prompting schemes.

arxiv Preprint - Large Language Models as Zero-Shot Conversational Recommenders

In this episode we discuss Large Language Models as Zero-Shot Conversational Recommenders
by Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bodhisattwa Prasad Majumder, Nathan Kallus, Julian McAuley. This paper presents empirical studies on conversational recommendation tasks using large language models (LLMs) in a zero-shot setting, without fine-tuning. The authors introduce a new dataset of recommendation-related conversations, the largest public real-world conversational recommendation dataset to date, and find that LLMs outperform existing fine-tuned conversational recommendation models on this dataset and two others. The authors also propose probing tasks to investigate the mechanisms behind LLM performance and analyze both the models' behaviors and the characteristics of the datasets.

arxiv Preprint - A Survey on Large Language Model based Autonomous Agents

In this episode we discuss A Survey on Large Language Model based Autonomous Agents
by Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, Ji-Rong Wen. The authors of this paper conducted a comprehensive survey on the topic of autonomous agents based on large language models (LLMs). They propose a unified framework for constructing LLM-based agents and provide a systematic review of previous work in this area. Additionally, they discuss the applications, evaluation strategies, and future directions for LLM-based AI agents.

arxiv Preprint - EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding

In this episode we discuss EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding
by Karttikeya Mangalam, Raiymbek Akshulakov, Jitendra Malik. The paper presents EgoSchema, a benchmark dataset and evaluation metric for assessing the long-form video language understanding capabilities of vision and language systems. The dataset consists of over 5000 multiple choice question-answer pairs based on 250 hours of real video data, and the questions require selecting the correct answer from five options based on a three-minute video clip. The authors highlight that existing video understanding datasets lack long temporal structures, and they show that state-of-the-art video and language models have limitations in long-term video understanding.

ICCV 2023 - UnLoc: A Unified Framework for Video Localization Tasks

In this episode we discuss UnLoc: A Unified Framework for Video Localization Tasks
by Shen Yan, Xuehan Xiong, Arsha Nagrani, Anurag Arnab, Zhonghao Wang, Weina Ge, David Ross, Cordelia Schmid. The paper introduces UnLoc, a unified framework for video localization using large-scale image-text pretrained models. UnLoc eliminates the need for action proposals, motion-based features, and representation masking by combining moment retrieval, temporal localization, and action segmentation in a single stage model. Experimental results show that UnLoc outperforms previous methods and achieves state-of-the-art results in all three localization tasks.