AI Breakdown

arxiv preprint - Language Models can be Logical Solvers

In this episode we discuss Language Models can be Logical Solvers
by Jiazhan Feng, Ruochen Xu, Junheng Hao, Hiteshi Sharma, Yelong Shen, Dongyan Zhao, Weizhu Chen. The paper presents LOGIPT, a new language model designed to tackle complex logical reasoning by directly mimicking the reasoning process of logical solvers, which avoids errors caused by parsing natural language into symbolic representations. LOGIPT is fine-tuned using a dataset that captures the hidden reasoning steps of deductive solvers, ensuring strict adherence to solver syntax and grammar. The model's performance surpasses that of existing solver-augmented language models and few-shot prompting techniques on benchmark deductive reasoning datasets.

ArXiv Preprint - Prompt Engineering a Prompt Engineer

In this episode we discuss Prompt Engineering a Prompt Engineer
by Qinyuan Ye, Maxamed Axmed, Reid Pryzant, Fereshte Khani. The paper presents PE2, an advanced method for automatically engineering prompts for large language models (LLMs), enabling them to perform better at complex tasks. By incorporating elements like a step-by-step reasoning template and verbalized optimization concepts (akin to batch size and momentum), PE2 significantly improves LLMs' task performance, surpassing previous methods on various datasets. The versatility and effectiveness of PE2 are demonstrated through successful applications across different benchmarks, including the Instruction Induction benchmark and real-world industrial prompts, with the method showing a strong ability to refine and correct existing prompts.

arxiv preprint - CogVLM: Visual Expert for Pretrained Language Models

In this episode we discuss CogVLM: Visual Expert for Pretrained Language Models
by Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, Jie Tang. CogVLM is an open-source visual language foundation model that significantly improves the integration of vision and language by incorporating a trainable visual expert module within a pre-trained language model's attention and feed-forward layers. Unlike other models, CogVLM deeply fuses visual and language features without losing any natural language processing capabilities. It delivers state-of-the-art results on several cross-modal benchmarks and is competitive on others, with resources and code accessible publicly.

ArXiv Preprint - De-Diffusion Makes Text a Strong Cross-Modal Interface

In this episode we discuss De-Diffusion Makes Text a Strong Cross-Modal Interface
by Chen Wei, Chenxi Liu, Siyuan Qiao, Zhishuai Zhang, Alan Yuille, Jiahui Yu. The paper introduces De-Diffusion, a new approach that uses text to represent images. An autoencoder is used to transform an image into text, which can be reconstructed back into the original image using a pre-trained text-to-image diffusion model. The De-Diffusion text representation of images is shown to be accurate and comprehensive, making it compatible with various multi-modal tasks and achieving state-of-the-art performance on vision-language tasks.

ArXiv Preprint - E3 TTS: Easy End-to-End Diffusion-based Text to Speech

In this episode we discuss E3 TTS: Easy End-to-End Diffusion-based Text to Speech
by Yuan Gao, Nobuyuki Morioka, Yu Zhang, Nanxin Chen. The paper introduces Easy End-to-End Diffusion-based Text to Speech (E3 TTS), an innovative text-to-speech model that converts text to audio using a diffusion process without the need for intermediate representations or alignment information. E3 TTS functions through iterative refinement directly from plain text to audio waveform, supporting flexible latent structures that enable zero-shot tasks like editing. The model has been tested and offers high-fidelity audio generation, comparable to the performance of advanced neural TTS systems, with samples available online for evaluation.

ArXiv Preprint - Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges

In this episode we discuss Holistic Analysis of Hallucination in GPT-4V(ision): Bias and Interference Challenges
by Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, Huaxiu Yao. The study introduces the Bingo benchmark to analyze hallucination behavior in GPT-4V(ision), a model processing both visual and textual data. Hallucinations, categorized as either bias or interference, reveal that GPT-4V(ision) prefers Western-centric images and is sensitive to how questions and images are presented, with established mitigation strategies proving ineffective. The findings expose similar issues in other leading visual-language models, suggesting an industry-wide challenge that necessitates novel solutions.

ArXiv Preprint - Learning From Mistakes Makes LLM Better Reasoner

In this episode we discuss Learning From Mistakes Makes LLM Better Reasoner
by Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, Weizhu Chen. The paper introduces LEarning from MistAkes (LEMA), a method that improves large language models' (LLMs) ability to solve math problems by fine-tuning them using GPT-4-generated mistake-correction data pairs. LEMA involves identifying an LLM's errors in reasoning, explaining why the mistake occurred, and providing the correct solution. LEMA showed significant performance enhancements on mathematical reasoning tasks, surpassing state-of-the-art performances of open-source models, with the intention to release the code, data, and models publicly.

ArXiv Preprint - The Generative AI Paradox: ”What It Can Create, It May Not Understand”

In this episode we discuss The Generative AI Paradox: "What It Can Create, It May Not Understand"
by Peter West, Ximing Lu, Nouha Dziri, Faeze Brahman, Linjie Li, Jena D. Hwang, Liwei Jiang, Jillian Fisher, Abhilasha Ravichander, Khyathi Chandu, Benjamin Newman, Pang Wei Koh, Allyson Ettinger, Yejin Choi. The paper examines the paradox in generative AI models where they excel in output generation but struggle with comprehension. The authors propose the Generative AI Paradox hypothesis, stating that the models acquire superior generative abilities without corresponding understanding abilities. They compare the performance of humans and models in language and image tasks and find that while models outperform humans in generation, they consistently lag behind in understanding, cautioning against comparing AI to human intelligence.

ArXiv Preprint - TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise

In this episode we discuss TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language Modeling Likewise
by Nan He, Hanyu Lai, Chenyang Zhao, Zirui Cheng, Junting Pan, Ruoyu Qin, Ruofan Lu, Rui Lu, Yunchen Zhang, Gangming Zhao, Zhaohui Hou, Zhiyuan Huang, Shaoqing Lu, Ding Liang, Mingjie Zhan. The paper introduces TeacherLM, a series of language models designed to teach other models. The TeacherLM-7.1B model achieved a high score on MMLU and outperformed models with more parameters. It also has data augmentation abilities and has been used to teach multiple student models.

ArXiv Preprint - MM-VID: Advancing Video Understanding with GPT-4V(ision)

In this episode we discuss MM-VID: Advancing Video Understanding with GPT-4V(ision)
by Kevin Lin, Faisal Ahmed, Linjie Li, Chung-Ching Lin, Ehsan Azarnasab, Zhengyuan Yang, Jianfeng Wang, Lin Liang, Zicheng Liu, Yumao Lu, Ce Liu, Lijuan Wang. The paper introduces MM-VID, a system that incorporates GPT-4V with vision, audio, and speech experts to enhance video understanding. It focuses on handling complex tasks like tracking character storylines across multiple episodes. The paper showcases the capabilities of MM-VID through detailed responses and demonstrations in various figures.