AI Breakdown

Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents

In this episode, we discuss Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents by Davide Paglieri, Bartłomiej Cupiał, Jonathan Cook, Ulyana Piterbarg, Jens Tuyls, Edward Grefenstette, Jakob Nicolaus Foerster, Jack Parker-Holder, Tim Rocktäschel. The paper introduces a framework enabling large language model agents to dynamically decide when to plan during task execution, improving efficiency and performance. They propose a two-stage training process combining supervised fine-tuning and reinforcement learning to develop this capability. Experiments show these dynamically planning agents are more sample-efficient, achieve complex goals better, and can be guided by human plans.

Why Language Models Hallucinate

In this episode, we discuss Why Language Models Hallucinate by The authors of the paper are:
- Adam Tauman Kalai
- Ofir Nachum
- Santosh S. Vempala
- Edwin Zhang. The paper explains that hallucinations in large language models arise because training and evaluation reward guessing over admitting uncertainty, framing the issue as errors in binary classification. It shows that models become incentivized to produce plausible but incorrect answers to perform well on benchmarks. The authors propose that addressing hallucinations requires changing how benchmarks are scored, promoting more trustworthy AI by discouraging penalization of uncertain responses.

Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens

In this episode, we discuss Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens by Chengshuai Zhao, Zhen Tan, Pingchuan Ma, Dawei Li, Bohan Jiang, Yancheng Wang, Yingzhen Yang, Huan Liu. The paper investigates Chain-of-Thought (CoT) reasoning in large language models, revealing it may not reflect true inferential processes but rather learned patterns tied to training data distributions. Using a controlled environment called DataAlchemy, the authors show CoT reasoning breaks down when models face out-of-distribution tasks, lengths, or formats. This highlights the limitations of CoT prompting and the challenge of achieving authentic, generalizable reasoning in LLMs.

Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models

In this episode, we discuss Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models by Vlad Sobal, Wancong Zhang, Kyunghyun Cho, Randall Balestriero, Tim G. J. Rudner, Yann LeCun. The paper compares model-free reinforcement learning and model-based control methods for solving navigation tasks using offline, reward-free data. It finds that reinforcement learning performs best with large, high-quality datasets, while model-based planning with latent dynamics models generalizes better to new environments and handles suboptimal data more efficiently. Overall, latent model-based planning is highlighted as a robust approach for offline learning and adapting to diverse tasks.

Persona Vectors: Monitoring and Controlling Character Traits in Language Models

In this episode, we discuss Persona Vectors: Monitoring and Controlling Character Traits in Language Models by Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans, Jack Lindsey. The paper introduces persona vectors in large language models’ activation space that correspond to traits like evil or sycophancy and can track personality changes. These vectors help predict, control, and mitigate unintended personality shifts during training and deployment. Additionally, the method automates persona vector extraction from natural language descriptions and aids in identifying problematic training data.

Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning

In this episode, we discuss Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoning by Jaedong Hwang, Kumar Tanmay, Seok-Jin Lee, Ayush Agrawal, Hamid Palangi, Kumar Ayush, Ila Fiete, Paul Pu Liang. The paper introduces GEOFACT-X, a multilingual factual reasoning benchmark with annotated reasoning traces in five languages to better evaluate language consistency in LLM reasoning. It proposes BRIDGE, a training method using supervised fine-tuning and reinforcement learning with a language-consistency reward to align model reasoning with the input language. Experiments show that BRIDGE significantly improves multilingual reasoning fidelity, highlighting the importance of reasoning-aware multilingual reinforcement learning for cross-lingual generalization.

Position: The AI Conference Peer Review Crisis Demands Author Feedback and Reviewer Rewards

In this episode, we discuss Position: The AI Conference Peer Review Crisis Demands Author Feedback and Reviewer Rewards by Jaeho Kim, Yunseok Lee, Seulki Lee. The paper addresses challenges in AI conference peer review caused by massive submission volumes and declining review quality. It proposes a bi-directional review system where authors evaluate reviewers, and reviewers receive formal accreditation to improve accountability. The paper focuses on reforming reviewer responsibility through a two-stage feedback loop and incentive mechanisms to promote sustainable, high-quality reviews.

Working with AI: Measuring the Occupational Implications of Generative AI

In this episode, we discuss Working with AI: Measuring the Occupational Implications of Generative AI by Kiran Tomlinson, Sonia Jaffe, Will Wang, Scott Counts, Siddharth Suri. The paper analyzes 200,000 anonymized interactions between users and Microsoft Bing Copilot to understand how AI assists with various work activities. It identifies information gathering, writing, teaching, and advising as key activities supported by AI and calculates an AI applicability score across occupations. The study finds the highest AI impact on knowledge work and communication-related jobs, highlighting correlations with wage, education, and real-world AI usage patterns.

Towards physician-centered oversight of conversational diagnostic AI

In this episode, we discuss Towards physician-centered oversight of conversational diagnostic AI by Elahe Vedadi, David Barrett, Natalie Harris, Ellery Wulczyn, Shashir Reddy, Roma Ruparel, Mike Schaekermann, Tim Strother, Ryutaro Tanno, Yash Sharma, Jihyeon Lee, Cían Hughes, Dylan Slack, Anil Palepu, Jan Freyberg, Khaled Saab, Valentin Liévin, Wei-Hung Weng, Tao Tu, Yun Liu, Nenad Tomasev, Kavita Kulkarni, S. Sara Mahdavi, Kelvin Guu, Joëlle Barral, Dale R. Webster, James Manyika, Avinatan Hassidim, Katherine Chou, Yossi Matias, Pushmeet Kohli, Adam Rodman, Vivek Natarajan, Alan Karthikesalingam, David Stutz. The paper proposes g-AMIE, a multi-agent AI system that performs patient history intake within safety guardrails and then presents assessments to a primary care physician (PCP) for asynchronous oversight and final decision-making. In a randomized virtual study, g-AMIE outperformed nurse practitioners, physician assistants, and PCPs in intake quality and diagnostic recommendations, while enabling more time-efficient physician oversight. This demonstrates the potential for asynchronous human-AI collaboration in diagnostic care, maintaining safety and accountability.

Learning without training: The implicit dynamics of in-context learning

In this episode, we discuss Learning without training: The implicit dynamics of in-context learning by Benoit Dherin, Michael Munn, Hanna Mazzawi, Michael Wunder, Javier Gonzalvo. The paper investigates how Large Language Models (LLMs) can learn new patterns during inference without weight updates, a phenomenon called in-context learning. It proposes that the interaction between self-attention and MLP layers in transformer blocks enables implicit, context-dependent weight modifications. Through theoretical analysis and experiments, the authors show that this mechanism effectively produces low-rank weight updates, explaining the model's ability to learn from prompts alone.