arxiv preprint - Conditioned Language Policy: A General Framework for Steerable Multi-Objective Finetuning

In this episode, we discuss Conditioned Language Policy: A General Framework for Steerable Multi-Objective Finetuning by Kaiwen Wang, Rahul Kidambi, Ryan Sullivan, Alekh Agarwal, Christoph Dann, Andrea Michi, Marco Gelmi, Yunxuan Li, Raghav Gupta, Avinava Dubey, Alexandre Ramé, Johan Ferret, Geoffrey Cideron, Le Hou, Hongkun Yu, Amr Ahmed, Aranyak Mehta, Léonard Hussenot, Olivier Bachem, Edouard Leurent. The paper presents Conditioned Language Policies (CLP), a framework for finetuning language models to balance multiple conflicting objectives. CLP leverages multi-task training and parameter-efficient finetuning to allow a single model to navigate trade-offs between objectives during inference. Experiments show that CLP outperforms existing methods, making it a superior approach for creating steerable and flexible language models.