Monday Aug 07, 2023
arxiv Preprint - Exploring Format Consistency for Instruction Tuning
In this episode we discuss Exploring Format Consistency for Instruction Tuning by Shihao Liang, Kunlun Zhu, Runchu Tian, Yujia Qin, Huadong Wang, Xin Cong, Zhiyuan Liu, Xiaojiang Liu, Maosong Sun. The paper investigates the impact of format inconsistency on the performance of instruction tuning and proposes a framework called "Unified Instruction Tuning" (UIT) that utilizes OpenAI APIs for automatic format transfer. The authors demonstrate that UIT improves generalization performance on unseen instructions, emphasizing the importance of format consistency. They also propose a perplexity-based denoising method and a smaller offline model to make UIT more practical, with codes and trained models publicly available.
Comments (0)
To leave or reply to comments, please download free Podbean or
No Comments
To leave or reply to comments,
please download free Podbean App.