ICLR 2023 - Copy Is All You Need

In this episode we discuss Copy Is All You Need by Tian Lan, Deng Cai, Yan Wang, Heyan Huang, Xian-Ling Mao. The paper presents a novel approach to text generation by using copy-and-paste operations from an existing text collection instead of selecting from a fixed vocabulary. Contextualized representations of text segments are computed and indexed for efficient retrieval. Experimental results show improved generation quality compared to traditional models, with comparable inference efficiency. The approach also enables effective domain adaptation and performance enhancement with larger text collections.