Neurips 2023 spotlight - Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems

In this episode we discuss Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems by Benjamin Coleman, Wang-Cheng Kang, Matthew Fahrbach, Ruoxi Wang, Lichan Hong, Ed H. Chi, Derek Zhiyuan Cheng. The paper introduces a framework called Feature Multiplexing, which allows for the use of a single representation space across multiple categorical features in web-scale machine learning systems. This framework addresses the high parameter count issue that arises from representing each feature value as a d-dimensional embedding. The paper also proposes a practical approach called Unified Embedding, which simplifies feature configuration, adapts to dynamic data distributions, and is compatible with modern hardware. The effectiveness of Unified Embedding is demonstrated in improving offline and online metrics across various web-scale systems.