arxiv preprint - An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels

In this episode, we discuss An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels by Duy-Kien Nguyen, Mahmoud Assran, Unnat Jain, Martin R. Oswald, Cees G. M. Snoek, Xinlei Chen. This paper questions the necessity of locality inductive bias in modern computer vision architectures by showing that vanilla Transformers can treat each individual pixel as a token and still achieve high performance. The authors demonstrate this across three tasks: object classification, self-supervised learning via masked autoencoding, and image generation with diffusion models. Despite its computational inefficiency, this finding suggests reconsidering design principles for future neural architectures in computer vision.