CVPR 2023 - MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Driving

In this episode we discuss MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Driving by Jiale Li, Hang Dai, Hao Han, Yong Ding. This paper proposes a multi-modal 3D semantic segmentation model (MSeg3D) for autonomous driving, combining LiDAR and camera data. The authors address several challenges with multi-modal solutions, including modality heterogeneity, limited sensor field of view intersection, and multi-modal data augmentation. MSeg3D uses joint intra-modal feature extraction and inter-modal feature fusion, and achieves state-of-the-art results on several datasets. The authors also provide their code on GitHub for public use.