CVPR 2023 - SunStage: Portrait Reconstruction and Relighting using the Sun as a Light Stage

In this episode we discuss SunStage: Portrait Reconstruction and Relighting using the Sun as a Light Stage by Yifan Wang, Aleksander Holynski, Xiuming Zhang, Xuaner Zhang. The paper presents SunStage, a lightweight alternative to a light stage that captures facial appearance and relighting data using only a smartphone camera and the sun. The method requires the user to capture a selfie video outdoors and uses the varying angles between the sun and face for joint reconstruction of facial geometry, reflectance, camera pose, and lighting parameters. The approach is able to reconstruct detailed facial appearance and geometry, enabling compelling effects such as relighting, novel view synthesis, and reflectance editing.