CVPR 2023 - NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illumination

In this episode we discuss NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illumination by Haoqian Wu, Zhipeng Hu, Lincheng Li, Yongqiang Zhang, Changjie Fan, Xin Yu. The paper proposes an end-to-end inverse rendering pipeline that decomposes materials and illumination from multi-view images, while considering near-field indirect illumination. They introduce Monte Carlo sampling based path tracing, cache the indirect illumination as neural radiance, and leverage Spherical Gaussians to represent smooth environment illuminations and apply importance sampling techniques to enhance efficiency. They also develop a novel radiance consistency constraint between implicit neural radiance and path tracing results of unobserved rays to significantly improve decomposition performance. Experimental results demonstrate that their method outperforms state-of-the-art methods on multiple synthetic and real datasets.