CVPR 2023, highlight paper - HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling

In this episode we discuss HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling by Benjamin Attal, Jia-Bin Huang, Christian Richardt, Michael Zollhoefer, Johannes Kopf, Matthew O'Toole, Changil Kim. The paper discusses the challenges of creating a memory-efficient and high-quality 6-DoF (Six-Degrees-of-Freedom) video representation for dynamic scenes. Existing methods fail to achieve real-time rendering, high-quality rendering, and small memory footprint for challenging real-world scenes. To address these issues, the authors present HyperReel - a novel 6-DoF video representation that relies on a ray-conditioned sample prediction network and a memory-efficient dynamic volume representation. The system achieves state-of-the-art quality, real-time rendering, and small memory requirements, even for scenes with view-dependent appearance.