arxiv preprint - NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis

In this episode we discuss NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis by Nilesh Kulkarni, Davis Rempe, Kyle Genova, Abhijit Kundu, Justin Johnson, David Fouhey, Leonidas Guibas. The paper presents a method called NIFTY, which utilizes a neural interaction field to generate 3D human motions interacting with objects in a scene. The interaction field guides the sampling of an object-conditioned human motion diffusion model to ensure plausible contacts and affordance semantics. To overcome data scarcity, the paper introduces a synthetic data pipeline using a pre-trained motion model and interaction-specific anchor poses to train a guided diffusion model, resulting in realistic motions for sitting and lifting with various objects.