CVPR 2023 - Ego-Body Pose Estimation via Ego-Head Pose Estimation

In this episode we discuss Ego-Body Pose Estimation via Ego-Head Pose Estimation by Authors: - Jiaman Li - C. Karen Liu† - Jiajun Wu† Affiliation: - Stanford University Contact: - {jiamanli,karenliu,jiajunwu}@cs.stanford.edu. The paper proposes a new method, EgoEgo, for estimating 3D human motion from egocentric video sequences that addresses two challenges. The first challenge is that the user's body is often unobserved by the front-facing camera, and the second challenge is that collecting large-scale, high-quality datasets with paired egocentric videos and 3D human motions requires accurate motion capture devices. EgoEgo decomposes the problem into two stages connected by the head motion as an intermediate representation. It first estimates accurate head motion using SLAM and a learning approach and then generates multiple plausible full-body motions using conditional diffusion. The approach eliminates the need for paired egocentric video and human motion datasets, enabling the leverage of large-scale datasets separately. The EgoEgo model performs significantly better than the current state-of-the-art methods on both a synthetic dataset developed by the authors and real data.