CVPR 2023 - TimeBalance: Temporally-Invariant and Temporally-Distinctive Video Representations for Semi-Supervised Action Recognition

In this episode we discuss TimeBalance: Temporally-Invariant and Temporally-Distinctive Video Representations for Semi-Supervised Action Recognition by Ishan Rajendrakumar Dave, Mamshad Nayeem Rizve, Chen Chen, Mubarak Shah. The paper proposes a semi-supervised learning framework for action recognition using self-supervised video representations, called TimeBalance. They suggest using temporally-invariant and temporally-distinctive representations that complement each other for different types of actions. TimeBalance distills knowledge from both representations and dynamically combines them using a novel temporal similarity-based reweighting scheme. The approach achieves state-of-the-art performance on three action recognition benchmarks.