CVPR 2023 - Practical Network Acceleration with Tiny Sets

In this episode we discuss Practical Network Acceleration with Tiny Sets by Guo-Hua Wang, Jianxin Wu. The paper proposes a new method called PRACTISE for accelerating networks using only small training sets. It suggests dropping blocks as a better approach than filter-level pruning for achieving higher acceleration ratio and improved latency-accuracy performance under few-shot settings. The paper introduces a new concept called "recoverability" to measure the difficulty of recovering the compressed network and proposes an algorithm using it to select which blocks to drop. PRACTISE outperforms previous methods by a significant margin and also shows high generalization ability under data-free or out-of-domain data settings.