Convolutional neural network with adaptive inferential framework for skeleton-based action recognition
Affiliation:
1. School of Data Science and Technology, North University of China, Taiyuan, Shanxi 030051, China;2. School of Computer Science, Chongqing University, 174 Shazheng Street, Shapingba District 400044, Chongqing, China;3. The State Key Lab of Internet of Things for Smart City, University of Macau, Taipa, Macau 999078, China;1. Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai 200444, China;2. School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China;3. Department of Computer Science, University of Manitoba, Winnipeg, MB R3T 2N2, Canada;4. School of Medical Instrument and Food Engineering, University of Shanghai for Science and Technology, Shanghai, China
Abstract:
In the task of skeleton-based action recognition, CNN-based methods represent the skeleton data as a pseudo image for processing. However, it still remains as a critical issue of how to construct the pseudo image to model the spatial dependencies of the skeletal data. To address this issue, we propose a novel convolutional neural network with adaptive inferential framework (AIF-CNN) to exploit the dependencies among the skeleton joints. We particularly investigate several initialization strategies to make the AIF effective with each strategy introducing the different prior knowledge. Extensive experiments on the dataset of NTU RGB+D and Kinetics-Skeleton demonstrate that the performance is improved significantly by integrating the different prior information. The source code is available at: https://github.com/hhe-distance/AIF-CNN.
