Hierarchical Recurrent Neural Network for
Skeleton Based Action Recognition
 |
| The proposed hierarchical recurrent neural network. |
People
Yong Du
Wei Wang
Liang Wang
Overview
Human actions can be represented by the trajectories of skeleton joints. Traditional methods generally model the spatial structure and temporal dynamics of human skeleton with hand-crafted features and recognize human actions by well-designed classifiers. In this paper, considering that recurrent neural network (RNN) can model the long-term contextual information of temporal sequences well, we propose an end-to-end hierarchical RNN for skeleton based action recognition. Instead of taking the whole skeleton as the input, we divide the human skeleton into five parts according to human physical structure, and then separately feed them to five subnets. As the number of layers increases, the representations extracted by the subnets are hierarchically fused to be the inputs of higher layers. The final representations of the skeleton sequences are fed into a single-layer perceptron, and the temporally accumulated output of the perceptron is the final decision. We compare with five other deep RNN architectures derived from our model to verify the effectiveness of the proposed network, and also compare with several other methods on three publicly available datasets. Experimental results demonstrate that our model achieves the state-of-the-art performance with high computational efficiency.
Paper
 |
Hierarchical Recurrent Neural Network for Skeleton Based Action Recognition Yong Du, Wei Wang, Liang Wang Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR2015) |
Experimental Results
| Experimental results on the Berkeley MHAD. | Experimental results on the HDM05. |
 |
 |
Acknowledgments
This work was supported by the National Basic Research Program of China (2012CB316300) and National Natural Science Foundation of China (61175003, 61135002, 61202328, 61420106015, U1435221).