Deep Learning for Hand Gesture Recognition on Skeletal Data

Guillaume Devineau (1) , Wang Xi (2) , Fabien Moutarde (1) , Jie Yang (2)

1 CAOR - Centre de Robotique
2 Shanghai Jiao Tong University [Shanghai]

— In this paper, we introduce a new 3D hand gesture recognition approach based on a deep learning model. We introduce a new Convolutional Neural Network (CNN) where sequences of hand-skeletal joints' positions are processed by parallel convolutions; we then investigate the performance of this model on hand gesture sequence classification tasks. Our model only uses hand-skeletal data and no depth image. Experimental results show that our approach achieves a state-of-the-art performance on a challenging dataset (DHG dataset from the SHREC 2017 3D Shape Retrieval Contest), when compared to other published approaches. Our model achieves a 91.28% classification accuracy for the 14 gesture classes case and an 84.35% classification accuracy for the 28 gesture classes case.

Domaines: Informatique [cs]; Vision par ordinateur et reconnaissance de formes [cs.CV]

Guillaume Devineau, Wang Xi, Fabien Moutarde, Jie Yang. Deep Learning for Hand Gesture Recognition on Skeletal Data. 13th IEEE Conference on Automatic Face and Gesture Recognition (FG'2018), May 2018, Xi'An, China. ⟨10.1109/FG.2018.00025⟩. ⟨hal-01737771⟩