A Workflow for Real-time Visualization and Data Analysis of Gesture using Motion Capture

Jean-François Jégo (1, 2, 3) , Vincent Meyrueis (4) , Dominique Boutet (5, 6)

1 UP8 - Université Paris 8 Vincennes-Saint-Denis
2 CAOR - Centre de Robotique
3 AIAC - Arts des Images et Art Contemporain
4 LCPI - Laboratoire Conception de Produits et Innovation
5 UNIROUEN - Université de Rouen Normandie
6 DYLIS - Dynamique du Langage In Situ

In this paper, we investigate new ways to understand and to analyze human gesture in a research context applied on co-verbal gesture across language. The research project focuses on the quality of the movement and consider the gesture “pulse of effort.“ We propose a workflow for real-time gesture analysis to visualize gesture kinematics features (Velocity, Acceleration, Jerk) from heterogeneous data (Video, Motion Capture and Gesture Annotations) at the same time base. The tools designed here provide immersive and interactive explorations of data: users can test hypotheses and embody gesture visualization and descriptors adopting different Frames of Reference using augmented reality. We have conducted an evaluation protocol in the field of linguistics that compares 496 annotated gestures to benchmark the workflow.

Mots clés: Gesture; Visualization; Motion Capture; Embodiment; Workflow; Real-time; Augmented Reality

Domaines: Génie des procédés

Jean-François Jégo, Vincent Meyrueis, Dominique Boutet. A Workflow for Real-time Visualization and Data Analysis of Gesture using Motion Capture. MOCO '19: 6th International Conference on Movement and Computing, School of Arts, Media and Engineering at ASU, Oct 2019, TEMPE AZ, United States. pp.1-6, ⟨10.1145/3347122.3359598⟩. ⟨hal-02474193⟩