A Natural User Interface for Gestural Expression and Emotional Elicitation to access the Musical Intangible Cultural Heritage 

Christina Volioti (1) , Sotiris Manitsaris (2) , Edgar Hemery (2) , Vasileios Charisis , Leontios Hadjileontiadis , Stelios Hadjidimitriou (3) , Eleni Katsouli (1) , Fabien Moutarde (2) , Athanasios Manitsaris (1)

1 UoM - University of Macedonia [Thessaloniki]
2 CAOR - Centre de Robotique
3 Université Aristote de Thessalonique

This paper describes a prototype natural user interface, named the Intangible Musical Instrument, which aims to facilitate access to the knowledge of the performers that constitutes musical Intangible Cultural Heritage, using off-the-shelf motion capturing that is easily accessed by the public at large. This prototype is able to capture, model and recognize musical gestures (upper body including fingers) as well as to sonify them. The emotional status of the performer affects the sound parameters at the synthesis level. Intangible Musical Instrument is able to support both learning and performing/composing by providing to the user not only intuitive gesture control but also a unique user experience. In addition, the first evaluation of the Intangible Musical Instrument is presented, in which all the functionalities of the system are assessed. Overall, the results with respect to this evaluation were very promising.

Mots clés: Gesture Recognition; Sonification; Emotional status; Intangible Cultural Heritage; Natural User Interface; Musical gestures; Expert knowledge; Evaluation

Domaines: Sciences de l'ingénieur [physics]; Traitement du signal et de l'image [eess.SP]

Christina Volioti, Sotiris Manitsaris, Edgar Hemery, Vasileios Charisis, Leontios Hadjileontiadis, et al.. A Natural User Interface for Gestural Expression and Emotional Elicitation to access the Musical Intangible Cultural Heritage . Journal on Computing and Cultural Heritage, In press, ⟨10.1145/3127324⟩. ⟨hal-01692849⟩