Demo of Audio-driven dance performance analysis
By S. Essid
This work addressed the Huawei/3Dlife Grand challenge by proposing a set of audio tools for a virtual dance-teaching assistant. These tools are meant to help dance students develop a sense of rhythm to correctly synchronize their movements and steps to the musical timing of the choreographies to be executed. They consist of three main components, namely a music (beat) analysis module, a source separation and remastering module and a dance step segmentation module. These components enable to create augmented tutorial videos highlighting the rhythmic information using, for instance, a synthetic dance teacher voice, but also videos highlighting the steps executed by a student to help in the evaluation of his/her performance. Examples of such videos, prepared by Robin Tournemenne, are given hereafter. Check the related publications.
This is an example of dance teacher videos augmented with audio effects highlighting the musical timing information.
Original video for the 5 th choreography
|Synthetic teacher voice
|Synthetic hand clapping
|Step evaluation with beeps sounding on detected steps only
Step verification with a mix of onfloor piezo sounds and
beeps on steps detected
Musical evaluation to control if steps are consistent
with the musical timing