Project #06.3: Bodily Benchmark (Gestural/Physiological Analysis by Remote/Wearable Sensing)

Abstract

This project was run in col­lab­o­ra­tion with two artis­tic projects: Musi­choré­gra­phie d’appartement by André Serre-Milan and Bio­Diva by Nios Karma. The scope of this project was to offer tech­no­log­i­cal fore­cast­ing and devel­op­ment con­sul­tancy to these two artists that both share a com­mon goal of using ges­tures to con­trol on audio­vi­sual ren­der­ing on stage.

We devel­oped a first pro­to­type for syn­chro­nized record­ing, visu­al­iza­tion and edit­ing of mul­ti­modal sig­nals (audio, video, sen­sors). We updated our tech­nolo­gies for ges­ture recog­ni­tion using Dynamic Time Warp­ing (DTW) and map­ping inter­po­la­tion. We ported our longterm atten­tion com­puter vision algo­rithm from Eye­sWeb to Max/MSP. We achieved ini­tial promis­ing results for the pro­to­typ­ing of the Bio­Diva ges­tural inter­face and for the analy­sis using the long-term atten­tion model of the Musi­choré­gra­phie d’appartement record­ing sessions.

We plan to improve our mul­ti­modal sig­nal record­ing and analy­sis tool towards a more effi­cient anno­ta­tion tool.  We plan to build a ded­i­cated wire­less sen­sor inter­face to over­come all the related issues we met so far. We plan to use new tex­tiles for the improve­ment of sen­sors, sen­sor inter­faces and inte­gra­tion in per­form­ers’ costumes.

Fig­ure: Evo­lu­tion of Nios Karma’s per­for­mance setup con­strained by micro­phone, com­puter stand, midi fader box, etc…

Evolution of Nios Karma performance setup constrained by microphone, computer stand, midi fader box, etc…

Results

Doc­u­ments

  • [2009,inproceedings] bib­tex Go to document
    C. Fris­son, L. Rebour­sière, T. Todo­r­off, and J. Fila­triau, “Bod­ily Bench­mark: Gestural/Physiological Analy­sis by Remote/Wearable Sens­ing,” in QPSR of the nume­di­art research pro­gram, 2009, pp. 41–57.
    @INPROCEEDINGS{numediart_2009_s06_p3_report,
      author = {Christian Frisson and Lo\"{i}c Reboursi\`{e}re and Todor Todoroff and Jean-Julien Filatriau},
      title = {Bodily Benchmark: Gestural/Physiological Analysis by Remote/Wearable Sensing},
      booktitle = {{QPSR} of the numediart research program},
      year = {2009},
      editor = {Thierry Dutoit and Beno\^{i}t Macq},
      volume = {2},
      number = {2},
      pages = {41-57},
      month = {6},
      organization = {numediart Research Program on Digital Art Technologies},
      url = {http://www.numediart.org/docs/numediart_2009_s06_p3_report.pdf}
    }