Project #15.3 MAGE / pHTS in Collaborative Vocal Puppetry (CoVoP) – Multi-User Performative HMM-based Voice Synthesis on Distributed Platforms

Demo

Abstract

Speech production is a complex phenomenon with many parameters. It is very difficult for one performer to control all aspects of a synthesizer that models this phenomenon. We designed and developed a distributed, multi-user system to tackle this difficulty, where users control different aspects of the synthesizer simultaneously and interactively; treating the complex production process as a social game. HMM-based synthesizers provide flexibility at a high level of naturalness, thus we chose HTS as our synthesizer. However, HTS needs severe architectural modifications to be used reactively, and a major achievement of this work was creating MAGE/pHTS, a library for performative HMM-based speech and singing synthesis. The resulting system provides interactive controls for phonetic content and context, as well as prosody using the previously existing HandSketch controller.

Documents

  • [2011,inproceedings] bibtex Go to document
    Astrinaki Maria, Babacan Onur, d’Alessandro Nicolas, Dutoit Thierry, Fels Sidney, « MAGE / pHTS in Collaborative Vocal Puppetry (CoVoP) – Multi-User Performative HMM-based Voice Synthesis on Distributed Platforms », in « QPSR of the numediart research program », 2011, pp. 59-64.
    @INPROCEEDINGS{numediart_2011_s15_p3_report,
      author = {Maria Astrinaki and Onur Babacan and Nicolas d'Alessandro and Thierry Dutoit and Sidney Fels},
      title = {MAGE / pHTS in Collaborative Vocal Puppetry (CoVoP) - Multi-User Performative HMM-based Voice Synthesis on Distributed Platforms},
      booktitle = {{QPSR} of the numediart research program},
      year = {2011},
      editor = {Thierry Dutoit},
      volume = {4},
      number = {3},
      pages = {59-64},
      month = {9},
      organization = {numediart Research Program on Digital Art Technologies},
      url = {http://www.numediart.org/docs/numediart_2011_s15_p3_report.pdf}
    }

This project was featured at eNTERFACE’11, the 7th International Summer Workshop on Multimodal Interfaces in Plzeň (Pilsen), Czech Republic, in August 1-26 2011.