Comedia: How to become a multimedia director?Comedia is an R&D axis which aims at integrating interactivity between artists and the multimedia context on stage. Until now, both are quite independent: performing artists versus prerecorded audiovisual environment, directed by a stage manager. With the arrival of digital arts, this interactivity starts existing to add smoothness to the performance. Hence, the main aim of Comedia is to analyze actions from artists (their gesture, position, voice) to automatically turn them into corresponding multimedia events through image and video understanding, sensor-based signal interpretation and speech and audio processing. Artists may then become their own director of the multimedia performance and in live realization.Along the analysis of audio and video to detect stage events, Comedia will also target to design ergonomic tools for multime  dia script encoding. It is not only more a temporal or sequential description of the storyboard but a n-dimensional storyboard evolution, which is required for digital arts performance. This advanced storyboard will use concepts of interactive storytelling, existing in several applications, among which 3D games. The main challenge to reach all these breakthroughs will be their realtime realization through dedicated techniques, using programmation onto GPU for example.

 For more details, Comedia will exploit expertise of all partners in terms of image processing, pattern recognition, multisensor fusion, audio and speech processing, augmented reality, 3D image synthesis and grid computation.