Project #06.1: Laughter Cycle (Query-by-Example and Navigation Approaches for Audio Content)

Idea

This project aims at devel­op­ing a web-based ser­vice for effi­ciently nav­i­gat­ing inside a laugh­ter data­base. It will enable users to quickly and eas­ily find laugh­ter utter­ances cor­re­spond­ing to their wishes. Two main approaches to explore the data­base will be pro­vided: query-by-example (the user gives a laugh­ter utter­ance to the sys­tem, which finds sim­i­lar sam­ples in the data­base) and query-by-keyword (using tags stored as meta-data and char­ac­ter­iz­ing the utterances).

The query-by-example approach requires an eval­u­a­tion of sim­i­lar­i­ties between laugh­ter occur­rences. Sev­eral acoustic prop­er­ties will be inves­ti­gated to assess to what extent they model human per­cep­tions of sim­i­lar­i­ties between laugh­ters. Char­ac­ter­is­tics usu­ally com­puted in speech and music analy­sis (e.g. descrip­tors of pitch, tim­bre, har­mony, melody or rhythm) will be adapted to laugh­ter. Fea­tures spe­cific to laugh­ter will also be con­sid­ered, for exam­ple the ratio of voiced ver­sus unvoiced seg­ments or descrip­tors char­ac­ter­iz­ing the peri­od­ic­ity and dura­tions of such seg­ments. The laugh­ter utter­ances from the data­base will be visu­ally orga­nized accord­ing to their sim­i­lar­ity with the exam­ple along one dimen­sion and the sim­i­lar­i­ties between them along another space dimen­sion, in order to ease the local­iza­tion of the region of inter­est. The user will be able to lis­ten to any dis­played sam­ple, define it as the new exam­ple towards which sim­i­lar­i­ties must be esti­mated and reor­ga­nize the data­base visu­al­iza­tion by, for exam­ple, weight­ing the acoustic prop­er­ties involved in the sim­i­lar­ity evaluation.

The query-by-keyword approach will only dif­fer in the way data is accessed. Accord­ing to the requested keyword(s) and the data­base tags (pos­si­bly auto­mat­i­cally extracted through char­ac­ter­is­tic fea­tures afore­men­tioned), the soft­ware will pro­pose the most rel­e­vant sam­ples and place all the objects around them. The user will then be able to nav­i­gate inside the orga­nized data­base, using either key­words or acoustic sim­i­lar­i­ties to refine his search.

The soft­ware will be embed­ded in a web-based ser­vice. This implies pro­vid­ing a quick access to the laugh­ter data­base remotely stored and man­aged on one hand and, on the other hand, devel­op­ing a web appli­ca­tion to run the audio­vi­sual ren­der­ing on the client com­puter. In addi­tion, this appli­ca­tion must enable the user to record its own laugh­ter and send it to the server for start­ing a query-by-example search. In con­se­quence, world-wide users will be able to enrich the data­base with new laugh­ter utter­ances and find sim­i­lar­i­ties with other laugh­ers. The web­site will sup­port a laugh­ing com­mu­nity. Design­ing this web­site, start­ing by estab­lish­ing its spec­i­fi­ca­tions and study­ing solu­tions to develop it (Rich Inter­net Appli­ca­tions being good can­di­dates), will be an impor­tant part of the project. It can be split into 3 aspects cor­re­spond­ing to 3 devel­op­ers profiles:

  • server side appli­ca­tion, man­ag­ing the data­base of laugh­ters and com­mu­nity mem­bers, likely based on PHP/MySQL or some equiv­a­lent technologies.
  • client side GUI and inter­ac­tion, prob­a­bly using Flash or AJAX.
  • client side graph­ics design.

Moti­va­tions

Laugh­ter is an essen­tial sig­nal in human com­mu­ni­ca­tions. It con­veys infor­ma­tion about our feel­ings and helps to cheer up our mood. More­over, it is com­mu­nica­tive and has the poten­tial to elicit emo­tions to its lis­ten­ers. Laugh­ter is also known to have healthy effects, and espe­cially to be one of the best med­i­cines against stress. Laugh­ter ther­a­pies, “yoga” ses­sions or groups are emerg­ing every­where, ampli­fied by these cri­sis times where peo­ple need to relax and for­get about the neg­a­tive things. We can cite the laugh­ter chain launched on Skype, laugh­ters clubs or the World Laugh­ter Day. The first objec­tive of Laugh­ter Cycle is con­cerned by this strong social impact. Inspired by singing com­mu­ni­ties like Midomi and the mis­cel­la­neous social net­works emerg­ing over the inter­net, we would like to cre­ate a laugh­ing com­mu­nity, where users could enjoy all the ben­e­fits of laughter.

In addi­tion, due to the grow­ing inter­est for vir­tual agents mod­el­ing human behav­iors, there is a need to enable these agents to per­ceive and express emo­tions. Laugh­ter is clearly an impor­tant cue for under­stand­ing emo­tions and dis­course events on one hand, and, on the other, to man­i­fest cer­tain emo­tions. How­ever, from an engi­neer­ing point of view, laugh­ter is an under­stud­ied sig­nal. The sec­ond goal of this project will be to extend the state-of-the art in laugh­ter pro­duc­tion. Pure laugh­ter syn­the­sis is very dif­fi­cult and, to date, unef­fi­cient. We would like to help peo­ple find­ing eas­ily a desired laugh­ter sound. We have acquired 2 data­bases con­tain­ing sev­eral thou­sands of laugh­ters, cov­er­ing the largest pos­si­ble range of laugh­ter kinds, that will be pri­mar­ily used to this end. Orga­niz­ing the data­base accord­ing to sim­i­lar­i­ties will facil­i­tate and accel­er­ate the search.

This project lies in the con­ti­nu­ity of the work done within a pre­vi­ous nume­di­art project Audio Cycle (#04.1), where a graph­i­cal inter­face to browse a large music library and help cre­at­ing new per­for­mances has been designed. Sim­i­lar­i­ties between music loops have been used to spa­tially orga­nize the cor­pus and syn­chro­nized 3D Audio ren­der­ing of the acti­vated loops has been implemented.

 

Results

The project report is avail­able as well as its ref­er­ence entry .