Project #11.2 CoMediAnnotate: towards a usable multimodal annotation framework

Abstract

This project aims at improv­ing the user expe­ri­ence regard­ing mul­ti­me­dia con­tent anno­ta­tion. We eval­u­ated and com­pared cur­rent timeline-based anno­ta­tion tools, so as to elicit user require­ments. We address two issues: 1) adapt­ing the user inter­face, by sup­port– ing more input modal­i­ties through a rapid pro­to­typ­ing tool and by offer­ing alter­na­tive visu­al­iza­tion tech­niques of tem­po­ral sig­nals; and 2) cov­er­ing more steps of the anno­ta­tion work­flow besides the task of anno­ta­tion itself: notably record­ing mul­ti­modal sig­nals.
We devel­oped input devices com­po­nents for the Open­In­ter­face (OI) plat­form for rapid pro­to­typ­ing of mul­ti­modal inter­faces: mul­ti­touch screen, jog wheels and pen-based solu­tions. We mod­i­fied an anno­ta­tion tool cre­ated with the Smart Sen­sor Inte­gra­tion (SSI) toolkit and com­po­nen­tized it in OI so as to bind its con­trols to dif­fer­ent input devices. We pro­duced mock­ups sketches towards a new design of an improved user inter­face for mul­ti­me­dia con­tent anno­ta­tion, and started devel­op­ing a rough pro­to­type using the Pro­cess­ing Devel­op­ment Environment.

Our solu­tion allows to pro­duce sev­eral pro­to­types by vary­ing the inter­ac­tion pipeline: chang­ing input modal­i­ties and using either the ini­tial GUI of the anno­ta­tion tool, or the newly-designed one. We tar­get usabil­ity test­ing to val­i­date our solu­tion and deter­mine which input modal­i­ties com­bi­na­tion best suits given use cases.

Doc­u­ments

  • [2010,inproceedings] bib­tex Go to document
    Fris­son Chris­t­ian, Alaçam Sema, Coşkun Emirhan, Ertl Dominik, Kay­alar Ceren, Law­son Lionel, Lin­gen­felser Flo­rian, Wag­ner Johannes, « CoMe­di­An­no­tate: towards more usable mul­ti­me­dia con­tent anno­ta­tion by adapt­ing the user inter­face », in « QPSR of the nume­di­art research pro­gram », 2010, pp. 45–55.
    @INPROCEEDINGS{numediart_2010_s11_p2_report,
      author = {Christian Frisson and Sema Ala\c{c}am and Emirhan Co\c{s}kun and Dominik Ertl and Ceren Kayalar and Lionel Lawson and Florian Lingenfelser and Johannes Wagner},
      title = {CoMediAnnotate: towards more usable multimedia content annotation by adapting the user interface},
      booktitle = {{QPSR} of the numediart research program},
      year = {2010},
      editor = {Thierry Dutoit and Beno\^{i}t Macq},
      volume = {3},
      number = {3},
      pages = {45-55},
      month = {9},
      organization = {numediart Research Program on Digital Art Technologies},
      url = {http://www.numediart.org/docs/numediart_2010_s11_p2_report.pdf}
    }
  • [2010,inproceedings] bib­tex Go to document
    Fris­son Chris­t­ian, Alaçam Sema, Coşkun Emirhan, Ertl Dominik, Kay­alar Ceren, Law­son Lionel, Lin­gen­felser Flo­rian, Wag­ner Johannes, « CoMe­di­An­no­tate: towards more usable mul­ti­me­dia con­tent anno­ta­tion by adapt­ing the user inter­face », in « Pro­ceed­ings of the eNTERFACE’10 Sum­mer Work­shop on Mul­ti­modal Inter­faces », Ams­ter­dam, The Nether­lands, 2010, pp. 1–11.
    @INPROCEEDINGS{CoMediAnnotate,
      author = {Christian Frisson and Sema Ala\c{c}am and Emirhan Co\c{s}kun and Dominik Ertl and Ceren Kayalar and Lionel Lawson and Florian Lingenfelser and Johannes Wagner},
      title = {CoMediAnnotate: towards more usable multimedia content annotation by adapting the user interface},
      booktitle = {Proceedings of the e{NTERFACE}'10 Summer Workshop on Multimodal Interfaces},
      year = {2010},
      editor = {Albert Ali Salah and Theo Gevers},
      pages = {1-11},
      address = {Amsterdam, The Netherlands},
      month = {July 12th - August 6th},
      url = {http://enterface10.science.uva.nl}
    }