MULTIMODAL SYSTEMS

PI:   Nuno Correia

• Multimodal Systems Group main goal is to access, process and interact with multimodal content in different contexts using multiple devices and modes of interaction.
  
  Multimodal and natural computer interaction is a key aspect of our research pursuing novel interaction mechanisms based on computer vision, touch and other devices to augment user interaction. We research methods to assess user interaction including the expression, assessment and evaluation of emotions. Handling geo-referenced information is fundamental to explore interaction with multiple information sources in context to better support collaborative awareness. We develop tools and applied our results in several domains, creating collaborative geo-aware context dependent platforms.
  
  We work on algorithms to analyse, search, and visualize large amounts of multimodal data including text, images, audio, video and sensor data. Information retrieval is a key aspect and it targets search over diverse information sources and the scalability of multimedia information retrieval systems. We research information analysis algorithms over dynamic sources such as the social web, and to effectively index multimedia information.
• The research is done with strong connections to society regarding tools for creative, cultural and healthcare industries and institutions.
  
  Our research provides solutions, and the drive for novel methods also arises from this interaction with the society. This activity is fundamental to strengthen our position as a leading national and a key international research group in the area by leveraging on the EU cultural and creative sectors.
  
  The group publishes at major venues such as ACM Multimedia, IFIP Interact, ACM CIVR, ACM CHI, ACE, MobileHCI, ACM ITS, IFIP Interact, Computer & Graphics, Multimedia Tools and Applications, ACM Transactions on Information Systems, Multimedia Systems journal, UIST, ACL and ACM WSDM.