Detail

Publication date: 1 de June, 2021

Dialogs and Ontologies for Multimedia Information Retrieval

To go beyond text retrieval, the analysis of document components such as audio or image segments is required. The Metamedia prototype (FEUP) currently incorporates the extraction of audiovisual features and the automatic association of the corresponding descriptors to the document. Further work is required on audiovisual extraction, in order to create more expressive descriptors. Ontologies will be used at this point: both the inclusion of audiovisual descriptors in ontologies and their combination with domain ontologies will be explored.

A second component is dialog management. The retrieval task can be significantly improved by gathering information from the user interaction and analyzing the dialog to extract user intentions and plans. Retrieval is an intrinsically imprecise task and therefore this line has to be complemented by appropriate evaluation procedures and tools.

A third line of research is the refinement of the database model to encompass the association of metadata to objects at different levels, the compliance with audiovisual standards and the use of heterogeneous descriptors in the computation of similarity measures for retrieval. Audiovisual descriptors are commonly multi-dimensional and quantitative; the similarity measures required in retrieval open a large ground for new approaches.

Team

Irene Rodrigues, Paulo Quaresma, Gomes Saias, Luis Jorge Catela Quintano, Maria Cristina de Carvalho Alves Ribeiro,

Sname DOMIR
Reference FCT POSC/EIA/61109/2004
Funding Total 90000 Euro
Results Working prototype.
State Concluded
Startdate 01/03/2005
Enddate 01/01/2007