Chrisa Tsinaraki

A Semantic-Based Framework for Multimedia Management and Interoperability

Supervisor(s) and Committee member(s): Supervisor: Stavros Christodoulakis


Interoperable, semantic-based audiovisual content services are necessary in the open Internet environment, where the volume of the available audiovisual information is growing rapidly. Such services can be built on top of structured, semantic-based audiovisual content descriptions.

This thesis focuses on the representation and management of the audiovisual content semantics. The representation and management of the audiovisual content semantics are based on the dominant standards for audiovisual content description and ontology representation which are, respectively, the MPEG-7 and the OWL. In particular, the following components have been developed for the representation and management of the audiovisual content semantics:

  • An MPEG-7 based model that allows: (a) The representation of the audiovisual content semantics; and (b) The representation of domain ontologies that extend the general-purpose MPEG-7 semantics with domain knowledge and may be utilized in the audiovisual content description.
  • An ontological infrastructure, which allows the representation and management of the audiovisual content semantics. This ontological infrastructure represents the MPEG-7 semantics and allows their extension with application-specific and domain-specific knowledge.
  • A mapping model that allows interoperability support between MPEG-7 and OWL. This model maps the MPEG-7 constructs to OWL constructs and allows the transformation of OWL domain ontologies and OWL/RDF audiovisual content descriptions into MPEG-7 descriptions.

The MP7QL (MPeg-7 Query Language) query language and the MP7QL user preference model have also been developed, in order to allow semantic-based retrieval and filtering of the audiovisual content. The MP7QL query language and the MP7QL user preference model allow for the transparent access to the audiovisual material and the expression of conditions for all the components of the MPEG-7 descriptions. In addition, they allow the explicit specification of boolean operators and preference values for the combination of the conditions according to the user intentions.

The above-referred components (ontological infrastructure, model for the representation of the audiovisual content semantics, mapping model, query language and user preference model) comprise the theoretical basis of the DS-MIRF (Domain-Specific Multimedia Information and Filtering Framework). DS-MIRF allows the development of domain knowledge based applications and services for audiovisual content that utilize and extend the MPEG-7 standard.

The DS-MIRF framework comprises of the following components:

  • The DS-MIRF ontological infrastructure, which includes: (a) An OWL-DL Upper Ontology, which captures the semantics of the MPEG-7 standard. Since the MPEG-7 has been expressed in XML Schema syntax, the development of the upper ontology has been based on mapping XML Schema constructs to OWL constructs. The generalization of this methodology led to the development of the XS2OWL mapping model. The XS2OWL mapping model maps XML Schema constructs to OWL constructs, thus allowing the use of Semantic Web tools and methodologies by XML Schema based applications; (b) A set of application ontologies, which extend the upper ontology with application knowledge, so that advanced application support can be provided that utilizes the domain-specific semantics of the different application domains. A semantic user preference ontology, which captures the semantics of the MP7QL user preference model, and a typed relationship ontology have been integrated in the DS-MIRF ontological infrastructure; and (c) A methodology that allows integrating domain ontologies in the DS-MIRF ontological infrastructure. The domain ontologies extend the semantics of the upper ontology and the application ontologies with domain knowledge. The methodology has been tested through the integration of a soccer ontology and a formula 1 ontology.
  • Functionality that allows managing MPEG-7 audiovisual content descriptions, user preference descriptions and domain ontologies.
  • Audiovisual content browsing, retrieval and filtering functionality, as well as audiovisual content service personalization functionality. The retrieval functionality is based on the MP7QL query language, while the personalization functionality and the filtering functionality are based on the MP7QL user preference model.
  • The GraphOnto software component, which was used for the development of the DS-MIRF ontological infrastructure. In addition, the OWL/MPEG-7 mapping model developed in the context of the thesis was implemented in GraphOnto, thus allowing the automatic transformation of OWL domain ontologies and OWL/RDF descriptions into MPEG-7 descriptions.

The audiovisual content semantics representation model developed in the context of this thesis has been applied in the domains of sports and cultural heritage. In the sports domain, the proposed model was evaluated in comparison with the existing approaches, and was shown to be more effective than them in semantic-based retrieval and filtering support for audiovisual content.



The TUC/MUSIC Laboratory was established in 1990 in the Department of Electronics and Computer Engineering of the Technical University of Crete which is located in Chania, Crete, Greece. The TUC/MUSIC Laboratory is a center of research, development and teaching in the areas of distributed information systems, application engineering, computer graphics, and simulation engineering.
In the general area of systems development, the TUC/MUSIC Laboratory has performed research in the areas of high performance distributed multimedia architectures, information systems offering advanced functionalities, data base systems, information retrieval systems, digital libraries, service oriented architectures, and graphics systems.
In the area of application engineering, the TUC/MUSIC Laboratory has performed research in the topics of large distributed multimedia delivery networks for intelligent TV applications, semantic interoperability infrastructures, web and mobile based application development methodologies, natural language processing, as well as standard-based software infrastructures for multimedia applications in areas such as e-learning, culture and tourism, business applications, TV Applications and medicine.
In the area of simulation engineering, the TUC/MUSIC Laboratory is conducting research in the areas of real-time perceptually-based selective rendering algorithms, fidelity metrics for immersive simulations, uncertainty modeling and visualization and human factors engineering. The TUC/MUSIC Lab has participated in over 40 EU projects and Excellence Networks as Partner, Coordinator and Technical Leader.

Bookmark the permalink.