Title: A Conceptual Architecture and a Framework for Dealing with Variability in Mulsemedia Systems

Author: Estêvão Bissoli Saleme

Supervisor(s) and Committee member(s): Dr. Celso Alberto Saibel Santos, Federal University of Espírito Santo (supervisor),
Dr. George Ghinea, Brunel University London (co-supervisor),
Dr. José Gonçalves Pereira Filho, Federal University of Espírito Santo (internal examiner),
Dr. Magnos Martinello, Federal University of Espírito Santo, Brazil (internal examiner),
Dr. Débora Christina Muchaluat Saade, Fluminense Federal University (external examiner),
Dr. Niall Murray, Athlone Institute of Technology (external examiner)

URL: http://tiny.cc/rch8iz

Estêvão Bissoli Saleme

The increasing interest in digital immersive experiences has drawn the attention of researchers into understanding human perception whilst adding sensory effects to multimedia systems such as VR (Virtual Reality) and AR (Augmented Reality) applications, multimedia players, and games. These so-called mulsemedia—multiple sensorial media—systems are capable of delivering wind, smell, vibration, among others, along with audiovisual content with the aim of enhancing users’ Quality of Experience (QoE) in areas such as entertainment, healthcare, education, culture, and marketing. To support the researchers’ investigation, there have been developed many standalone software solutions and incipient architectural proposals to bind these applications to sensory effects devices, such as wind fans, scent emitters, vibration chairs, etc. These devices, in turn, are constantly evolving, making it difficult to update applications to be compatible with them. There is little or no interoperability between software and hardware in this realm, hindering reuse in other contexts. Every time a mulsemedia application is needed, new software is built mostly from scratch. This model has proven to be demanding, time-consuming, and costly mainly because it requires researchers and developers alike to gain knowledge about new devices, connectivity, communication protocols, and other particulars. The fact is that building such systems imposes a number of challenges and requirements (which are discussed in this thesis) due mainly to their ever-evolving and heterogeneous traits. As a result, few mulsemedia systems have remained reusable to be applied to different research purposes as opposed to the use of open mulsemedia datasets. Therefore, the main contribution of this thesis is a decoupled conceptual architecture to deal with variability of scenarios in mulsemedia delivery systems, which includes recommendations to cope with the variation of end-user applications and sensory effect devices through the support and reuse of even unforeseen communication and connectivity protocols, and sensory effects metadata (SEM). To evaluate it, an open-source and robust mulsemedia framework was developed. Then, a performance assessment was carried out on communication protocols for the integration between event-based applications, whereby temporal restrictions play a role, and the framework. Results indicated statistically significant differences in response time providing directions for optimized integrations. Finally, a user QoE subjective evaluation comparing a monolithic mulsemedia system with this framework was undertaken with results suggesting no evinced statistically significant differences in user-perceived QoE between the systems under different aspects. Therefore, it is hoped that this work fosters the area of mulsemedia and HCI (Human-Computer Interaction) in the sense that researchers can leverage either the conceptual architecture to design mulsemedia delivery systems or the framework to carry out their experiments.

Multimedia and Networking Research Lab (LPRM)

URL: https://lprm.inf.ufes.br/

Multimedia and Networking Research Lab (LPRM) is located at UFES (Federal University of Espírito Santo), Vitória-ES, Brazil. Currently, our research spans across the following fields (not limited to them):

– Immersive experiences (VR and mulsemedia)
– Collaborative systems (crowdsourcing)
– Network architectures
– Ubiquitous computing
– The Internet of Things
– Wireless sensor network

Read about our main projects at https://lprm.inf.ufes.br/projects/

Bookmark the permalink.