IEEE Multimedia

IEEE Multimedia

New Signals in Multimedia Systems and Applications: Sensing and Understanding Human Behavior and Interactions

The objective of this special issue is to revisit how sensor technology is transforming the way context and human behavior is reasoned about, enabling personal and enriched media-experiences. In particular, we are interested in the different stages of the life cycle from the moment multimodal signals are captured and processed, to the inferring of meaning, to the reaction of the environment. The number of existing sensors and promises surrounding them constantly increase. Some examples include social sensors, accessories such as watches, and biosensors embedded in textiles. They provide a plethora of multimodal raw data that needs to be processed and analyzed, yielding actionable insights out of them. Sensors provide immense amounts of data at very high rates. Making sense of such timed streams of information is extremely complex, and new algorithms are probably required. These algorithms should effectively fuse data coming from different sensors to acquire a better understanding of the user and her situation. We know, based on previous attempts, that it is extremely complicated to build such algorithms to robustly run in the wild. The final stage after sensing and understanding, reaction, refers to the reactivity of the environment (the delivered multimedia content, the surroundings of the user, or the network used for communication). This piece of the puzzle provides the adaptation techniques that will improve the quality of the user experience.