An end-to-end tool chain for Sensory Experience based on MPEG-V

This paper provides an overview of our research conducted in the area of Sensory Experience including our implementations using MPEG-V Part 3 entitled “Sensory Information”. MPEG-V Part 3 introduces Sensory Experience as a tool to increase the Quality of Experience by annotating traditional multimedia data with sensory effects. These sensory effects are rendered on special devices like fans, vibration chairs, ambient lights, scent disposers, water sprayers, or heating/cooling devices stimulating senses beyond the traditional ones. The paper's main focus is on the end-to-end aspects including the generation, transmission, and synchronized rendering of sensory effects with the traditional multimedia data taking movie clips as an example. Therefore, we present in this paper an open source tool chain that provides a complete end-to-end sensory effect generation and consumption framework. Furthermore, we summarize results from various subjective quality assessments conducted in this area. Finally, we point out research challenges that may encourage further research within this emerging domain.