End-to-end framework for 4-D broadcasting based on MPEG-V standard

Advances in media technologies brought so-called 4-D movies to your neighbour. The 4-D movies represent movies (preferably in 3-D) with additional sensory effects such as motion chairs and wind effects. We present an end-to-end framework to provide sensory effects for home theaters based on MPEG-V standard. The key technologies of the 4-D broadcasting framework, required to provide sensory effect through home theaters consist of creating sensory effects synchronized with audiovisual contents, delivering the sensory effects with content to home theater devices, recognizing home devices for rendering sensory effects, and rendering contents through home theater system with connected sensory devices in a synchronized way. This paper proposes end-to-end technology required, in addition to the conventional broadcasting framework, based on international standard of MPEG-V, specifically Part 2 to describe capabilities of the home sensory devices, Part 3 to provide contents with sensory effects, and Part 5 to command the sensory devices among other parts of MPEG-V standard.     

Assessing the quality of sensory experience for multimedia presentations

This paper introduces the concept of sensory experience by utilizing sensory effects such as wind or lighting as another dimension which contributes to the quality of the user experience. In particular, we utilize a representation format for sensory effects that are attached to traditional multimedia resources such as audio, video, and image contents. Sensory effects (e.g., wind, lighting, explosion, heat, cold) are rendered on special devices (e.g., fans, ambient lights, motion chair, air condition) in synchronization with the traditional multimedia resources and shall stimulate other senses than audition and vision (e.g., mechanoreception, equilibrioception, thermoreception), with the intention to increase the users Quality of Experience (QoE). In particular, the paper provides a comprehensive introduction into the concept of sensory experience, its assessment in terms of the QoE, and related standardization and implementation efforts. Finally, we will highlight open issues and research challenges including future work.     

Authoring multisensorial content

In this paper, a sensory effect authoring tool is introduced to effectively generate sensory effect metadata for sensible media. Sensible media manifests sensory effects in the media to the real world by stimulating human sensory organs. It is emerging as a new type of media to enhance the user’s immersion. For the successful deployment of the sensible media industry, an effective authoring tool is required to provide easy authoring of sensory effect metadata. The actual implementation of the authoring tool and its development based on MPEG-V Part 3 (ISO/IEC 23005-3) International Standard is presented in this paper.     

FreeBW-RPL: A New RPL Protocol Objective Function for Internet of Multimedia Things

Most of the current research has been restricted to scalar sensor data based IoT applications. However, today’s research and development activities rely on multimedia-based services and applications while this kind of applications has several requirements in terms of storage, bandwidth, latency, etc. Furthermore, protocols dedicated to IoT applications have proved their weakness in multimedia environments. Hence, a new paradigm called internet of multimedia things (IoMT) has been proposed to fulfill the requirements of multimedia applications. In this paradigm different multimedia things can interact and cooperate with each other over the Internet. Moreover, IETF ROLL working group standardized an IPv6 routing protocol for low-power and lossy networks (RPL for LLNs) for resource constrained devices. In this paper, we propose an enhanced version of RPL for IoMT called free bandwidth (FreeBW)-RPL in which the sensed data is essentially provided by multimedia devices. FreeBW-RPL protocol proposes a new objective function called FreeBW that takes the FreeBW calculation in the network layer. We set the QoS routing challenge as the amount of the bandwidth while selecting the routing path in order to measure the maximum FreeBW so as to deliver better performance of the multimedia applications. Simulations have been conducted over COOJA simulator. The obtained results proved that our proposal outperforms the basic ones in terms of end-to-end delay, throughput, packet delivery ratio and energy consumption and provides better performance than other protocols.

     

Construction of a haptic-enabled broadcasting system based on the MPEG-V standard

With rapid developments in communications technology and digital multimedia, there has been increasing demand in recent years for realistic broadcasting technology beyond conventional audio-visual media. In response to this demand, this paper presents an example of the construction of a haptic-enabled broadcasting system based on the MPEG-V standard that was established recently. The construction processes of the proposed haptic-enabled broadcasting system include various types of media acquisition, haptic content creation by modeling and authoring, transmission, rendering, and interaction. This paper illustrates the data flow within the system, from the creation of haptic contents to the rendering of these contents to the end user, and explains a method of building the system with the MPEG-V standard. The constructed haptic-enabled broadcasting system allows users to have more immersive interaction with the synthesized haptic multimedia, which is closely synchronized with audio-visual data.     

Virtual world control system using sensed information and adaptation engine

The sensed information (e.g., temperature, humidity, light intensity, gas intensity) obtained from a user's vicinity plays an important role in the generation of environmental changes, the main character's reactions, and in storyline changes in the virtual world. In this paper, a method is proposed to transform sensed information from real world to standardized XML instances and to control virtual world objects. The sensed information from the real world is transformed into XML instances standardized by MPEG-V Part 5 (Data formats for Interaction Device) schemes and then delivered to a Real-to-Virtual (RV) adaptation engine. The engine generates another XML instance standardized by MPEG-V Part 4 (Virtual world object characteristics) that describes the control information of avatars and objects in the virtual world. The system is demonstrated by a real-world mock-up equipped with a sensor set and a virtual space realized in Second Life.     

User-aware object-based video transmission over the next generation Internet

In this paper, we present a new user-aware adaptive object-based video transmission approach to heterogeneous users over the next generation Internet. Firstly, we describe a new transport framework for complex multimedia applications over the next generation Internet, which provides differentiation functionality within one IP session as well as among different IP sessions. It includes application-aware intelligent resource control at the edge of the network, fast transcoding and signaling in the network. Secondly, we propose a new bitstream classification, prioritization and packetization scheme in which different types of data such as shape, motion and texture are reassembled, assigned to different priority classes, and packetized separately based on their priorities. Thirdly, we present a simple but effective mechanism of object-based dynamic rate control and adaptation by selectively dropping packets in conjunction with differentiated services (Diffserv) to minimize the end-to-end quality distortion. Finally, we perform the queuing analysis for our mechanism and explore how to extend our approach to the multicast case. Experimental results demonstrate effectiveness of our proposed approach.     

Memory-access-aware data structure transformations for embedded software with dynamic data accesses

Embedded systems are evolving from traditional, stand-alone devices to devices that participate in Internet activity. The days of simple, manifest embedded software [e.g. a simple finite-impulse response (FIR) algorithm on a digital signal processor (DSP] are over. Complex, nonmanifest code, executed on a variety of embedded platforms in a distributed manner, characterizes next generation embedded software. One dominant niche, which we concentrate on, is embedded, multimedia software. The need is present to map large scale, dynamic, multimedia software onto an embedded system in a systematic and highly optimized manner. The objective of this paper is to introduce high-level, systematically applicable, data structure transformations and to show in detail the practical feasibility of our optimizations on three real-life multimedia case studies. We derive Pareto tradeoff points in terms of accesses versus memory footprint and obtain significant gains in execution time and power consumption with respect to the initial implementation choices. Our approach is a first step to systematically applying high-level data structure transformations in the context of memory-efficient and low-power multimedia systems.     

Connecting users to virtual worlds within MPEG-V standardization

Virtual world such as Second life and 3D internet/broadcasting services have been increasingly popular. A life-scale virtual world presentation and the intuitive interaction between the users and the virtual worlds would provide more natural and immersive experience for users. The emergence of novel interaction technologies, such as facial-expression/body-motion tracking and remote interaction for virtual object manipulation, could be used to provide a strong connection between users in the real world and avatars in the virtual world. For the wide acceptance and the use of the virtual world, various types of novel interaction devices should have a unified interaction format between the real world and the virtual world. Thus, MPEG-V Media Context and Control (ISO/IEC 23005) standardizes such connecting information. The paper provides an overview and its usage example of MPEG-V from the real world to the virtual world (R2V) on interfaces for controlling avatars and virtual objects in the virtual world by the real world devices. In particular, we investigate how the MPEG-V framework can be applied for the facial animation and hand-based 3D manipulation using intelligent camera. In addition, in order to intuitively manipulate objects in a 3D virtual environment, we present two interaction techniques using motion sensors such as a two-handed spatial 3D interaction approach and a gesture-based interaction approach.     

Multimedia friendly home wireless LANs

Home wireless local area networks (LANs) will carry diverse multimedia applications such as data, video, voice, and time sensitive control information. To enable quality delivery of such applications, these networks should incorporate an efficient quality of service (QoS) support mechanism. However, existing home wireless LANs do not provide support for multimedia applications. In this paper, we introduce our software based solution that provides QoS support for voice, video and data while using existing wireless home networks. The experimental results we have obtained provide evidence that our solution provides QoS support for multimedia applications.     

Intelligent and Pervasive Multimedia Systems

Recent advances in pervasive computing and the proliferation of multimedia-capable devices have stimulated the development of intelligent and pervasive multimedia applications. This special issue provides excellent coverage of this area, including interactive multimedia education, quality control and personalization of multimedia services, peer-to-peer multimedia streaming, mobile TV, and VoIP systems.     

Multimedia group synchronization protocols for integrated servicesnetworks

Unlike traditional data traffic, real time multimedia traffic requires synchronization. Temporal relationships among media must be maintained. Yet delay jitter and the absence of a global clock may disrupt these temporal relationships. This paper introduces new group synchronization protocols for real-time, multimedia applications, including teleconference, teleorchestration and multimedia on demand services. The proposed protocols achieve synchronization for all configurations (one-to-one, one-to-many, many-to-one, and many-to-many), and does so without prior knowledge of the end-to-end delay distribution, or the distribution of the clock drift. The only a-priori knowledge the protocols require is an upper bound on the end-to-end delay. The paper concludes with simulation experiments showing that the protocols work effectively in both LAN and WAN environments     

Design, implementation, and experiences of the OMEGA end-pointarchitecture

The OMEGA architecture provides end-to-end quality-of-service (QoS) guarantees for distributed applications. QoS parameters are translated between application and network requirements by the QoS broker, thus integrating media and network QoS management into a single entity. Admission control uses a schedulability test derived from application requirements. A novel task priority and precedence-based scheme is used to represent complex application requirements and ensure correct feasible schedules. A prototype of OMEGA has been implemented using workstations connected by a 155 Mb/s dedicated ATM local-area network (LAN). To simplify implementation, we assumed networked multimedia application with periodic media streams, specifically a master/slave telerobotics application. This application employs media with highly diverse QoS requirements (e.g., interarrival times, loss rate, and bandwidth) and therefore provides a good platform for testing how closely one can achieve QoS guarantees with workstation hosts and cell-switching. Experience with this implementation has helped to identify new challenges to extending these techniques to a larger domain of applications and systems, and suggests promising new research questions     

Managing IP traffic in radio access networks of next-generation mobile systems

The strong research, standardization, and development activity currently focusing on the enabling technologies for the transition to a new generation of mobile networks represent a great opportunity to accomplish the goal of IP multimedia service provisioning to mobile users. This will contribute to the migration toward an all-IP platform with guaranteed end-to-end QoS provisioning, and will definitively fuel the deployment of new multimedia mobile applications and the further development of multimedia ubiquitous communications. This objective is not easily achievable, mainly due to the difficulties in handling IP multimedia applications in a mobile environment with a guaranteed QoS level. To contribute to this issue, in this article we discuss an evolutionary scenario from the current UMTS system to a next-generation QoS-aware mobile platform based on IP as the transport solution in the radio access network, UTRAN. Furthermore, we propose to fit the IntServ-over-DiffServ approach, in order to best manage the QoS level inside the UTRAN.     

一种异构网络多媒体业务QoS类弹性映射方法

 多媒体业务在异构网络传输过程中,由于现有的QoS(Quality of Service)类映射方法存在灵活性不足的问题,从而降低了系统端到端效能。针对这个问题,该文在深入分析当前的QoS类映射方法基础上,结合用户QoE (Quality of Experience)特点,借助于网络微积分理论,构建了QoS类映射的数学分析模型,并进行了理论分析。基于该数学分析模型,该文从用户QoE角度提出了具有弹性的QoS类映射方法(Elastic QoS Class Mapping Method, EQCMM),该方法根据当前网络资源的使用情况,通过灵活地调整QoS类映射,充分利用网络现有资源,提高了端到端带宽资源的利用率,改善了系统端到端的效能。最后,通过仿真验证了该方法的有效性。     

一种基于蚁群优化的多媒体传感器网络服务感知路由算法

本文针对多媒体传感器网络中三类基本服务:异常事件告警服务、信息查询服务以及流媒体查询服务,分析了不同服务的QoS需求,并抽象出多媒体传感器网络QoS路由模型.进而,利用改进的蚁群算法,设计了一个基于蚁群优化的服务感知路由算法ASAR (Ant-colony optimization based Service Aware Routing).本文设计服务感知的路由协议,旨在依据各类媒体数据包不同的QoS需求,选择相应的路由以合理利用全网资源、提高网络性能.最后,我们通过NS2仿真表明了较其他路由算法,本文提出的方法能够为多媒体传感器网络的多种服务提供更好的QoS保障,同时,比传统的蚁群算法具有更好的收敛性.     

Cross-layer design proposals for wireless mobile networks: a survey and taxonomy

Third-generation (3G) and beyond 3G mobile communication systems must provide interoperability with the Internet, increase throughput for mobile devices, and optimize their operation for multimedia applications. The limited ability of traditional layered architectures to exploit the unique nature of wireless communication has fostered the introduction of cross-layer design solutions that allow optimized operation for mobile devices in the modern heterogeneous wireless environment. In this article we present the major cross-layer design solutions that handle such problems, and discuss cross-layer implementations with a focus on functional entities that support cross-layer processes and the respective signaling. In addition, we consider the associated architectural complexity and communication overhead they introduce. Furthermore, we point out the major open technical challenges in the cross-layer design research area. Finally, we conclude our article with a summary of cross-layer approaches developed thus far and provide directions for future work.     

Securing the wireless internet

Internet-enabled wireless devices continue to proliferate and are expected to surpass traditional Internet clients in the near future. This has opened up exciting new opportunities in the mobile e-commerce market. However, data security and privacy remain major concerns in the current generation of "wireless Web" offerings. All such offerings today use a security architecture that lacks end-to-end security. This unfortunate choice is driven by perceived inadequacies of standard Internet security protocols like SSL on less capable CPUs and low-bandwidth wireless links. This article presents our experiences in implementing and using standard security mechanisms and protocols on small wireless devices. We have created new classes for the Java 2 Micro-Edition platform that offer fundamental cryptographic operations such as message digests and ciphers as well as higher level security protocols like SSL. Our results show that SSL is a practical solution for ensuring end-to-end security of wireless Internet transactions even within today's technological constraints     

Cross-Layer Design for Data Accessibility in Mobile Ad Hoc Networks

Mobile ad hoc networks (MANET) are becoming an integral part of the ubiquitouscomputing and communication environment, providing new infrastructure formultimedia applications such as video phone, multimedia-on-demand, and others.In order to access multimedia information in MANET, Quality of Service (QoS)needs to be considered, such as high success rate to access multimedia data,bounded end-to-end delay, and others. In this paper, we present a dataaccessibility service for a group of mobile users to access desired data withhigh success rate. This accessibility service is only possible if we utilizeadvanced data advertising, lookup and replication services, as well as a novelpredictive location-based QoS routing protocol in an integrated fashion. Usingcross-layer design, we illustrate how the QoS routing protocol assists dataadvertising, lookup and replication services to achieve high data accesssuccess rate. Simulation results have shown that our design is successful ina dynamic MANET.