MING-I: a distributed interactive multimedia document development mechanism

The next generation of interactive multimedia documents can contain both static media, e.g., text, graph, image, and continuous media, e.g., audio and video, and can provide user interactions in distributed environments. However, the temporal information of multimedia documents cannot be described using traditional document structures, e.g., Open Document Architecture (ODA) and Standard Generalized Mark-up Language (SGML); the continuous transmission of media units also raises some new synchronization problems, which have not been met before, for processing user interactions. Thus, developing a distributed interactive multimedia document system should resolve the issues of document model, presentation control architecture, and control scheme. In this paper, we (i) propose a new multimedia document model that contains the logical structure, the layout structure, and the temporal structure to formally describe multimedia documents, and (ii) point out main interaction-based synchronization problems, and propose a control architecture and a token-based control scheme to solve these interaction-based synchronization problems. Based on the proposed document model, control architecture, and control scheme, a distributed interactive multimedia document development mechanism, which is called MING-I, is developed on SUN workstations.     

Design and implementation of a multimedia CSCW platform

In this paper, a generic four-layer framework for computer supported cooperative work (CSCW) is proposed. With clear separation of functionalities in each layer of the framework, application developers could implement their specific CSCW applications easily. Based on the four-layer framework, the architecture of a general purpose CSCW platform is designed and implemented with the necessary functionalities to help the collaborations among group members. To provide the system with more features and user friendliness, multimedia processing and transmission capabilities for audio and video are incorporated into the platform. Features like application-sharing, group decision support, multimedia mail transmission, etc., are well implemented in the platform. Moreover, an efficient network transport protocol, called VXTP, is designed and implemented to avoid the transmission overhead of conventional TCP. The performance measurements using both VXTP and TCP are given in the paper for comparison. Lastly, compared with other CSCW systems, our design achieves greater flexibility and portability by adopting the generality philosophy in the framework.     

基于JAIN SIP的多媒体会议用户代理设计

会议初始化（SIP）协议是下一代网络应用层的核心控制协议。JAIN SIP是SUN公司用于实现SIP应用而提供的一套标准Java接口。文章提出了一个基于JAIN SIP的多媒体会议用户代理的框架，对每一层的功能模块进行了详细分析，包括JAIN SIP协议栈的体系结构、用户代理控制部分信令（创建会议、加入会议、删除会议等）、媒体信息处理等。用JAIN SIP开发基于SIP协议的应用不仅可以提高开发效率，而且将使应用程序获得较高的可靠性和较好的移植性。     

The IP multimedia domain: service architecture for the delivery of voice, data, and next generation multimedia applications

The IP Multimedia Subsystem (IMS) is defined by the 3 ^rd Generation Partnership Project (3GPP) as a new core network domain. IMS provides a service control platform that allows creation of new multimedia and multi-session applications utilizing wireless and wireline transport capabilities. In this paper we will cover the concepts and standards defining IMS and review the network architecture from a mobile perspective. We will see how IMS interacts with the Packet Switched Domain (e.g. Wireless LAN, GPRS, and UMTS networks), the Internet, and application services. Then we will examine the key IMS capabilities and show how they can be combined to create new mobile IP services. Finally, we present a software architecture, which is enabled by IMS and allows development of unique applications (with multimedia/multi-session functionality, single/multi-user, service to user). The software architecture is illustrated by an example of a prototype application. This work was done when the author was with Siemens Communications Inc., Boca Raton, FL     

Meet In the Middle Cross-Layer Adaptation for Audiovisual Content Delivery

This paper describes a new architecture and implementation of an adaptive streaming system (e.g., television over IP, video on demand) based on cross-layer interactions. At the center of the proposed architecture is the meet in the middle concept involving both bottom-up and top-down cross layer interactions. Each streaming session is entirely controlled at the RTP layer where we maintain a rich context that centralizes the collection of (i) instantaneous network conditions measured at the underlying layers (i.e.: link, network, and transport layers) and (ii) user- and terminal-triggered events that impose new real-time QoS adaptation strategies. Thus, each active multimedia session is tied to a broad range of parameters, which enable it to coordinate the QoS adaptation throughout the protocol layers and thus eliminating the overhead and preventing counter-productiveness among separate mechanisms implemented at different layers. The MPEG-21 framework is used to provide a common support for implementing and managing the end-to-end QoS of audio/video streams. Performance evaluations using peak signal to noise ratio (PSNR) and structural similarity index (SSIM) objective video quality metrics show the benefits of using the proposed Meet In the Middle cross-layer design compared to traditional media delivery approaches.     

The IP multimedia domain: service architecture for the delivery of voice, data, and next generation multimedia applications

The IP Multimedia Subsystem (IMS) is defined by the 3^rd Generation Partnership Project (3GPP) as a new core network domain. IMS provides a service control platform that allows creation of new multimedia and multi-session applications utilizing wireless and wireline transport capabilities. In this paper we will cover the concepts and standards defining IMS and review the network architecture from a mobile perspective. We will see how IMS interacts with the Packet Switched Domain (e.g. Wireless LAN, GPRS, and UMTS networks), the Internet, and application services. Then we will examine the key IMS capabilities and show how they can be combined to create new mobile IP services. Finally, we present a software architecture, which is enabled by IMS and allows development of unique applications (with multimedia/multi-session functionality, single/multi-user, service to user). The software architecture is illustrated by an example of a prototype application. This work was done when the author was with Siemens Communications Inc., Boca Raton, FL     

The “Always Best Packet Switching” architecture for SIP-based mobile multimedia services

This paper presents a distributed architecture for the provision of seamless and responsive mobile multimedia services. This architecture allows its user applications to use concurrently all the wireless network interface cards (NICs) a mobile terminal is equipped with. In particular, as mobile multimedia services are usually implemented using the UDP protocol, our architecture enables the transmission of each UDP datagram through the “most suitable” (e.g. most responsive, least loaded) NIC among those available at the time a datagram is transmitted. We term this operating mode of our architecture Always Best Packet Switching (ABPS). ABPS enables the use of policies for load balancing and recovery purposes. In essence, the architecture we propose consists of the following two principal components: (i) a fixed proxy server, which acts as a relay for the mobile node and enables communications from/to this node regardless of possible firewalls and NAT systems, and (ii) a proxy client running in the mobile node responsible for maintaining a multi-path tunnel, constructed out of all the node's NICs, with the above mentioned fixed proxy server. We show how the architecture supports multimedia applications based on the SIP and RTP/RTCP protocols, and avoids the typical delays introduced by the two way message/response handshake of the SIP signaling protocol. Experimental results originated from the implementation of a VoIP application on top of the architecture we propose show the effectiveness of our approach.     

Innovative directions in self-organized distributed multimedia systems

The way by which multimedia contents are produced, delivered across networks, and consumed by intended users have shifted significantly during the past 10 years. In this paper we postulate that, in the near future, flexible and self-organizing facilities will play a dominating role in distributed multimedia systems. We discuss how such systems can be designed, using a three-layer (sensor, distribution, and user layer) architecture, SOMA (Self Organizing Multimedia Architecture), as an example. We also present innovative directions in three main aspects of self-organized multimedia systems: (i) the self-organizing aspects of multimedia user communities, e.g., the wisdom, intentions, and needs of users; (ii) a fresh look at video streams that treat them as a collection of units that can be composed taking user and network aspects into account; and (iii) new delivery paradigms and how self-organization and multimedia delivery can be combined.     

SCTP: a proposed standard for robust Internet data transport

The stream control transmission protocol (SCTP) is an evolving general purpose Internet transport protocol designed to bridge the gap between TCP and UDP. SCTP evolved from a telephony signaling protocol for IP networks and is now a proposed standard with the Internet Engineering Task Force. Like TCP, SCTP provides a reliable, full-duplex connection and mechanisms to control network congestion. However, SCTP expands transport layer possibilities beyond TCP and UDP, offering new delivery options that are particularly desirable for telephony signaling and multimedia applications.     

A novel network mobility handoff scheme using SIP and SCTP for multimedia applications

In a heterogeneous wireless environment, seamless mobility is the basis of network support with which mobile users who roam between or among various wireless access networks are able to fully enjoy uninterrupted wireless services. When users are in a mass transportation vehicle, e.g., a bus or a train that provides network service, the vehicle can be regarded as a network which is serving users as it moves from one location to another. The movement of a network is called network mobility (NEMO). The network mobility protocol based on Mobile IPv6 as proposed by the Internet Engineering Task Force (IETF) in 2005 has some fundamental drawbacks, such as header overhead and the pinball problem. In this paper, we propose a novel hybrid method for network mobility called Hybrid-NEMO, which provides a soft handoff scheme at the transport layer basically utilizing SIP and SCTP protocols to ensure a lossless packet-transmission environment and less handoff-delay variation, which are critical in providing QoS voice and multimedia applications. Experimental validation and performance evaluation were also conducted in this study.     

An autonomic architecture for optimizing QoE in multimedia access networks

The recent emergence of multimedia services, such as Broadcast TV and Video on Demand over traditional twisted pair access networks, has complicated the network management in order to guarantee a decent Quality of Experience (QoE) for each user. The huge amount of services and the wide variety of service specifics require a QoE management on a per-user and per-service basis. This complexity can be tackled through the design of an autonomic QoE management architecture. In this article, the Knowledge Plane is presented as an autonomic layer that optimizes the QoE in multimedia access networks from the service originator to the user. It autonomously detects network problems, e.g. a congested link, bit errors on a link, etc. and determines an appropriate corrective action, e.g. switching to a lower bit rate video, adding an appropriate number of FEC packets, etc. The generic Knowledge Plane architecture is discussed, incorporating the triple design goal of an autonomic, generic and scalable architecture. The viability of an implementation using neural networks is investigated, by comparing it with a reasoner based on analytical equations. Performance results are presented of both reasoners in terms of both QoS and QoE metrics.     

Ethersim: a simulator for application-level performance modeling of wireless and mobile ATM networks

The paper describes Ethersim, a simulation tool to model and study the performance of multimedia-oriented integrated service ATM networks with mobile hosts and wireless links. The key motivation behind Ethersim is to study the application-level impact of host mobility and wireless channels. Ethersim has a discrete event based simulator core and incorporates models of user applications and transport, network and MAC layer protocols. It provides the capability to specify a cellular wireless ATM network topology and host mobility patterns. The software architecture of Ethersim employs five special entities: an air module, a map, a mover, mobile hosts, and basestations. We also present case-studies of using Ethersim to model and study the interaction of transport layer, connection rerouting protocol, and radio characteristics in the SWAN [P. Agrawal, A. Asthana, M. Cravatts, E. Hyden, P. Krzyzanowski, P. Mishra, B. Narendran, M. Srivastava, J. Trotter, SWAN: A Mobile Multimedia Wireless Network, in: IEEE Personal Commun. Mag., April 1996] mobile and wireless ATM based multimedia network.     

构建Ad Hoc网络跨层协议交互机制

Ad Hoc网络中无线信道的时变特性和网络拓扑的不断变化导致了链路的频繁断续,使得数据包大量丢失.这就要求高层应用协议能够快速访问底层传输协议甚至信道的状态信息,各层协议之间能够根据需要互相交换信息,迅速作出决策,提高分组接收成功率.基于QualNet网络仿真器设计并实现了一种网络协议栈跨层交互的机制.结果表明,利用该机制,能够快速建立跨层协议的交互,使得高层协议及时获取低层协议的有用信息,从而可根据信道的状况动态调节网络协议设置,提高分组发送成功率,优化网络传输的性能.     

对SIP协议的研究及其基本功能的一个实例

会话初始协议SIP是IETF的多媒体数据和控制体系的一部分,它是一个基于应用层的控制协议,用来邀请用户参加多媒体会议。SIP不依附于任何一种特定的会议控制协议。SIP是被设计成独立于低层的传输协议的并且可以用一些额外的功能来进行扩展。本文介绍了SIP的工作原理和一些操作并且给出了一人两个用户SIP进行初始化的简单例子。     

Development of virtual data acquisition systems based on multimedia internetworking

This paper proposes a novel approach centered on multimedia internetworking for the development of Distributed Virtual Instruments (DVI). Multimedia internetworking refers to network infrastructures, protocols, models, applications and techniques being currently deployed over the Internet to support multimedia applications, e.g., videoconferencing, video-on-demand, shared workspaces. It is applied to broaden the concept of virtual instrument and enable new measurement patterns leveraging efficiency and interactivity. A DVI is a virtual instrument split into possibly multiple and independent parts, sender and receiver, which are linked by real-time continuous media and control streams. Senders and receivers are built by using open, composable and modular components based on a time sensitive actor framework and glued by multimedia middleware. A prototype is described to demonstrate the potential and the benefits of the proposed approach.     

Scheduling and data layout policies for a near-line multimedia storage architecture

Recent advances in computer technologies have made it feasible to provide multimedia services, such as news distribution and entertainment, via high-bandwidth networks. The storage and retrieval of large multimedia objects (e.g., video) becomes a major design issue of the multimedia information system. While most other works on multimedia storage servers assume an on-line disk storage system, we consider a two-tier storage architecture with a robotic tape library as the vast near-line storage and an on-line disk system as the front-line storage. Magnetic tapes are cheaper, more robust, and have a larger capacity; hence, they are more cost effective for large scale storage systems (e.g., video-on-demand (VOD) systems may store tens of thousands of videos). We study in detail the design issues of the tape subsystem and propose some novel tape-scheduling algorithms which give faster response and require less disk buffer space. We also study the disk-striping policy and the data layout on the tape cartridge in order to fully utilize the throughput of the robotic tape system and to minimize the on-line disk storage space.     

Resource allocation for multimedia streaming over the Internet

This paper addresses the resource allocation problem for multiple media streaming over the Internet. First, we present an end-to-end transport architecture for multimedia streaming over the Internet. Second, we propose a new multimedia streaming TCP-friendly protocol (MSTFP), which combines forward estimation of network conditions with information feedback control to optimally track the network conditions. Third, we propose a novel resource allocation scheme to adapt media rate to the estimated network bandwidth using each media's rate-distortion function under various network conditions. By dynamically allocating resources according to network status and media characteristics, we improve the end-to-end quality of services (QoS). Simulation results demonstrate the effectiveness of our proposed schemes     

Distributed computing architecture for effective management of multimedia streams on DPE

We study a networking architecture model that is built on a distributed processing environment (DPE) for multimedia services suitable for high speed transport networks such as ATM networks. In this architecture, the applications are deployed as units of software building blocks. Each building block provides a layered view for the effective management and control of the multimedia network resources and services according to the concept of telecommunications management network (TMN) and telecommunications information networking architecture (TINA). For the purpose of flexible service provision to users and effective service introduction by service providers, this architecture proposes the adoption of ad hoc service building blocks such as a video on demand building block and a CSCW building block that have interactions with a general purpose building block. This paper also proposes a naming structure for the management of user profiles and session profiles using a directory service system, and an effective control model for multimedia logical device objects using a stream process approach. The proposed model is implemented on a DPE platform that provides various transparencies, ANSAware.     

Priority-based rate control for service differentiation and congestion control in wireless multimedia sensor networks

In wireless multimedia sensor networks (WMSNs) a sensor node may have different types of sensor which gather different kinds of data. To support quality of service (QoS) requirements for multimedia applications having a reliable and fair transport protocol is necessary. One of the main objectives of the transport layer in WMSNs is congestion control. We observe that the information provided may have different levels of importance and argue that sensor networks should be willing to spend more resources in disseminating packets carrying more important information. Some applications of WMSNs may need to send real time traffic toward the sink node. This real time traffic requires low latency and high reliability so that immediate remedial and defensive actions can be taken when needed. Therefore, similar to wired networks, service differentiation in wireless sensor networks is also an important issue. We present a priority-based rate control mechanism for congestion control and service differentiation in WMSNs. We distinguish high priority real time traffic from low priority non-real time traffic, and service the input traffic based on its priority. Simulation results confirm the superior performance of the proposed model with respect to delays, delay variation and loss probability.     

Extending multimedia languages to support multimodal user interactions

Historically, the Multimedia community research has focused on output modalities, through studies on timing and multimedia processing. The Multimodal Interaction community, on the other hand, has focused on user-generated modalities, through studies on Multimodal User Interfaces (MUI). In this paper, aiming to assist the development of multimedia applications with MUIs, we propose the integration of concepts from those two communities in a unique high-level programming framework. The framework integrates user modalities —both user-generated (e.g., speech, gestures) and user-consumed (e.g., audiovisual, haptic)— in declarative programming languages for the specification of interactive multimedia applications. To illustrate our approach, we instantiate the framework in the NCL (Nested Context Language) multimedia language. NCL is the declarative language for developing interactive applications for Brazilian Digital TV and an ITU-T Recommendation for IPTV services. To help evaluate our approach, we discuss a usage scenario and implement it as an NCL application extended with the proposed multimodal features. Also, we compare the expressiveness of the multimodal NCL against existing multimedia and multimodal languages, for both input and output modalities.