Real-Time Multimedia Synchronization based on Delay Offset and Playout Rate Adjustment

We investigate here an efficient and flexible multimedia synchronization method that can be applied to intramedia and intermedia synchronization in a consistent manner. Because the synchronization requirements can vary widely according to the multimedia application in use, it is necessary to control the individual synchronization (delay sensitivity, error tolerance etc.) for each media separately. In addition, the synchronization method should handle the performance variations in the underlying network and systems. We propose an adaptive synchronization scheme based on the delay offset and playout rate adjustment that can match the application's varying synchronization requirements effectively. The model, algorithm and operation of the proposed method are presented in this paper.     

An adaptive multiplexing algorithm of delay-sensitive multiple VBR-coded bit streams

This paper describes a novel multiplexing scheme for delay-sensitive multiple VBR-coded bit streams in live multimedia service offered to high-speed networks. The primary goal of multiplexing in this paper is to keep the delay limits of each bit stream and to enhance network resource utilization. For this end, this paper presents an adaptive control scheme which does not cause violation of any delay constraints inherent to each bit stream. The scheme is based on the assumption that the recent behavior of each bit stream has a high correlation with near-term future behavior. Such a property is used to make as flat as possible both temporal averaging on a stream-by-stream basis and spatial averaging on multiple VBR-coded bit streams. Through several computer simulations, it is shown that the proposed scheme can effectively reduce the peak rate, the coefficient of variation, and the effective bandwidth of the multiplexed transmission rate.     

Multimedia streaming gateway with jitter detection

This paper investigates a novel active buffer management scheme, "Jitter Detection" (JD) for gateway-based congestion control to stream multimedia traffics in packet-switched networks. The quality of multimedia presentation can be greatly degraded due to network delay variation or jitter when transported over a packet-switched network. Jitter degrades the timing relationship among packets in a single media stream and between packets from different media streams and, hence, creates multimedia synchronization problems. Moreover, too much jitter will also degrade the performance of the streaming buffer in the client. Packets received by the client will be rendered useless if they have accumulated enough jitter. The proposed active buffer management scheme will improve the quality of service in multimedia networking by detecting and discarding packets that accumulated enough jitter, such as to maintain a high bandwidth for packets within the multimedia stream's jitter tolerance. Simulation results have shown that the proposed scheme can effectively lower the average received packet jitter and increase the goodput of the received packets when compared to random early detection (RED) and DropTail used in gateway-based congestion control. Furthermore, simulation results have also revealed that the proposed scheme can maintain the same TCP friendliness when compared to that of RED and DropTail used for multimedia streams.     

分布式多媒体通信系统中音频和视频同步算法

文章简要介绍了分布式多媒体通信系统中实时多媒体同步问题,以及自适应同步算法的特点和良好的自适应能力：可以适应各种网络变化,各种延迟特性,并利用该算法实现音频和视频内外同步。     

An adaptive protocol for synchronizing media streams

Stream synchronization is widely regarded as a fundamental problem in the field of multimedia systems. Solutions to this problem can be divided into adaptive and rigid mechanisms. While rigid mechanisms are based on worst case assumptions, adaptive ones monitor the underlying network and are able to adapt themselves to changing network conditions. In this paper, we will present an adaptive stream synchronization protocol. This protocol supports any kind of distribution of the sources and sinks of the streams to be synchronized. It is based on a buffer-level control mechanism, allowing immediate corrections when the danger of a buffer overflow or underflow is recognized. Moreover, the proposed protocol is flexible enough to support a wide variety of synchronization policies, which can be dynamically changed while synchronization is in progress. Finally, the message overhead of this protocol is low, because control messages are only exchanged when network conditions change.     

基于RSVP和WFQ实现多媒体同步的模型

同步是在因特网中实现多媒体实时通信的关键问题。现有的网络元素（NE）的调度机制不能保持相关流的流间同步关系。结合RSVP和WFQ调度机制提出一个多媒体实时通信的模型方案,在同一端到端路径为多个相关流预留资源,沿路各网络元素对这些相关流进行相关调度,保持相关流数据单元间稳定的同步关系,从而大大减少目的端相关流数据单元的时差和抖动,方便目的端进行同步处理。     

嵌入式VOD码流传输同步优化方案

嵌入式视频点播在码流传输时流畅度、同步效果及资源利用率等方面存在冲突。为此,提出一种优化的码流传输同步方案。采用音视频分割同步发送算法,实现码流同步发送,使用音频自适应抖动缓冲算法、视频去抖动算法完成音视频去抖动,并降低丢包率,在解码前进行音视频2次同步,利用基于缓冲区的和式增加积式减少控制算法和实时流传输协议实现网络拥塞控制。测试结果表明,该方案的流内同步时延在100 ms内,流间同步时延在125 ms内,能优化码流流畅度和系统资源利用率。     

一种新的媒体内同步控制算法

提出了一种新的媒体同步播放方案，该方案是基于发送方的媒体同步控制，根据缓冲区的占用情况来检测失步，并将其反馈给发送方，发送方利用给出的控制函数对发送帧率进行调整，来保持接收方媒体的同步播放，调节算法简单易实现，实验表明该方案能够处理由于网络传输时延特性变化引起的失步，与其它反馈控制方案相比具有更低的数据丢失率，从而使数据表现更为平滑连续。     

一种面向多对象多媒体应用的同步控制方法

多对象多媒体应用允许多路媒体同时呈现,其同步控制比较复杂,可分为传输过程中与呈现过程中的同步。然而现有的同步方法忽略了媒体在最后呈现过程中的同步,也没有考虑各媒体对象之间的同步要求差异。针对以上问题,为视频点播提出了一种在应用层上控制多对象多媒体同步呈现质量的方法。在采用多级缓冲门限实现媒体内同步反馈控制的基础上,依据主要媒体对象的呈现时间戳,在每一个呈现场景内调整次要媒体对象的呈现速率,从而实现媒体间的同步呈现。实验结果表明该方案能够有效地提高同步质量,且媒体间的同步性能平均提高28.45%。     

Delay compensation protocols for synchronization of multimedia datastreams

Temporal synchronization of various data streams in multimedia information supporting voice, video, graphics and text, that are exchanged between users over a high speed network are discussed. During delivery of such data, maintaining the required association between data units across various streams in real-time is necessary to sustain quality of service in the presence of data loss and/or delay in the network. Solving this synchronization problem requires framing of data streams whereby various points in the data streams deliverable simultaneously to a user are identified. A solution in which the temporal axis of an application is segmented into intervals in which each interval is a unit of synchronization and holds a data frame is presented. Simultaneous data delivery involves delivering all data segments belonging to an interval within a certain real-time delay, as specifiable by the application. Based on the approach, end-to-end transport protocols that compensate for the data skew that may arise due to data loss/delay is described. Simulation experiments confirm the viability of the protocols. Major uses of the protocols are in broadband ISDNs and metropolitan area networks (MANs)     

远程教学直播系统中媒体同步的研究与实现

对于远程教学直播系统这样包含不同媒体流的分布式多媒体应用而言，媒体同步是一项重要内容。利用RTP（实时传输协议）传输机制中的时间戳和序列号信息，提出同步控制算法，实现了流内同步和流间同步。考虑到分组网络带来的延时抖动，算法可以动态地适应网络延时变化，从而保证了分布式环境中媒体同步的服务质量。     

扩展时间流Petri网多媒体同步模型

多媒体同步的一个主要问题是多媒体同步时间要求的描述与建模。文中基于时间流Ｐｅｔｒｉ网模型（ＴＳＰＮ）,通过分析ＴＳＰＮ模型的不足之处,提出了一种新的多媒体同步模型－－扩展时间流Ｐｅｔｒｉ网模型（ＳＴＳＰＮ）,ＳＴＳＰＮ不但可以完全、准确地描述媒体流内和流间的时间行为,而且通过引入容许阻塞时间、问题算法等对ＴＳＰＮ进行扩展,可以有铲地克服网络随机延迟、阻塞和丢包等因素的不良影响,提高了模型的建模能力     

A Multilayered Audiovisual Streaming System Using the Network Bandwidth Adaptation and the Two-Phase Synchronization

Synchronous audiovisual streaming and playout are two of the major issues in the multimedia communication network. However, the past corresponding researches of media synchronization mainly focused on the mono-quality and single-layer (nonscalable) audiovisual data. To overcome challenges of ubiquitous multimedia streaming, a scalable audiovisual coder that can provide flexible scalabilities and adaptive streaming control to adapt to complicated network situations are both required. This paper proposes a multilayered audiovisual streaming scheme to deliver layered audiovisual data synchronously, which is called ML-AVSS. Fine-granular scalability (FGS) and bit-sliced arithmetic coding (BSAC) techniques are used to segment video and audio data into one base-layer and multiple enhancement-layer bitstreams. With advantages of audiovisual layer coding, a de-jitter procedure, a conditional retransmission mechanism and a playout synchronization mechanism are designed to transmit hybrid multilayered audiovisual bitstreams in consideration of the result of a network bandwidth adaptation and the distinct decoding time-complexity. Experimental results show that the proposed ML-AVSS is a feasible streaming scheme to overcome challenges of ubiquitous multimedia streaming, e.g., constrained channel bandwidth, quality degradation, unsmooth playout, etc.     

ODMCA: An adaptive data mining control algorithm in multicarrier networks

Multicarrier communication is a promising technique to effectively deliver high data rate and combat delay spread over fading channel, and adaptability is an inherent advantage of multicarrier communication systems. It can be implemented in online data streams. This paper addresses a significant problem in multicarrier networks that arises in data streaming scenarios, namely, today’s data mining is ill-equipped to handle data streams effectively, and pays little attention to the network stability and the fast response [http://www-db.standford.edu/stream]. Furthermore, in analysis of massive data streams, the ability to process the data in a single pass, while using little memory, is crucial. For often the data can be transmitted faster than it can be stored or accessed from disks.To address the question, we present an adaptive control-theoretic explicit rate (ER) online data mining control algorithm (ODMCA) to regulate the sending rate of mined data, which accounts for the main memory occupancies of terminal nodes. This single-pass scheme considers limited memory space to process dynamic data streams, and also explores the adaptive capability, which is employed in a general network computation model for dynamic data streams. The proposed method uses a distributed proportional integrative plus derivative (PID) controller combined with data mining, where the control parameters can be designed to ensure the stability of the control loop in terms of sending rate of mined data. The basic PID approach for the computation network transmission is presented and z-transformation and Schur–Cohn stability test are used to achieve the stability criterion, which ensures the bounded rate allocation without steady state oscillation. We further show how the ODMCA can be used to design a controller, analyze the theoretical aspects of the proposed algorithm and verify its agreement with the simulations in the LAN case and the WAN case. Simulation results show the efficiency of our scheme in terms of high main memory occupancy, fast response of the main memory occupancy and of the controlled sending rates.     

Synchronized delivery and playout of distributed stored multimedia streams

Multimedia streams such as audio and video impose tight temporal constraints for their presentation. Often, related multimedia streams, such as audio and video, must be presented in a synchronized way. We introduce a novel scheme to ensure the continuous and synchronous delivery of distributed stored multimedia streams across a communications network. We propose a new protocol for synchronized playback and compute the buffer required to achieve both, the continuity within a single substream and the synchronization between related substreams. The scheme is very general and does not require synchronized clocks. Using a resynchronization protocol based on buffer level control, the scheme is able to cope with server drop-outs and clock drift. The synchronization scheme has been implemented and the paper concludes with our experimental results.     

基于自适应QoS保证的多媒体监控系统模型研究

分析分多媒体监控特点,认为多媒体流的QoS保证是一个重要的研究课题。提出了支持自适应QoS保证的嵌入式IP网络多媒体监控系统模型。QoS保证方案包括系统资源监测,网络带宽的预估计,借助PID算法实现多媒体自适应流的自调节,以及音频优先保证。实验表明,该模型结构简单,制定的QoS控制策略能够实时地对视音频参数进行动态调整,资源利用率高,图像传输流畅。     

Adaptive aperiodically intermittent control for pinning synchronization of directed dynamical networks

In this paper, we investigate the pinning synchronization of directed complex network with time‐varying delay under the adaptive aperiodically intermittent control. First, the model of a directed complex network is established, the interaction graph of which is not required to contain a directed spanning tree, and for which an adaptive aperiodically intermittent feedback controller is well designed. In addition, based on the Lyapunov function method and linear matrix inequality technique, several asymptotical synchronization criteria are derived via an aperiodically intermittent strategy, which is superior to the periodical scheme. Moreover, the adaptive strategy providing the exponential convergence rate and the pinning algorithm demonstrating how to select pinned nodes are presented. Finally, numerical examples are provided to illustrate the effectiveness of the derived theoretical results.     

Network-adaptive autonomic transcoding algorithm for seamless streaming media service of mobile clients

As a result of improvements in wireless communication technologies, a multimedia data streaming service can now be provided for mobile clients. Since mobile devices have low computing power and work on a low network bandwidth, a transcoding technology is needed to adapt the original streaming media for mobile environments. However, wireless networks have variable bandwidths depending on the movement of clients and the communication distance from Access Point (AP). These characteristics make it hard to support stable Quality of Service (QoS) streams for mobile clients. In this paper, a target transcoding bit-rate decision algorithm is proposed to provide stable QoS streams for mobile clients. In our experiments, the proposed algorithm provides seamless streaming media services based on the network adaptive bit rate control and reduces transmission failure.     

Adaptive fault tolerant synchronization with unknown propagation delays and actuator faults

The paper addresses a practical issue in adaptive fault tolerant synchronization of masterslave systems with a constant signal propagation delay and actuator faults. First, a new concept of chaos synchronization is proposed, where the real value of the propagation delay can be estimated online by a proper adaptation mechanism; then, a novel adaptive fault estimation algorithm is proposed which removes the classical assumption that the time-derivative of the output errors should be known. Further, based on a proper Lyapunov-Krasovskii function, a novel delay-dependent adaptive fault tolerant synchronization scheme is proposed where the synchronization criterion is derived. Finally, simulation results of the chaotic system demonstrate the effectiveness of the proposed approach.