From broadcast television to Internet audio/video: techniques and tools for VCR‐style interactivity

One of the new applications evolving in the Internet is streaming audio/video. A major reason for its growing popularity is interest in the compelling new services that become possible. Prototype services are being developed which are new to the Internet but offer the same look, feel, and functionality that have traditionally only been found in services delivered via other communication medium, e.g. broadcast television. In addition, the Internet is evolving to offer ‘value‐added’ services, like streaming audio/video with VCR‐style interactivity and embedded hyperlinks. We are poised both on seeing the development of new paradigms for interacting with audio/video, and on seeing the merging of broadcast television and Internet‐based broadcasts. Before this process can be considered successful, a number of technical challenges, derived from the various ways in which content is physically delivered, must be solved. In this paper, we focus on the value‐added service of VCR interactivity. VCR interactivity has long been a challenge for both broadcast television and streamed Internet audio/video. The challenge is how to provide individualized playout for content being streamed to a large group of users using one‐to‐many delivery. While some new companies are starting to offer devices which provide this kind of service for broadcast television, there are still numerous technical challenges for the Internet‐based version of a similar service. This paper has a three‐fold objective. First, we describe the types of services available in the traditional broadcast infrastructure and compare these to the types of services that are deployed or possible in Internet‐based services. Second, we describe our attempts to implement some of the more challenging and novel service types. In particular, we examine client‐based control of programs streamed over the Internet to tens, thousands, or even millions of users. Finally, we discuss the impact of these services on the protocols and applications used to support Internet‐based, multi‐party conferencing. Copyright © 2001 John Wiley & Sons, Ltd.     

Empirical evaluation of H.264/SVC streaming in resource-constrained multihomed mobile networks

Nomadic users of streamed multimedia content in mobile networks are often faced with resource-constrained network paths that suffer from low bandwidth. Streaming high-quality video in such a challenging scenario demands a set of highly adaptive schemes, which have not been sufficiently explored in particular for the emerging H.264 Scalable Video Coding (H.264/SVC) standard. In this paper, we empirically investigate the performance of streaming H.264/SVC scalable video streams to users in multihomed mobile networks containing multiple available transmission paths. Previous work has demonstrated the feasibility of aggregating bandwidth of multiple paths to deliver video streams when no single, sufficiently high bandwidth path is available. We focus on evaluating the enhanced performance of multipath bandwidth-aggregation streaming by exploiting a quality-layers based, H.264/SVC-specific packet prioritisation scheme for quality-aware multipath packet scheduling and selective packet dropping in case of bandwidth shortage even after aggregation. Additionally, we explore a base-layer rate control scheme for H.264/SVC delivery in ultra-low bandwidth environments. Through extensive experimentation on a realistic hardware-based testbed, we obtain a comprehensive and insightful understanding of the behaviour of H.264/SVC streams when transmitted across multiple paths in mobile networks. We quantify the improvements offered by the use of H.264/SVC-specific packet prioritisation schemes compared with an existing generic scalable video prioritisation scheme, and the benefits by the use of base-layer rate control in ultra-low bandwidth situations. The performance of the multipath streaming schemes is further compared with that of an ideal single high bandwidth path. We also identify the remaining challenges that must be overcome if such streaming schemes are to offer performance close to that of the ideal single high bandwidth path.     

运动感知基于缓存的自适应视频流传输

目的 基于缓存的自适应视频流传输策略无需估测实时带宽,直接通过缓存变化量与码率的映射函数选取符合当前网络状况的最佳质量码流传输。传统基于缓存的自适应视频传输不考虑内容特征,在码率选择上为不同运动级别视频内容均使用相同的码率映射函数,在不稳定的无线网络环境中高运动强度内容的码率急剧降低会严重伤害用户体验质量（QoE）,提出运动感知基于缓存的自适应视频流传输（MA-BBA）算法。方法 MA-BBA算法根据片段运动级别确定码率映射函数,对运动强度高的内容快速切换到较高码率,而对于运动强度较低的内容则使用较为保守的码率,从而使得缓存资源能够位于安全边界之上且较多分配给高级别运动内容,提高不同运动强度内容的平均质量,使整体QoE得到优化。结果 在公开的无线网络带宽数据集上实现本文MA-BBA算法,基于吞吐量的自适应传输算法（TBA）和基于缓存的自适应传输算法（BBA）。MA-BBA在高运动强度内容的平均质量上比TBA和BBA分别提高1.7%和1.2%,且质量波动区间更小。MA-BBA在平均缓存利用率上达到72%,大大高于TBA的45.9%和BBA的45.4%。结论 MA-BBA算法与现有的码率自适应算法TBA和BBA相比,大大提高了缓存资源利用率,提高了对资源要求最苛刻的高级别运动内容的传输质量,减小码率切换幅度频率,优化了视频服务的整体QoE。     

Content-adaptive wireless streaming of instructional videos

Video streaming over wireless networks is becoming increasingly important for a variety of applications. To accommodate the dynamic change of wireless network bandwidths, Quality of Service (QoS) scalable video streams need to be provided. This paper presents a system of content-adaptive streaming of instructional (lecture) videos over wireless networks for E-learning applications. We first provide a real-time content analysis method to detect and extract content regions from instructional videos, then apply a “leaking-video-buffer” model to adjust QoS of video streams dynamically based on video content. In content-adaptive video streaming, an adaptive feedback control scheme is also developed to transmit properly compressed video streams to video clients not only based on network bandwidth, but also based on video content and the preferences of users. Finally, we demonstrate the scalability and content adaptiveness of the proposed video streaming system with experimental results on several instructional videos.     

Content-adaptive wireless streaming of instructional videos

Video streaming over wireless networks is becoming increasingly important for a variety of applications. To accommodate the dynamic change of wireless network bandwidths, Quality of Service (QoS) scalable video streams need to be provided. This paper presents a system of content-adaptive streaming of instructional (lecture) videos over wireless networks for E-learning applications. We first provide a real-time content analysis method to detect and extract content regions from instructional videos, then apply a “leaking-video-buffer” model to adjust QoS of video streams dynamically based on video content. In content-adaptive video streaming, an adaptive feedback control scheme is also developed to transmit properly compressed video streams to video clients not only based on network bandwidth, but also based on video content and the preferences of users. Finally, we demonstrate the scalability and content adaptiveness of the proposed video streaming system with experimental results on several instructional videos.     

Placement of continuous media in wireless peer-to-peer networks

This paper investigates a novel streaming architecture consisting of home-to-home online (H2O) devices that collaborate with one another to provide on-demand access to large repositories of continuous media such as audio and video clips. An H2O device is configured with a high bandwidth wireless communication component, a powerful processor, and gigabytes of storage. A key challenge of this environment is how to place data across H2O devices in order to enhance startup latency, defined as the delay observed from when a user requests a clip, to the onset of its display. Our primary contribution is a novel replication technique that enhances startup latency, while minimizing the total storage space required from an environment consisting of N H2O devices. This technique is based on the following intuition: The first few blocks of a clip are required more urgently than its last few blocks, and should be replicated more frequently in order to minimize startup latency. We develop analytical models to quantify the number of replicas required for each block. In addition, we describe two alternative distributed implementation of our replication strategy. When compared with full replication, our technique provides on average greater than 97% (i.e., several orders of magnitude) savings in storage space, while ensuring zero startup latency and a hiccup-free reception.     

Super-Streaming: A New Object Delivery Paradigm for Continuous Media Servers

A number of studies have focused on the design of continuous media, CM, (e.g., video and audio) servers to support the real-time delivery of CM objects. These systems have been deployed in local environments such as hotels, hospitals and cruise ships to support media-on-demand applications. They typically stream CM objects to the clients with the objective of minimizing the buffer space required at the client site. This objective can now be relaxed due to the availability of inexpensive storage devices at the client side. Therefore, we propose a Super-streaming paradigm that can utilize the client side resources in order to improve the utilization of the CM server. To support super-streaming, we propose a technique to enable the CM servers to deliver CM objects at a rate higher than their display bandwidth requirement. We also propose alternative admission control policies to downgrade super-streams in favor of regular streams when the resources are scarce. We demonstrate the superiority of our paradigm over streaming with both analytical and simulation models.Moreover, new distributed applications such as distant-learning, digital libraries, and home entertainment require the delivery of CM objects to geographically disbursed clients. For quality purposes, recently many studies proposed dedicated distributed architectures to support these types of applications. We extend our super-streaming paradigm to be applicable in such distributed architectures. We propose a sophisticated resource management policy to support super-streaming in the presence of multiple servers, network links and clients. Due to the complexity involved in modeling these architectures, we only evaluate the performance of super-streaming by a simulation study.     

移动边缘计算中基于视频内容协作分发的联合激励机制

随着智能设备存储和传输能力的提升,用户可通过设备—设备连接共享视频服务,在移动边缘网络中实现视频内容协作分发.然而,现有架构缺少合适的激励机制解决中继传输节点经济收益受损和本地数据泄露问题.针对视频内容边缘协作分发系统存在的问题,提出适用于多码率编码视频的联合安全和经济激励机制,并对该机制的差分隐私性、近似诚实性和个体理性进行了理论上的证明.最后由实验数据表明,该机制能有效保护参与用户的个体利益,提高视频服务总体效益.     

Modeling and evaluating IPTV applications in WiMAX networks

Internet Protocol-based Television (IPTV) is a digital television service which delivers television content via an IP network. The rapid growth of wireless network technology in recent years has changed, the way people access the Internet. Adding mobility to IPTV can create a truly compelling ubiquitous service which spans different network domains and varied IP-enabled terminals and devices, such as set-top boxes, PCs and cell phones. However, extending IPTV service to wireless networks requires overcoming bandwidth bottlenecks and high packet loss rates. Following the IEEE 802.16 standard, worldwide interoperability for microwave access (WiMAX) features high data rates and large service coverage, offering a wireless broadband solution for IPTV services. While previous research has focused on creating a broadband IPTV service few studies have practically evaluated IPTV applications in a wireless broadband network environment. In this paper, we model and evaluate a common constant bit rate (CBR)¹ based IPTV application and an IPTV live streaming (PPStreaming)² application while retrieving IPTV content via a WiMAX network. We also use the NS2 simulation tool to evaluate the performance of these two IPTV applications. The evaluation metrics include latency, packet loss, data rate and throughput statistics when the two IPTV applications are run in the WiMAX network. ¹The simplest IPTV solution is to convey video content by CBR. IPTV operators and content delivery networks relay CBR streaming content to control the demand for network capacity. Broadcasters prefer CBR video as it conserves bandwidth resources, but CBR delivery can degrade video quality result in higher overall demand on network capacity. ²PPStreaming (also referred to as P2P streaming Internet TV) is a network for live media streaming. In principle it’s similar to BitTorrent (BT) in that it provides stable and smooth broadcast of TV programs to broadband users. Unlike traditional streaming media, PPStreaming adopts P2P-streaming technology and supports full-scale visits with tens of thousands of simultaneous users. Its client software can be used in the browser or as a standalone executable.     

CStream: neighborhood bandwidth aggregation for better video streaming

Despite the popularity of watching videos online, challenges still remain in video streaming in many scenarios. Limited home broadband and mobile phone 3G bandwidths mean many users stream videos at compromised quality. To provide additional bandwidth for streaming, we propose CStream, a system that aggregates bandwidth from multiple cooperating users in a neighborhood environment for better video streaming. CStream exploits the fact that wireless devices have multiple network interfaces and connects cooperating users with a wireless ad-hoc network to aggregate their unused downlink Internet bandwidth. CStream dynamically generates a streaming plan to stream a single video using multiple connections, continuously adapting to changes in the neighborhood and variations in the available bandwidth. CStream is developed and evaluated on a test bed of computers, allowing for a detailed, controlled evaluation of performance. Analysis of the results shows a linear increase in throughput over single-connection streaming and improved video quality as the number of cooperating users in a neighborhood increase.     

A systematic rate controller for MPEG-4 FGS video streaming

This paper proposes a systematic rate controller (SRC) for content-aware streaming of MPEG-4 FGS video over the Internet. An active layer dropping technique is proposed to provide both coarse-grain and fine-granularity scalability of smooth quality adaptation to bandwidth fluctuations and bit-rate variations of streamed video over a general time-scale. The smooth quality adaptation is realized through the mode and state transition of a state machine that implements the SRC. The SRC effectively uses available bandwidth and client buffer by forward-shifting the FGS video stream. It provides protection to video segments with important content by introducing a content-aware priority-based layer model for the MPEG-4 FGS video stream. RID="*" ID="*" The work reported in this paper was performed when this author was working at Microsoft Research Asia as a research intern.     

Adaptive interface selection over cloud-based split-layer video streaming via multi-wireless networks

As mobile devices such as tablet PCs and smartphones proliferate, the online video consumption over a wireless network has been accelerated. From this phenomenon, there are several challenges to provide the video streaming service more efficiently and stably in the heterogeneous mobile environment. In order to guarantee the QoS of real-time HD video services, the steady and reliable wireless mesh is necessary. Furthermore, the video service providers have to maintain the QoS by provisioning streaming servers to respond the clients’ request of different video resolution. In this paper, we propose a reliable cloud-based video delivery scheme with the split-layer SVC encoding and real-time adaptive multi-interface selection over LTE and WiFi links. A split-layer video streaming can effectively scale to manage the required channels on each layer of various client connections. Moreover, split-layer SVC model brings streaming service providers a remarkable opportunity to stream video over multiple interfaces (e.g. WiFi, LTE, etc.) with a separate controlling based on their network status. Through the adaptive interface selection, the proposed system aims to ensure the maximizing video quality which the bandwidth of LTE/WiFi accommodates. In addition, the system offers cost-effective streaming to mobile clients by saving the LTE data consumption. In our system, an adaptive interface selection is developed with two different algorithms, such as INSTANT and EWMA methods. We implemented a prototype of mobile client based on iOS particularly by using iPhone5S. Moreover, we also employ the split-layer SVC encodes in streaming server-side as the add-on module to SVC reference encoding tool in a virtualized environment of KVM hypervisor. We evaluated the proposed system in an emulated and a real-world heterogeneous wireless network environments. The results show that the proposed system not only achieves to guarantee the highest quality of video frames via WiFi and LTE simultaneous connection, but also efficiently saves LTE bandwidth consumption for cost-effectiveness to client-side. Our proposed method provides the highest video quality without deadline misses, while it consumes 50.6% LTE bandwidth of ‘LTE-only’ method and 72.8% of the conventional (non-split) SVC streaming over a real-world mobile environment.     

Utility-oriented resource allocation for 360-degree video transmission over heterogeneous networks

Immersive media streaming, especially virtual reality (VR)/360-degree video streaming which is very bandwidth demanding, has become more and more popular due to the rapid growth of the multimedia and networking deployments. To better explore the usage of resource and achieve better video quality perceived by users, this paper develops an application-layer scheme to jointly exploit the available bandwidth from the LTE and Wi-Fi networks in 360-degree video streaming. This newly proposed scheme explores the saliency of video contents, prediction of users' view and channel status information to maximize the system's utility in a multi-RAN environment. It also determines the optimal association of users with multiple Wi-Fi access points (APs). Besides, a novel buffer strategy is proposed to mitigate the influence of short-time prediction problem for transmitting 360-degree videos in time-varying networks. The promising performance and low complexity of the proposed scheme and algorithms are validated in simulations with various 360-degree videos.     

Tele-immersive environment with tiled display wall for intuitive remote collaborative work

In remote collaborative work via WAN, sharing and recognizing various high-quality visual contents such as photography, visualization image, and real video streaming is extremely important. Supporting a high-quality display of these contents on a large-scale display system is necessary to support intellectual remote collaborative work. However, these contents are currently magnified in low resolution on a general projector used as large-sized display equipment, and these equipments are difficult to display the contents with sufficient quality. In this paper, we have focused on the tiled display wall technology which configure a wide-area screen with two or more LCD and tried to display realistic high-resolution visual contents to solve these issues. We have constructed a tele-immersive environment with a tiled display wall, and have studied the availability for intuitive remote collaborative work by implementing various collaborative applications for the effective display of high-resolution contents as well as developing interaction techniques for enormous visual contents in this environment. As a result, we have showed that the practical use of a tiled display wall is useful in the construction of intellectual remote collaborative environment.     

多层次周期性广播策略及系统模型——具有良好可扩展性的异步视频传输策略

异步视频服务是基于高速互联网络的典型应用,但是长期以来一直面临着系统可扩展性较差的问题,大量的用户数与有限的带宽资源之间的矛盾成为其主要问题。该文提出了一种具有良好可扩展性的异步视频传输模型,在网络层基于下一代高速互联网络将广泛支持的组播技术,在应用层创造性地使用了多层次的周期性广播策略。从而较为有效地解决了在分布式多视频服务器环境下系统提供异步视频服务的性能、价格以及可扩展性问题。     

V3: A vehicle-to-vehicle live video streaming architecture

We propose V3, an architecture to provide a live video streaming service to driving vehicles through vehicle-to-vehicle (V2V) networks. V2V video streaming is challenging because: (1) the V2V network may be persistently partitioned; (2) the video sources are mobile and transient. V3 addresses these challenges by incorporating a novel signaling mechanism to continuously trigger vehicles into video sources. It also adopts a store-carry-and-forward approach to transmit video data in partitioned network environments. We first propose an initial design which supports video streaming to a single receiver. We then propose a multicasting framework that enables video streaming from multiple sources to multiple receivers. Simulation experiments demonstrate the feasibility of supporting V2V video streaming with acceptable performance.     

Frequent layer switching for perceived quality improvements of coarse-grained scalable video

Scalable video is an attractive option for adapting the bandwidth consumption of streaming video to the available bandwidth. Fine-grained scalability can adapt most closely to the available bandwidth, but this comes at the cost of a higher overhead compared to more coarse-grained videos. In the context of VoD streaming, we have therefore explored whether a similar adaptation to the available bandwidth can be achieved by performing layer switching in coarse-grained scalable videos. In this approach, enhancement layers of a video stream are switched on and off to achieve any desired longer term bandwidth. We have performed three user studies, two using mobile devices and one using an HDTV display, to evaluate the idea. In several cases, the far-from-obvious conclusion is that layer switching is a viable way of achieving bit-rate savings and fine-grained bit-rate adaptation even for rather short times between layer switches, but it does, however, depend on scaling dimensions, content and display device.     

Implementation of 4kUHD HEVC-content transmission

The Internet of things (IoT) has received a great deal of attention in recent years, and is still being approached with a wide range of views. At the same time, video data now accounts for over half of the internet traffic. With the current availability of beyond high definition, it is worth understanding the performance effects, especially for real-time applications. High Efficiency Video Coding (HEVC) aims to provide reduction in bandwidth utilisation while maintaining perceived video quality in comparison with its predecessor codecs. Its adoption aims to provide for areas such as television broadcast, multimedia streaming/storage, and mobile communications with significant improvements. Although there have been attempts at HEVC streaming, the literature/implementations offered do not take into consideration changes in the HEVC specifications. Beyond this point, it seems little research exists on real-time HEVC coded content live streaming. Our contribution fills this current gap in enabling compliant and real-time networked HEVC visual applications. This is done implementing a technique for real-time HEVC encapsulation in MPEG-2 Transmission Stream (MPEG-2 TS) and HTTP Live Streaming (HLS), thereby removing the need for multi-platform clients to receive and decode HEVC streams. It is taken further by evaluating the transmission of 4k UHDTV HEVC-coded content in a typical wireless environment using both computers and mobile devices, while considering well-known factors such as obstruction, interference and other unseen factors that affect the network performance and video quality. Our results suggest that 4kUHD can be streamed at 13.5 Mb/s, and can be delivered to multiple devices without loss in perceived quality.

     

基于Linux的网络直播平台设计与实现

不同带宽用户对流媒体直播服务器的音频/视频质量有不同需求,为此,设计并实现一个基于Linux平台的网络直播平台。该平台采用MPEG4IP编码库的方法实现MPEG4软编码,利用实时传输协议传输数据的方法实现客户端直播请求,通过调用简单媒体接口库的方法实现视频在Linux平台的显示播放。实验结果证明,该网络直播平台传输效果较好,并且其稳定性和负载均衡性均达到设计要求。     

矿用实时视频传输系统设计

针对现有煤矿井下视频传输系统存在视频清晰度低、传输速率不稳定、兼容性差等问题,设计了一种矿用实时视频传输系统。该系统采用960 nm红外激光作为辅助光源,利用MCCD图像传感器采集视频信号,提高了低光照强度或黑暗环境下视频清晰度;通过视频解码模块TVP5150将采集的PAL制式模拟视频信号转换为YUV数字信号,数字信号经多格式编码器进行H.264压缩编码,并在此基础上添加UDP报文头进行RTP封装,提高了视频数据传输的时效性;通过Live555流媒体服务器进行数据流化,使用ONVIF标准封装RTSP视频流,通过Socket网络编程实现实时视频流数据网络传输,提高了系统兼容性和传输速率稳定性。测试结果表明,该系统视频传输速率为2.190 Mbit/s,丢包率约为1.256%,达到实时视频传输要求。