HDS, a real-time multi-DSP motion estimator for MPEG-4 H.264 AVC high definition video encoding

H.264 AVC video compression standard achieves high compression rates at the cost of a high encoder complexity. The encoder performances are greatly linked to the motion estimation operation which requires high computation power and memory bandwidth. High definition context magnifies the difficulty of a real-time implementation. EPZS and HME are two well-known motion estimation algorithms. Both EPZS and HME are implemented in a DSP and their performances are compared in terms of both quality and complexity. Based on these results, a new algorithm called HDS for Hierarchical Diamond Search is proposed. HDS motion estimation is integrated in a AVC encoder to extract timings and resulting video qualities reached. A real-time DSP implementation of H.264 quarter-pixel accuracy motion estimation is proposed for SD and HD video format. Furthermore HDS characteristics make this algorithm well suited for H.264 SVC real-time encoding applications.     

基于H.264/AVC的视频通信抗分组丢失方法研究

H．264／AVC是新的视频编码标准,其高效的编码效率和良好的网络适配性,使它在多媒体通信中得到广泛应用,然而由于Internet网络是一种不保证QOS的“尽力”网络,基于Internet网络的视频业务将不呵避免地受到数据分组丢失的影响,因此文中基于H．264编码算法的特点提出多种新颖的视频抗分组丢失方法,包括Tornado码的不等保护,利用SEI域自适应改变纠错能力和交互式防误码扩散等．综合使用该文提出的多种抗分组丢失方法,在高丢包率环境下,与使用简单误码掩盖算法相比,恢复图像的平均PSNR可以提高8～15dB,且算法与H．264标准兼容,没有过多增加信道负担和运算复杂度,具有实用价值．     

Joint loss pattern characterization and unequal interleaved FEC protection for robust H.264 video distribution over wireless LAN

The recently adopted H.264 standard achieves efficient video encoding and bandwidth savings. Thus, designing communication protocols and QoS control mechanisms for H.264 video distribution over wireless IP networks is a topic of intense research interest. Delivering video streams to terminals via a wireless last hop is indeed a challenging task due to the varying nature of the wireless link. While a common approach suggests exploiting the variations of the wireless channel, an alternative is to exploit characteristics of the video stream to improve the transmission. In this paper, we combine both approaches through an efficient wireless loss characterization and a low-delay unequal interleaved FEC protection. Besides deriving new QoS metrics for FEC block allocation, the wireless loss characterization is as well used to adjust the interleaving level depending on the loss correlation exhibited by the wireless channel. This novel unequal interleaved FEC (UI-FEC) protocol allows graceful video quality degradation over error-prone wireless links while minimizing the overall bandwidth consumption and the end-to-end latency.     

容错视频编码与传输技术研究

随着无线网络和多媒体技术应用的广泛和深入,在不可靠信道上传输视频的需求日益增长,视频编码和传输中的错误控制问题已经引起了广泛关注.本文全面回顾了过去10多年来的容错编码与传输技术,包括基于编码端的容错、基于解码端的差错掩盖以及信源信道联合编码.此外还对最新的视频编码标准H.264/AVC的容错工具进行了介绍并展望了容错视频编码与传输技术的发展方向.     

H.264/AVC High Profile视频编码中自适应变换模块的设计

提出了一种可配置的整数变换运算单元并将其用于H.264/AVC HiProfile视频编码器的自适应变换模块中。通过变换类型信号的配置,该变换单元可以完成相应的变换操作。本设计采用Altera公司的CycloneⅡ系列FPGA进行实现和验证,布局布线后的最大工作频率为63 MHz,采用4个可配置变换单元的变换模块,可以满足HD1080P@50帧/s视频的实时编码要求。     

H.264 video transmissions over wireless networks: Challenges and solutions

Multimedia video streaming is becoming increasingly popular. Using multimedia services, there are more and more users in end-system over wireless networking environment. H.264/AVC is now the standard for video streaming because of its high compression efficiency, robustness against errors and network-friendly features. However, providing the desired quality of service or improving the transmission efficiency for H.264 video transmissions over wireless networks present numbers of challenges. In this paper, we consider those challenges and survey existing mechanisms based on the protocol layers they work on. Finally, we address some open research issues concerning for H.264 video transmission in wireless networks.     

Quality enhancement for scalable video delivery over wireless networks based on particle swarm optimization

In this paper, the main objective is to find an optimal rate allocation strategy that can maximize the total-weighted quality of experience (QoE) associated with multiple video sources over error-prone multi-hop wireless networks based on the particle swarm optimization (PSO) technique which belongs to the family of swarm intelligence algorithms. In video transmission over such wireless networks, many network-based (packet loss, delay etc.) and source-based (encoding quantization level etc.) parameters can impair the perceived video quality. The main contributions of the proposed work are twofold. At first, an optimal bandwidth allocation framework is being developed based on PSO in which by incorporating an accurate video quality metric, the total weighted quality of experience of some competing video sources is being optimized. Second, these optimal rates have been used for differentiated QoE enforcement between multiple competing scalable video sources. The resulting optimal rates can be used as rate-feedbacks for on-line rate adaptation of a moderate scalable video encoder such as H.264/MPEG4 AVC. The aforementioned weight parameters are selected based on the importance of each video sequence’s quality and can be associated with some previous service level agreement-based prices. A strong motivation for differentiated quality enforcement is that video sources need to be encoded differently for different resolutions to cater to diverse devices from mobile displays to HDTV displays. Some numerical analysis have been presented to validate the theoretical results and to verify the claims.     

An Adaptive Motion-Based Unequal Error Protection Approach for Real-Time Video Transport Over Wireless IP Networks

In this work, we consider the delivery of digital video over future 3G wireless IP networks and we propose a low-complexity adaptive motion-based unequal error protection (UEP) video coding and transmission system which efficiently combines three existing error-resilience techniques by exploiting knowledge of the source material as well as the channel operating conditions. Given this information, the proposed system can adaptively adjust the operating parameters of the video source encoder and the forward error correction (FEC) channel encoder to maximize the delivered video quality based upon both application-layer video motion estimates and link-layer channel estimates. We demonstrate the efficacy of this approach using the ITU-T H.264 video source coder. The results indicate that a significant performance improvement can be achieved with enhanced resilience to inaccurate channel feedback information and with substantially reduced computational complexity compared to competing approaches.     

Adaptive robust video broadcast via satellite

With increasing demand for multimedia content over channels with limited bandwidth and heavy packet losses, higher coding efficiency and stronger error resiliency is required more than ever before. Both the coding efficiency and error resiliency are two opposing processes that require appropriate balancing. On the source encoding side the video encoder H.264/AVC can provide higher compression with strong error resiliency, while on the channel error correction coding side the raptor code has proven its effectiveness, with only modest overhead required for the recovery of lost data. This paper compares the efficiency and overhead of both the raptor codes and the error resiliency techniques of video standards so that both can be balanced for better compression and quality. The result is also improved by confining the robust stream to the period of poor channel conditions by adaptively switching between the video streams using switching frames introduced in H.264/AVC. In this case the video stream is initially transmitted without error resiliency assuming the channel to be completely error free, and then the robustness is increased based on the channel conditions and/or user demand. The results showed that although switching can increase the peak signal to noise ratio in the presence of losses but at the same time its excessive repetition can be irritating to the viewers. Therefore to evaluate the perceptual quality of the video streams and to find the optimum number of switching during a session, these streams were scored by different viewers for quality of enhancement. The results of the proposed scheme show an increase of 3 to 4 dB in peak signal to noise ratio with acceptable quality of enhancement.     

SIMD并行运算的参考图像组织方法

从单指令多数据并行运算的角度出发,将面向对象的思想引入到SAD值计算的并行操作过程中,给出了一种改进的图像组织优化算法,通过对多个标准测试序列进行运动预测的测试结果知,在当前最复杂的视频编码H.264/AVC上,该算法的实施可以明显地提高编码器的编码速度,为实现窄带中的实时视频通信提供了保障.     

Joint Source Coding and Network-Supported Distributed Error Control for Video Streaming in Wireless Multihop Networks

Real-time video communication over wireless multihop networks has gained significant interest in the last few years. In this paper, we focus our attentions on the problem of source coding and link adaptation for packetized video streaming in wireless multihop networks when network nodes are media-aware. We consider a system where source coding is employed at the video encoder by selecting the encoding mode of each individual macro-block, while error control is exercised through application-layer retransmissions at each media-aware network node. For this system model, the contribution of each communication link on the end-to-end video distortion is considered separately in order to achieve globally optimal source coding and ARQ error control. To reach the globally optimal solution, we formulate the problem of joint source and distributed error control (JSDEC) and devise a low-complexity solution algorithm based on dynamic programming. Extensive experiments have been carried out on the basis of H.264/AVC codec to demonstrate the effectiveness of the proposed algorithm over the existing joint source and channel coding (JSCC) algorithm in terms of PSNR perceived at the decoder under time-varying multihop wireless links.     

基于MMX技术的SIMD并行运算优化算法

将面向对象思想引入到SAD值计算的并行操作过程中,并从SIMD并行运算的角度出发,给出了改进的图像组织优化算法,通过对MMX优化后的编码器速度的测试结果知,在目前H.264/AVC的视频编码上,该编码器的编码速度有明显地提高,为实现窄带中的实时视频通信提供了保障.     

An efficient parallelization technique for x264 encoder on heterogeneous platforms consisting of CPUs and GPUs

H.264/AVC video encoders have been widely used for its high coding efficiency. Since the computational demand proportional to the frame resolution is constantly increasing, it has been of great interest to accelerate H.264/AVC by parallel processing. Recently, graphics processing units (GPUs) have emerged as a viable target for accelerating general purpose applications by exploiting fine-grain data parallelisms. Despite extensive research efforts to use GPUs to accelerate the H.264/AVC algorithm, it has not been successful to achieve any speed-up over the x264 algorithm that is known as the fastest CPU implementation, mainly due to significant communication overhead between the host CPU and the GPU and intra-frame dependency in the algorithm. In this paper, we propose a novel motion-estimation (ME) algorithm tailored for NVIDIA GPU implementation. It is accompanied by a novel pipelining technique, called sub-frame ME processing, to effectively hide the communication overhead between the host CPU and the GPU. Further, we incorporate frame-level parallelization technique to improve the overall throughput. Experimental results show that our proposed H.264 encoder has higher performance than x264 encoder.     

3G对话式视频业务中H．264／AVC的容错策略

3G通信技术的出现使对话式无线视频业务成为可能，而且最新的3GPP／3GPP2标准要求3G终端支持H．264／AVC视频编解码技术，但由于无线网络传输的误率高，因此必须为3G对话式视频业务提供相关的容错技术；同时由于硬件的限制，3G终端只支持部分H．264／AVC的容错工具。针对上述问题，根据3GPP／3GPP2标准，在对H．264／AVC中的各种容错工具在基于分组交换的3G对话式视频业务中的适用性进行分析的基础上，给出了一些实用性的容错方案。这些容错方案的性能经过一般测试条件测试，实验结果表明，能够得到比较满意的纠错效果。同时通过对各种实验数据的分析可见，在这种低带宽、高误码率的环境中，若编码时采用复杂度低的FMO方式，再辅以合适的帧内编码块刷新模式，则可以充分利用帧内宏块的空间相关性，以便在有限的带宽内能得到更好的纠错能力。     

H.264编码器存储带宽分析及DRAM控制器设计

在分析H.264/AVC编码过程中存储器带宽需求的基础上,提出一种DRAM控制器结构,并实现了几种不同调度策略的DRAM控制器结构设计。实现了令牌环、固定优先级和抢占式等三种结构,结合已有的存储空间映射方法,通过减少换行及Bank切换过程中的冗余周期,进一步提高存储器的带宽利用率。实验结果表明,提出的三种存储器结构中抢占式调度具有最高的宽利用率,可满足150 MHz时钟频率条件下HDTV1080P实时编码的应用。     

基于IP网络的H.264/AVC视频实时传输

随着新的视频编码标准H.264/AVC的逐步推广应用,如何基于IP网络应用环境来实现H.264/AVC视频的实时传输,对于发展高质量视频传输具有重要意义,为此,提出了一种基于IP网络的H.264/AVC实时传输解决方案,并着重从H.264-FGS编码器和速率控制两方面进行阐述。     

IEEE 802.11e中基于失真度度量的视频包传输方法

为改善H. 264编码的视频流在802. 11e中的传输性能, 提出了一种结合H. 264/AVC中不同类型的数据分割对视频重建质量的重要性因子和队列状态的视频包映射方法。首先定量分析H. 264/AVC中A、B、C三种分割的丢失对视频重建质量的影响, 得到其重要因子; 然后依据重要因子和队列长度将视频数据包映射到802. 11e的不同EDCA队列中。算法改进了EDCA机制中数据包的静态映射机制, 根据视频分割数据的不等重要性, 提供差异性服务。仿真结果表明, 与目前的视频包静态映射机制相比, 该算法提高了视频重建质量, 最好可提高1 dB以上。     

Dynamic Resource Allocation for MGS H.264/AVC Video Transmission Over Link-Adaptive Networks

In this paper, we address the problem of efficiently allocating network resources to support multiple scalable video streams over a constrained wireless channel. We present a resource allocation framework that jointly optimizes the operation of the link adaptation scheme in the physical layer (PHY), and that of a traffic control module in the network or medium access control (MAC) layer in multirate wireless networks, while satisfying bandwidth/capacity constraints. Multirate networks, such as IEEE 802.16 or IEEE 802.11, adjust the PHY coding and modulation schemes to maintain the reliability of transmission under varying channel conditions. Higher reliability is achieved at the cost of reduced PHY bit-rate which in turn necessitates a reduction in video stream bit-rates. The rate reduction for scalable video is implemented using a traffic control module. Conventional solutions operate unaware of the importance and loss tolerance of data and drop the higher layers of scalable video altogether. In this paper, we consider medium grain scalable (MGS) extension of H.264/AVC video and develop new rate and distortion models that characterize the coded bitstream. Performance evaluations show that our proposed framework results in significant gains over existing schemes in terms of average video PSNR that can reach 3 dB in some cases for different channel SNRs and different bandwidth budgets.      

Delay Constraint Error Control Protocol for Real-Time Video Communication

Real-time video communication over wireless channels is subject to information loss since wireless links are error-prone and susceptible to noise. Popular wireless link-layer protocols, such as retransmission (ARQ) based 802.11 and hybrid ARQ methods provide some level of reliability while largely ignoring the latency issue which is critical for real-time applications. Therefore, they suffer from low throughput (under high-error rates) and large waiting-times leading to serious degradation of video playback quality. In this paper, we develop an analytical framework for video communication which captures the behavior of real-time video traffic at the wireless link-layer while taking into consideration both reliability and latency conditions. Using this framework, we introduce a delay constraint packet embedded error control (DC-PEEC) protocol for wireless link-layer. DC-PEEC ensures reliable and rapid delivery of video packets by employing various channel codes to minimize fluctuations in throughput and provide timely arrival of video. In addition to theoretically analyzing DC-PEEC, the performance of the proposed scheme is analyzed by simulating real-time video communication over “real” channel traces collected on 802.11b WLANs using H.264/AVC JM14.0 video codec. The experimental results demonstrate performance gains of 5–10 dB for different real-time video scenarios.      

基于Cauchy分布的H.264/AVC码率控制优化算法

由于通信信道带宽有限,有必要对视频编码的输出码率进行控制。通过引入能量均方误差(EMSE),进行视频序列统计分析,建立了一种简单的线性图像失真模型,并基于编码器中量化系数的概率分布特性,提出了改进的Cauchy分布码率模型,然后利用Lagrangian方法实现了率失真优化的位分配。实验结果表明,与H.264参考软件所采用的JVT-G012算法相比,该算法提高了码率控制精度和峰值信噪比,从而改善了解码图像的质量。