A New Dual-bitstream Video Streaming System with VCR Functionalities Using SP/SI-frames

With the proliferation of digital video and the popularity of video streaming applications, it is highly desirable to find and access video segments of interest by searching through the content of video at a speed that is faster than a normal playback. The key functions that enable quick browsing of video are fast-forward and fast-reverse playbacks. However, motion-compensated prediction adopted in the current video coding standards drastically complicates these operations. One approach to implement the fast-forward/reverse playback is to store an additional reverse-encoded bitstream in the server. Once the client requests a fast-forward/reverse operation, the server can select an appropriate frame for the client from either the forward-encoded bitstream or the reverse-encoded bitstream by considering the cost of network bandwidth and decoder complexity. Unfortunately, these two bitstreams are encoded separately. The frame in one bitstream may not be exactly identical to the frame in another bitstream. If one of these frames is then used as the reference for the requested frame, which is in another bitstream, it induces mismatch errors. In this paper, a novel H.264 dual-bitstream system aiming at providing the fast-forward/reverse playback based on SP/SI-frames is proposed. The proposed system can completely eliminate mismatch errors when the frame in the reverse-encoded bitstream replaces the frame in the forward-encoded bitstream and vice versa. Experimental results confirm that the proposed system is effective in eliminating mismatch errors so as to enhance the performance of the dual-bitstream system.     

基于邻域运动一致性的运动矢量快速构建算法

在时移电视应用中,视频快进播放要求视频服务器能够通过跳帧抽取视频序列中的I帧和P帧的方式输出快速播放码流,当所跳帧中含有P帧时,必需对抽取出来的快进码流中的P帧进行重编码.为提高抽取后重编码效率,可利用码流中的原始运动矢量对跳帧后新参考帧条件下的运动矢量进行预测.在FDVS和ADVS算法基础上,提出了一种基于宏块及其相邻宏块运动一致性的算法来修正原始运动矢量,并通过实验将其与现有的FDVS和ADVS算法进行比较.     

Recovery of motion vectors for error concealment based on an edge-detection approach

A new approach to error concealment is presented which exploits both spatial and temporal information from the current and previous frames. The technique consists of two stages, motion vector estimation and enhancement of the estimated motion vector. In the first stage the proposed method estimates a replacement for the corrupted motion vector by applying dynamic weights to related motion vectors from the top, bottom, left and right sub-macroblocks. The estimated motion vectors are then enhanced using a new approach based on edge detection in the second stage. The experimental results for several test video sequences are compared with conventional error concealment methods and higher performance is achieved in both objective peak signal-to-noise ratio measurements and subjective visual quality.     

Motion estimation in low-delay hierarchical p-frame coding using motion vector composition

Low-delay hierarchical prediction structure is currently adopted in various new video coding standards. The only hurdle of this structure is the need of motion estimation in distant reference frames. To maintain high coding efficiency, a large search range for motion estimation can improve the coding efficiency in distant reference pictures. Computational complexity will thus be increased dramatically. In this paper, a fast motion estimation scheme for a low-delay hierarchical P-frame structure is proposed. The proposed scheme adopts a motion vector composition strategy to expedite the motion estimation process for distant reference frames in the hierarchical P structure. In addition, a motion vector composition algorithm is tailor-made with the proposed hierarchical P coding scheme to further improve the coding efficiency. Simulation results show that the proposed scheme can deliver a remarkable complexity savings and coding efficiency improvement on coding a frame in low temporal layers of the hierarchical P structure.     

Small-diamond-based search algorithm for fast block motion estimation

A good fast motion search algorithm should efficiently speed up the encoding time and keep the quality of encoded video stable at the same time. Researches have shown that many fast algorithms lose the quality requirement in some special video sequences. These video sequences often have heavy motions and need large search windows for motion vector search. E3SS, DS, and E-HEXBS, which are famous algorithms, are not good enough in these sequences. As to UMHexagonS, it is able to meet the high video quality requirement very well, but it costs too much computation. This paper introduces a multi-stage motion estimation algorithm. The algorithm ensures getting good video quality while decreases the motion search time efficiently. It divides the search regions into many un-overlapped small-diamond regions and forces the motion search to go outward for larger motion vectors. This method is also designed to avoid mistaking local optimal motion vectors. For this reason, the selected motion vector is refined by several stages. Experimental results show that the proposed algorithm uses almost the same number of checking points as E3SS but achieves a better quality. Furthermore, the proposed algorithm is also tested in H.264/AVC JM9.5 encoder; the experimental results show that this algorithm is also suitable for variable block-size motion estimation.     

Motion adaptive video coding scheme for time‐varying network

Adaptive video coding algorithms can encode a video stream dynamically on the basis of the amount of bandwidth available in a network. In this paper, a novel approach for adaptive video coding based on look‐up tables has been proposed. On the basis of the network conditions, the proposed codec estimates quantization parameter (QP) and also the spatial resolution for a group of pictures from the look‐up table. Then, QP for each frame is estimated on the basis of the motion content in each frame. More motion leads to more burden of bits while encoding the motion vectors. Also, quantization noise is less prominent in high‐motioned frames than frames possessing low motion information because of the temporal masking phenomenon of the human eyes. So, the main trick of the proposed scheme is to assign higher QP to the frames having higher motion than those having less motion. This method also reduces the requirement of excess bits for encoding of motion vectors having higher motion content. Additionally, QP is adjusted on the basis of the buffer availability in order to avoid bit loss error. Experimental results show the efficacy of the proposed codec. Copyright © 2013 John Wiley & Sons, Ltd.     

压缩视频码流中运动矢量的提取

MPEG-2标准中，码流中的运动矢量代表了编码时相邻图像间的运动信息。提出一种针对MPEG-2中不同运动补偿方法的运动矢量两步法提取方法：首先选择性提取解码视频码流中的运动矢量，然后其进行滤波系数调整。分析结果表明经过简单滤波后的运动矢量符合显示时相邻图像间的运动信息。提取出的运动矢量经简单处理后可以直接应用于帧率增强。生成的图像保持了运动连贯性，显示效果良好，方法复杂度低。     

基于运动矢量场的运动目标区域检测

基于稀疏运动矢量场,提出一种动态背景下的运动 目标区域检测方法。根据运动矢量场特性分析进行全局运动 参数估计和全局运动补偿,实现动态场景中的背景校正;利用最大树数据结构, 基于运动矢量补偿误差分级表示视频帧中 运动基本一致的连通区域,进行运动区域初始分类;根据运动目标在空间上的连通性和运动 一致性的特点,选择区域相似性 度量准则,进行区域合并和滤波,将具有相似运动的连通区域合并,实现运动目标区域检测 。将检测出的运动目标区域作为 运动矢量外点反过来又应用于全局运动参数估计过程中,全局运动估计和运动目标区域检测 交替地进行,不仅加快了它们的 计算速度,同时也提高了它们计算和检测的准确性。实验结果表明,本文算法能较好地补偿 序列的全局运动,有效地检测出 局部目标运动区域。     

Performance analysis of motion-compensated de-interlacing systems.

A lot of research has been conducted on motion-compensated (MC) de-interlacing, but there are very few publications that discuss the performances of de-interlacing quantatively. The various methods are compared through their performance on known video sequences. Linear system analysis of interlaced video and de-interlacer are proposed in. It is well established that the performance of the MC methods outperform the fixed or motion-adaptive methods when the motion vectors used are reliable and true to the scene content. Being an open-loop process the performance of the MC de-interlacers degrade drastically when there are motion vector errors. In this paper, a linear system analysis of MC video upconversion systems is presented and the effects of motion vector accuracy on system performance are analyzed. We investigate the various factors that contribute to the motion vector inaccuracy, such as incorrect motion modelling, acceleration between the frames, and insufficient interpolation kernel.     

A Multi-Pass True Motion Estimation Scheme With Motion Vector Propagation for Frame Rate Up-Conversion Applications

This paper presents a robust true motion estimation algorithm, designated as MPMVP (Multi-pass and Motion Vector Propagation), to enhance the accuracy of the motion vector fields in frame rate up-conversion applications. The MPMVP uses a multi-pass scheme to progressively refine approximate motion vectors to true motion vectors based upon the motion information acquired in previous passes. The multi-pass motion estimation process uses a large block size to detect the motion vectors within the objects themselves and small block sizes to detect the motion vectors along the object boundaries. Actually, the block size is progressively reduced during the search process. When the motion vector of a block is considered to be sufficiently accurate for motion estimation purposes, the block is said to be converged and the local motion vector search process terminates. A novel technique, referred to as motion vector propagation, is then applied to propagate the motion vector of the converged block to its neighboring blocks. This technique not only ensures that neighboring motion vectors within the same object have a high degree of spatial correlation, but also accelerates the convergence of the motion vectors in the neighboring blocks and therefore reduces the overall computational time and expense of the multi-pass motion vector search procedure. A novel distortion criterion is proposed to enhance the tolerance of the traditional sum-of-absolute-difference measurement technique applied in the motion estimation scheme to noise and shadow effects. The experimental results demonstrate that the proposed true motion estimation algorithm outperforms the traditional full search, 3DRS and TCSBP algorithms in terms of both the smoothness of the generated motion vector fields and the visual quality of the up-converted frames.     

A low-cost unrestricted fast playback scheme for video streaming

In this brief, we propose a low cost fast-forward and fast-backward playback scheme for video streaming applications. By reshaping the ordinary linear encoded group-of-pictures (GOP) structure into a hierarchical structure, the transmission overhead caused by frame dependencies can be reduced. The resulting video streaming system can provide all directions video playback with any speed-up factor. A set-top box architecture that supports the binary tree structured GOP playback is also described.     

DCT-based video downscaling transcoder using split and merge technique.

For a conventional downscaling video transcoder, a video server has firstly to decompress the video, perform downscaling operations in the pixel domain, and then recompress it. This is computationally intensive. However, it is difficult to perform video downscaling in the discrete cosine transform (DCT)- domain since the prediction errors of each frame are computed from its immediate past higher resolution frames. Recently, a fast algorithm for DCT domain image downsampling has been proposed to obtain the downsampled version of DCT coefficients with low computational complexity. However, there is a mismatch between the downsampled version of DCT coefficients and the resampled motion vectors. In other words, significant quality degradation is introduced when the derivation of the original motion vectors and the resampled motion vector is large. In this paper, we propose a new architecture to obtain resampled DCT coefficients in the DCT domain by using the split and merge technique. Using our proposed video transcoder architecture, a macroblock is splitted into two regions: dominant region and the boundary region. The dominant region of the macroblock can be transcoded in the DCT domain with low computational complexity and re-encoding error can be avoided. By transcoding the boundary region adaptively, low computational complexity can also be achieved. More importantly, the re-encoding error introduced in the boundary region can be controlled more dynamically. Experimental results show that our proposed video downscaling transcoder can lead to significant computational savings as well as videos with high quality as compared with the conventional approach. The proposed video transcoder is useful for video servers that provide quality service in real-time for heterogeneous clients.     

Multimedia extensions for a 550-MHz RISC microprocessor

This paper describes circuits used to implement the motion video instructions (MVI) in the 550-MHz Alpha 21 164PC Microprocessor. The chip is fabricated in a 0.35-μm CMOS process and is the first implementation of the MVI instruction set in the Alpha architecture. The MVI instruction set, coupled with the high performance of the 550-MHz processor, delivers 30 frames/s digital video disk (DVD) playback with stereo-quality audio, enables video teleconferencing at 30 frames/s, and significantly improves the performance of MPEG encode algorithms     

针对多视角视频编码的快速运动估计方法

为了提高多视角视频编码系统中运动估计速度,提出了一种基于马尔科夫链模型的快速运动矢量估计算法。该算法充分利用帧间、视角间相关性,建立预测矢量状态集合。通过马尔科夫链模型的状态转移概率,对预测矢量进行提前测试。然后利用提前退出准则,实现快速运动矢量估计。实验结果表明,与单独采用全搜索算法比较,该算法对于多视角视频序列运动估计速度可以提高99%以上,PSNR平均降低0.16dB。提出的算法对多视角视频编码系统的运动估计效率提升明显。     

Motion estimation with low resolution distortion metric

Considering all possible candidate motion vectors in a given search area and calculating a distortion measure at every search position, as with the full-search block-matching motion estimation algorithm (FSBME), places a prohibitively high computational burden on the video encoder, making it unsuitable for real-time/portable video applications. To reduce computational complexity while maintaining accuracy, a new version of the reduced-bit, sum of absolute difference (RBSAD) algorithm is presented, which allows for optional correction to full resolution (FSBME) when appropriate. Analysis of a number of video sequence shows that this correction is required for blocks for which there is little motion between frames.     

An efficient gray search algorithm for the estimation of motionvectors

Motion vector estimation plays an important role in motion-compensated video coding. An efficient and fast search algorithm is proposed for the estimation of motion vectors. With the help of gray prediction, the algorithm can determine the motion vectors of image blocks quickly and correctly. Since the proposed algorithm performs better than other search algorithms [e.g. the three-step search (TSS), cross-search (CS), new three-step search (NTSS), four-step search (FSS), block-based gradient descent search (BBGDS), simple-and-efficient search (SES), prediction search (PS) and gray prediction search (GPS)], it is very beneficial in applications where the video coding speed is important     

A Hybrid Frame Concealment Algorithm for H.264/AVC

In packet-based video transmissions, packets loss due to channel errors may result in the loss of the whole video frame. Recently, many error concealment algorithms have been proposed in order to combat channel errors; however, most of the existing algorithms can only deal with the loss of macroblocks and are not able to conceal the whole missing frame. In order to resolve this problem, in this paper, we have proposed a new hybrid motion vector extrapolation (HMVE) algorithm to recover the whole missing frame, and it is able to provide more accurate estimation for the motion vectors of the missing frame than other conventional methods. Simulation results show that it is highly effective and significantly outperforms other existing frame recovery methods.     

基于可变块运动矢量的边信息生成算法*

在分布式视频编码系统中,针对图像中细节丰富的区域易造成严重的块效应,提出了一种基于可变块运动矢量的边信息生成算法。根据前后相邻关键帧对应块的相关性,将像素块分为保留块和运动块。对保留块直接作保留处理,对运动块中的像素块继续进行分割并计算子块的初始运动矢量,最后将所有对应块的运动矢量进行加权自适应运动补偿得到改进的边信息。实验结果表明,对于运动较剧烈复杂的视频序列,该算法能够提高边信息生成质量, 并且使得改进后的边信息PSNR值提高了0.98~1.33 dB。     

Fast motion estimation for surveillance video compression

In this article, novel approaches to perform efficient motion estimation specific to surveillance video compression are proposed. These includes (i) selective (ii) tracker-based and (iii) multi-frame-based motion estimation. In selective approach, motion vector search is performed for only those frames that contain some motion activity. In another approach, contrary to performing motion estimation on the encoder side, motion vectors are calculated using information of a surveillance video tracker. This approach is quicker but for some scenarios it degrades the visual perception of the video compared with selective approach. In an effort to speed up multi-frame motion estimation, we propose a fast multiple reference frames-based motion estimation technique for surveillance videos. Experimental evaluation shows that significant reduction in computational complexity can be achieved by applying the proposed strategies.