一种新的视点间预测结构

多视点视频编码除应具有较高的编码效率外,还应该包括后向兼容性、时间随机访问和视点可分级性等,这些都主要取决于所采用的预测结构。目前所提供的多视点视频编码(Joint Multi-view Video Coding, JMVC)采用固定的视点间预测结构,难以适应复杂情况的多视点视频编码。该文综合考虑编码效率和用户随机访问等因素,根据多视点视频相关性分析自适应调整视点间预测结构,以获得较好的编码综合性能。试验结果表明,与JMVC相比,该文的方法在提高编码效率的同时,有较好的随机访问性能。     

Multi-view video coding with view interpolation prediction for 2D camera arrays

An efficient compression algorithm for multi-view video sequences, which are captured by two-dimensional (2D) camera arrays, is proposed in this work. First, we propose a novel prediction structure, called three-dimensional hierarchical B prediction (3DHBP), which can efficiently reduce horizontal inter-view redundancies, vertical inter-view redundancies, and temporal redundancies in multi-view videos. Second, we develop a view interpolation scheme based on the bilateral disparity estimation. The interpolation scheme yields high quality view frames by adapting disparity estimation and compensation procedures using the information in neighboring frames. Simulation results demonstrate that the proposed multi-view video coding algorithm provides significantly better rate–distortion (R–D) performance than the conventional algorithm, by employing the 3DHBP structure and using interpolated view frames as additional reference frames.     

Efficient view-temporal prediction structures for multi-view video coding

To compress multi-view video, spatial redundancy between adjacent view sequences as well as temporal redundancy need to be eliminated. View-temporal prediction structures are proposed, which can be adjusted to various characteristics of multi-view videos. The proposed prediction structure achieves better coding performance than the reference prediction structure for the standardisation of multi-view video coding.     

Joint depth-motion dense estimation for multiview video coding

The multiview video coding (MVC) extension of H.264/MPEG-4 AVC [1] is one of the most promising visual encoders for three-dimensional television and free viewpoint video applications. In this paper, we propose a joint dense motion/disparity estimation algorithm, designed to replace the classical temporal/inter-view unit within MVC, which uses a block-based motion/disparity estimation. The motion vector fields and the disparity vector fields are therefore simultaneously derived using the stereo-motion consistency constraint in a set theoretic convex optimization framework. The obtained displacement vector fields are then jointly segmented by minimizing a rate-distortion cost function, in line with the multiple reference frame strategy used in H.264/MPEG-4 AVC. Experimental results demonstrate the benefits of the proposed method compared to the separated dense estimation scheme or the block-based estimation technique.     

应用于多视点视频压缩的多参考B帧快速编码算法

基于H.264平台就多视点视频压缩中多参考帧技术存在计算复杂度高的现状,根据多视点视频序列特性,提出了一种针对多参考B帧的快速多视点视频编码压缩算法。新算法改进了多参考帧列表策略,并结合多视点视频序列的时空域相关性特性,提出了适用于多视点视频编码的参考帧列表调整策略。此外,还针对多种块模式编码引入了块模式预判策略。新算法在保证率失真性能的基础上,有效地降低了计算复杂度,提高了多视点视频编码速度。虽然本文是基于M-Picture编码结构实现了新算法,但它同样适用于其它应用多参考B帧预测技术的多视点视频编码结构中。     

A novel multi-view image coding scheme based on view-warping and 3D-DCT

Efficient compression of multi-view images and videos is an open and interesting research issue that has been attracting the attention of both academic and industrial world during the last years. The considerable amount of information produced by multi-camera acquisition systems requires effective coding algorithms in order to reduce the transmitted data while granting good visual quality in the reconstructed sequence. The classical approach of multi-view coding is based on an extension of the H.264/AVC standard, still based on motion prediction techniques. In this paper we present a novel approach that tries to fully exploit the redundancy between different views of the same scene considering both texture and geometry information. The proposed scheme replaces the motion prediction stage with a 3D warping procedure based on depth information. After the warping step, a joint 3D-DCT encoding of all the warped views is provided, taking advantage of the strong correlation among them. Finally, the transformed coefficients are conveniently quantized and entropy coded. Occluded regions are also taken into account with ad-hoc interpolation and coding strategies. Experimental results performed with a preliminary version of the proposed approach show that at low bitrates it outperforms the H.264 MVC coding scheme on both real and synthetic datasets. Performance at high bitrates are also satisfactory provided that accurate depth information is available.     

多视点视频编码中的视频目标提取与视差匹配

在单通道视频目标分割算法的基础上,首先提出一种多视点视频编码中的视频目标提取方案.然后提出一种基于多级块匹配视差估计算法.应用Moravec算子对视差矢量图进行检测,提取特征视差矢量并对其进行精确度检验.若特征视差矢量不够精确,则其所在块将重新进行小范围的块匹配.若块匹配误差仍太大则将其分裂为四个子块重新匹配.与传统块匹配算法相比,新方法能获得较为精确和平滑的视差场,提高重建后的图像质量.     

基于多视点伪序列的光场图像压缩

近年来，作为一种能够提供更富有沉浸感的多媒体媒质，光场图像（Light Field Image，LFI）引起广泛的关注。针对光场图像数据量巨大的问题，本文提出了一种基于多视点伪序列的光场图像高效压缩方案。在编码端，所提方法首先将光场相机捕获得到的原始光场图像根据相机的微透镜阵列分解成子孔径图像。接着根据子孔径图像存在较强视点内和视点间相关性，选取部分子孔径图像进行多视点伪序列构建，基于MV-HEVC设计适用于多视点伪序列的预测编码结构进行编码。在解码端，所提方法基于已解码多视点伪序列通过视频帧插值方法重建出未编码传输的子孔径视图，从而重建出全部光场图像。实验结果表明本文所提算法优于现有基于视差引导稀疏编码的光场图像压缩方法，BD-rate平均节约18.5%，BD-PSNR平均提高1.28dB。      

多视点视频编码中图像分组技术的研究

在多视点视频编码中,引入了视点间运动补偿预测技术,而在视点间预测过程中,由于视差矢量的使用,使得GoP长度对编码效率的影响与一般H.264/AVC编码有所不同。在分析了可分级B帧的预测结构对多视点视频编码性能的影响基础上,通过对不同视频序列设置不同GoP的大小,经过多次反复实验,而后对实验结果进行分析,找到了在一定数值范围内的GoP。通过该GoP得到了较好的PSNR和模值较小的MV或者DV,节约了一定的编码时间。     

A motion vector prediction method for multi-view video coding

This paper proposes a new motion vector (MV) prediction method in multi-view video coding (MVC). In order to exploit the information in adjacent views, inter-view MVs as well as temporal MVs are used in conventional MVC. Since the inter-view MVs are usually uncorrelated with the temporal MVs and most neighboring partitions have temporal MVs only, the conventional DPCM coding gain of inter-view MV is very low and thus the inter-view MVs are seldom selected. In order to increase the probability of inter-view MV selection, we define a virtual inter-view MV which can be generated from temporal MVs. Then, an inter-view MV is predicted using these neighboring virtual inter-view MVs, leading to less prediction error than using the temporal MVs. As a result, bit-rates are decreased by up to 9% for the view-temporal prediction structure.     

一种基于双域拉格朗日插值的视频错误隐藏方法

本文提出了一种基于双域拉格朗日插值的错误隐藏方法,编码采用H.264标准,分别在时域和空域进行插值获得两个运动矢量,通过建立插值系数表构造混合的插值模型,经过边缘匹配算法判断后获得最优的运动矢量作为丢失宏块的运动矢量.为了减小网络突发错误对插值效果的影响,本文在编码端提出了一种类交织的slice划分方法,实验证明本文方法由于充分利用了时域和空域的信息冗余,取得了较好效果,在15％网络丢包率的环境下平均PSNR比空域拉格朗日插值方法高0.5dB~1dB左右.     

Very low bit-rate wavelet video coding

An image sequence coding scheme for very low bit-rate coding is presented. The scheme is based on spatial redundancy reduction via the new edge-sensitive subband coding method and temporal redundancy reduction via windowed overlapped block-matching motion compensation. In this scheme, only significant regions of difference images are coded. Thus, the computational cost can be kept low. Significant regions are considered to be regions where motion is present (temporal activity) and/or regions where the image is very detailed (spatial activity). The significant regions are determined by an adaptive strategy based on both the motion vectors and an edge map of the current image. The scheme has the main advantage that there are no blocking effects in the decoded images at all due to the properties of both the temporal and the spatial coding used. The overall visual performance of the coding scheme is very good     

Fast color correction for multi-view video by modeling spatio-temporal variation

In multi-view video, a number of cameras capture the same scene from different viewpoints. Color variations between the camera views may deteriorate the performance of multi-view video coding or virtual view rendering. In this paper, a fast color correction method for multi-view video is proposed by modeling spatio-temporal variation. In the proposed method, multi-view keyframes are defined to establish the spatio-temporal relationships for accurate and fast implementation. For keyframes, accurate color correction is performed based on spatial color discrepancy model that disparity estimation is used to find correspondence points between views, and linear regression is performed on these sets of points to find the optimal correction coefficients. For non-keyframes, fast color correction is performed based on temporal variations model that time-invariant regions are detected to reflect the change trends of correction coefficients. Experimental results show that compared with other methods, the proposed method can promote the correction speed greatly without noticeable quality degradation, and obtain higher coding performance.     

Coding statistics based fast mode decision for multi-view video coding

Reduction of high computational complexity of multi-view video coding (MVC) is necessary for realization in consumer electronics. Since mode decision is one of the key computational bottlenecks of multi-view video encoders, this paper proposes a coding statistics based fast mode decision algorithm. First of all, a rate–distortion cost based fast DIRECT mode decision algorithm early terminates the mode decision process if possible. Next, the candidates for Inter modes are reduced by taking the advantage of the correlation between an optimal mode and motion cost. The proper thresholds to reduce the candidates for the above two fast algorithms can be easily derived from exponential functions at run time. Finally, motion vector difference based motion characteristics is referred to further speed up the mode decision process of Inter modes. The experimental results show that the proposed scheme reduces up to 70.82% of encoding time with negligible degradation of RD performance.     

Iterative search strategy with selective bi-directional prediction for low complexity multiview video coding

The multiview video coding (MVC) extension of H.264/AVC is the emerging standard for compression of impressive 3D and free-viewpoint video. The coding structure in MVC adopts motion and disparity estimation to exploit temporal and inter-view dependencies in MVC. It results in a considerable increase in encoding complexity. Most of the computational burden comes from uni-directional and bi-directional prediction. In this paper, an iterative search strategy is designed to speed up the uni-directional prediction in MVC. It can work with an adaptive search range adjustment through a confidence measure of a loop constraint to obtain both motion and disparity vectors jointly. Furthermore, a selective bi-directional prediction algorithm is proposed to enhance the coding performance by analyzing the statistical characteristics of bi-directional prediction in MVC. Experimental results demonstrate that, by using the proposed fast search, the temporal and inter-view redundancies of multiview video can be eliminated sufficiently with low complexity.     

An efficient rate–distortion optimization method for dependent view in MV-HEVC based on inter-view dependency

Rate–distortion optimization (RDO) is utilized to select the optimal coding parameters in multi-view video coding (MVC), which employs a Lagrange multiplier to balance the relationship between the distortion and the bitrate. In this paper, an efficient RDO method for the dependent view (DV) in multi-view video (MVV) is proposed based on inter-view dependency. First of all, by investigating the sources of the distortion in the DV, a new distortion model for the DV is established. In addition, based on the proposed distortion model, an efficient Lagrangian multiplier decision for B frame is proposed by considering the inter-view dependency. Finally, the optimized Lagrangian multiplier for P frame is designed using the scaling factor which is deduced to have a linear relationship with the disparity between I frame and P frame. Experiment results demonstrate that compared with the original HTM-16.0 encoder, the proposed overall method reduces 12.19% BD-rate for the DV on average, bringing 0.40 dB BD-PSNR gain.     

Soft Information Combining for Turbo-MIMO Packet Retransmission

We studied three types of retransmission scheme for turbo-MIMO packet: Chase combining, incremental redundancy, and soft information combining, these three schemes are suitable for different situations. The MIMO channel in each retransmission is correlated in temporal dimension, and a standard method is utilized to simulate the retransmission channel model. Interleaving can shuffle the MIMO channel artificially, so the outage capacity of channel with interleaving is much better than the capacity without interleaving. If using different interleaver in retransmission, the receiver can only combine the retransmitted data after MIMO symbol demapping, we call it “soft information combing”. We find soft information combing is much useful in the true environment, we also find coding gain of incremental redundancy over Chase combining in most cases.

Disparity field and depth map coding for multiview 3D image generation

In the present paper techniques are examined for the coding of the depth map and disparity fields for stereo or multiview image communication applications. It is assumed that both the left and right channels of the multiview image sequence are coded using block- or object-based methods. A dynamic programming algorithm is used to estimate a disparity field between each stereo image pair. Depth is then estimated and occlusions are optionally detected, based on the estimated disparity fields. Spatial interpolation techniques are examined based on the disparity/depth information and the detection of occluded regions using either stereoscopic or trinocular camera configurations. It is seen that the presence of a third camera at the transmitter site improves the estimation of disparities, the detection of occlusions and the accuracy of the resulting spatial interpolation at the receiver. Various disparity field and depth map coding techniques are then proposed and evaluated, with emphasis given to the quality of the resulting intermediate images at the receiver site. Block-based and wireframe modeling techniques are examined for the coding of isolated depth or disparity map information. Further, 2D and 3D motion compensation techniques are evaluated for the coding of sequences of depth or disparity maps. The motion fields needed may be available as a byproduct of block-based or object-based coding of the intensity images. Experimental results are given for the evaluation of the performance of the proposed coding and spatial interpolation methods.     

Selective Disparity Estimation and Variable Size Motion Estimation Based on Motion Homogeneity for Multi-View Coding

Multi-view video coding (MVC) is an ongoing standard in which variable size disparity estimation (DE) and motion estimation (ME) are both employed to select the best coding mode for each macroblock (MB). This technique achieves the highest possible coding efficiency, but it results in extremely large encoding time which obstructs it from practical use. In this paper, a fast DE and ME algorithm based on motion homogeneity is proposed to reduce MVC computational complexity. The basic idea of the method is to utilize the spatial property of motion field in prediction where DE and variable size ME are needed, and only in these regions DE and variable size ME are enabled. The motion field is generated by the corresponding motion vectors (MVs) in spatial window. Simulation results show that the proposed algorithm can save 63% average computational complexity, with negligible loss of coding efficiency.      

A new fast motion estimation algorithm based on the loop–epipolar constraint for multiview video coding

There lie geometric constraints between neighboring frames in multiview video sequences. The geometric constraints are valuable for reducing spatial and temporal redundancy in multiview video coding (MVC). In this paper, we propose a new fast motion estimation algorithm based on the loop–epipolar constraint which combines loop and epipolar constraints. A practical search technique is designed according to the characteristics of the loop–epipolar constraint. Experimental results show that the proposed algorithm is efficient for sequences under different multiview camera setups.