Object-based stereo video compression using fractals and shape-adaptive DCT

Based on the classical fractal video compression method, an improved object-based stereo video compression scheme with Shape-Adaptive DCT is proposed in this paper. Firstly, we use more effective macroblock partition scheme instead of classical quadtree partition scheme; thus reducing the block searching strategy. The stereo fractal video coding is proposed which matches the macroblock with two reference frames in left and right view results in increasing compression ratio and reducing bit rate when transmitting compressed stereo data. The stereo codec combines the Motion Compensation Prediction (MCP) and Disparity Compensation Prediction (DCP). Fractal coding is adopted and each object is encoded independently by a prior video segmentation alpha plane, which is defined exactly as in MPEG-4. The testing results with the nature monocular and stereo video sequences provide promising performances at low bit rate coding. We believe it will be a powerful and efficient technique for the object-based monocular and stereo video sequences coding.     

基于图割和视域相关性的立体图像分割

立体图像分割是对象基立体图像处理中的关键和难点。基于改进Grabcut图割算法和视域相关性,提出一种新的立体图像分割算法。首先基于改进Slic方法将左图像转换成超像素图像,然后基于Grabcut框架通过重新定义能量函数对其分割以提取出左图像目标。最后,基于左右图像的视域相关性通过融合颜色和纹理特征的轮廓匹配提取右图像目标。实验结果表明,与现有方法相比,所提算法能获得更高的分割效率和更准确的分割结果。     

A novel block truncation coding of color images using aquaternion-moment-preserving principle

Block truncation coding (BTC) is an efficient tool for image compression. To compress color-pixel blocks, a novel color BTC algorithm, called quaternion-moment block truncation coding (QMBTC), is presented. Analytical formulas for QMBTC, whose computation time is on the order of pixel block size, are derived by using quaternion arithmetic and the moment-preserving principle. The proposed color BTC algorithm can adaptively truncate a pixel block into one or two output classes according to the distribution of the color values inside the blocks. The experimental results show that the compression ratio is increased as compared with existing color BTC algorithms, and the picture quality of the reconstructed images is satisfactory. In addition, a post-BTC data compression scheme is proposed to further compress the subimage constructed by reproduction colors of truncated pixel blocks. Using a lookup table to display decoded data, this postprocessing scheme can output images acceptable to human eyes     

Stereo object tracking with fusion of texture,color and disparity information

A novel method for visual object tracking in stereo videos is proposed, which fuses an appearance based representation of the object based on Local Steering Kernel features and 2D color–disparity histogram information. The algorithm employs Kalman filtering for object position prediction and a sampling technique for selecting the candidate object regions of interest in the left and right channels. Disparity information is exploited, for matching corresponding regions in the left and right video frames. As tracking evolves, any significant changes in object appearance due to scale, rotation, or deformation are identified and embodied in the object model. The object appearance changes are identified simultaneously in the left and right channel video frames, ensuring correct 3D representation of the resulting bounding box in a 3D display monitor. The proposed framework performs stereo object tracking and it is suitable for application in 3D movies, 3D TV content and 3D video content captured by consuming stereo cameras. Experimental results proved the effectiveness of the proposed method in tracking objects under geometrical transformations, zooming and partial occlusion, as well as in tracking slowly deforming articulated 3D objects in stereo video.     

A wavelet implementation of the pioneering block-based disparity compensated predictive coding algorithm for stereo image pair compression

In this paper, we propose a novel, discrete wavelet transform (DWT) domain implementation of our previously proposed, pioneering block-based disparity compensated predictive coding algorithm for stereo image compression. Under the present research context we perform predictive coding in the form of pioneering block search in the sub-band domain. The resulting transform domain predictive error image is subsequently converted to a so-called wavelet-block representation, before being quantized and entropy coded by a JPEG-like CODEC. We show that the proposed novel implementation is able to effectively transfer the inherent advantages of DWT-based image coding technology to efficient stereo image pair compression. At equivalent bit rates, the proposed algorithm achieves peak signal to noise ratio gains of up to 5.5 dB, for reconstructed predicted images, as compared to traditional and state of the art DCT and DWT-based predictive coding algorithms.     

Vector Lifting Schemes for Stereo Image Coding

Many research efforts have been devoted to the improvement of stereo image coding techniques for storage or transmission. In this paper, we are mainly interested in lossy-to-lossless coding schemes for stereo images allowing progressive reconstruction. The most commonly used approaches for stereo compression are based on disparity compensation techniques. The basic principle involved in this technique first consists of estimating the disparity map. Then, one image is considered as a reference and the other is predicted in order to generate a residual image. In this paper, we propose a novel approach, based on vector lifting schemes (VLS), which offers the advantage of generating two compact multiresolution representations of the left and the right views. We present two versions of this new scheme. A theoretical analysis of the performance of the considered VLS is also conducted. Experimental results indicate a significant improvement using the proposed structures compared with conventional methods.     

基于方向性小波子树分形图像编码的改进算法

本文在分析混合分形零树小波图像编码算法(FZW)优缺点的基础上，提出一种新的基于方向性小波子树的分形图像编码算法。该算法结合零树小波编码和分形编码，通过在匹配搜索过程中使用方向性range和domain子树，提高匹配精度，改善了传统分形小波图像压缩中的方块效应，更大限度的保留了图像的边缘信息。实验结果表明，该算法在提高压缩比和去除图像的方块效应方面，均取得了良好的效果。     

一种基于立体视邻接帧时空相关性的最小代价函数帧估计算法

为了在立体视频序列编码中获得高的压缩率,需要对立体视频序列中一个视的序列按传统方法进行独立编码;另一个视的序列中,只对其中一些参考帧(I帧或P帧)按视差补偿预测的方法进行编码,其余帧不进行编码和传输,而在解码端用立体视帧估计的方法得到重建.本文提出了一种基于立体视中邻接帧在图像、视差场和运动矢量场之间高度相关性的方法.对于因遮挡而缺乏估计的区域,则结合了图像强度的连续性和运动,视差矢量的分布特性,构造了代价方程并估计出该部分的运动矢量及强度值.实验证明,重建出来的帧图像在视觉和信噪比意义上均具有较好的效果.     

应用分层MRF/GRF模型的立体图像视差估计及分割

视差估计与分割是立体图像编码及立体视觉匹配的核心问题,本文提出一种基于分层MRF/GRF模型和交叠块匹配(HMOM)视差估计算法以及结合主动轮廓模型的视差分割提取算法.该混合视差估计方法,可得到光滑准确,且具有清晰边缘的视差场;并便于用主动轮廓模型提取感兴趣对象(OOI)的视差轮廓.与通常的变尺寸块匹配(VSBM)相比,本算法得到的视差补偿图像的峰值信噪比可提高2.5dB左右.本文得到的视差场及对应的轮廓可进一步用于立体图像编码以及视频对象分割.     

Edge-preserving disparity estimation and disparity-compensated intermediate view reconstruction for stereo images

A constrained disparity estimation method is proposed which uses a directional regularization technique to efficiently preserve edges for stereo image coding. The proposed method smoothes disparity vectors in smooth regions and preserves edges in object boundaries well, without creating an oversmoothing problem. The differential pulse code modulation (DPCM) technique for disparity map coding is used prior to entropy coding, in order to improve the overall coding efficiency. The proposed disparity estimation method can also be applied to intermediate view reconstruction. Intermediate views between a left image and a right image provide reality and natural motion parallax to multiviewers. Intermediate views are synthesized by appropriately exploiting an interpolation or an extrapolation technique according to the characteristics of each region after identifying the regions as occluded regions, normal regions, and regions having ambiguous disparities.The experimental results show that the proposed disparity estimation method gives close matches between a left image and a right image and improves coding efficiency. In addition, we can subjectively confirm that the application of our proposed intermediate view reconstruction method leads to satisfactory intermediate views from a stereo image pair.This work was supported by the Korea Institute of Science and Technology (KIST) under Grant No. 99HI-054.     

Very low bit-rate coding algorithm for stereo video withspatiotemporal HVS model and binary correlation disparity estimator

A strategy for efficiently coding stereo video sequences is investigated. To fully utilize the suppression and the contrast sensitivity property of the human visual system, a novel coding scheme with two special mechanisms, the spatiotemporal HVS model and the binary correlation disparity estimator, is proposed to efficiently reduce the video signal redundancy and the computational complexity, while maintaining a high subjective image quality. Compared with existing stereo video coding systems, the proposed coding scheme supports a lower transmission bit rate and has less computational complexity. The simulation results also show that the subjective image quality of the reconstructed full color stereo sequences at 0.25-0.4 bits per pixel (bpp) is satisfactory     

多层离散小波变换系数的稀疏向量构造的改进

融合离散小波变换和压缩感知的图像压缩方案很好避免了采用离散余弦变换和压缩感知时所带来的块效应,但当前基于单层离散小波变换的算法压缩比较低,基于多层离散小波变换的算法重构质量不佳。为了解决这些不足,根据离散小波变换系数的特点,对现有基于多层离散小波变换的算法提出了改进。图像经小波变换后,保留图像最高层低频系数,高频系数的构造方式给予适当改进。实验结果表明,与现有算法相比,重构图像的PSNR值得到2～4 dB提高。     

Light field compression using disparity-compensated lifting and shape adaptation.

We propose disparity-compensated lifting for wavelet compression of light fields. With this approach, we obtain the benefits of wavelet coding, such as scalability in all dimensions, as well as superior compression performance. Additionally, the proposed approach solves the irreversibility limitations of previous light field wavelet coding approaches, using the lifting structure. Our scheme incorporates disparity compensation into the lifting structure for the transform across the views in the light field data set. Another transform is performed to exploit the coherence among neighboring pixels, followed by a modified SPIHT coder and rate-distortion optimized bitstream assembly. A view-sequencing algorithm is developed to organize the views for encoding. For light fields of an object, we propose to use shape adaptation to improve the compression efficiency and visual quality of the images. The necessary shape information is efficiently coded based on prediction from the existing geometry model. Experimental results show that the proposed scheme exhibits superior compression performance over existing light field compression techniques.     

Optimization of quadtree segmentation and hybrid two-dimensionaland three-dimensional motion estimation in a rate-distortion framework

A rate-distortion framework is used to define a very low bit-rate coding scheme based on quadtree segmentation and optimized selection of motion estimators. This technique achieves maximum reconstructed image quality under the constraint of a target bit rate for the coding of the vector field and segmentation information. First, a complete scheme is proposed for hybrid two-dimensional (2-D) and three-dimensional (3-D) motion estimation and compensation. The quadtree object segmentation is optimized for hybrid motion estimation in the rate-distortion sense. This scheme adapts to the depth of the quadtree and the technique used for motion estimation for each leaf of the tree. A more sophisticated technique, adapted to the requirements of a very low bit-rate coder, is also proposed which also considers the transmission of the prediction error corresponding to the particular choice of the motion estimator. Based on these coding schemes, two versions of a very low bit-rate image sequence coder are developed. Experimental results illustrating the performance of the proposed techniques in very low bit-rate image sequence coding application areas are presented and evaluated     

立体图像序列的压缩方法

首先介绍了立体视觉的基本原理，然后对立体图像的压缩方法分四类进行了综述。对其中用于立体图像序列的两种主要方法：基于“块”匹配的立体图像压缩方法和基于物体的立体图像压缩方法进行了深入探讨。通过对已有成果进行总结和分类，剖析了两种方法的优、缺点，并提出了一些还需要深入研究的问题，如：残差图像编码、遮挡检测、更精确的场景分割等。     

Recurrent pattern matching based stereo image coding using linear predictors

The Multidimensional Multiscale Parser (MMP) is a pattern-matching-based generic image encoding solution which has been investigated earlier for the compression of stereo images with successful results. While first MMP-based proposals for stereo image coding employed dictionary-based techniques for disparity compensation, posterior developments have demonstrated the advantage of using predictive methods. In this paper, we focus on recent investigations on the use of predictive methods in the MMP algorithm and propose a new prediction framework for efficient stereo image coding. This framework comprises an advanced intra directional prediction model and a new linear predictive scheme for efficient disparity compensation. The linear prediction model is the main novelty of this work, combining adaptive linear models estimated by least-squares algorithm with fixed linear models provided by the block-matching algorithm. The performance of the proposed intra prediction and disparity compensation methods when applied in an MMP encoder has been evaluated experimentally. Comparisons with the current stereo image coding standards showed that the proposed MMP algorithm significantly outperforms the Stereo High Profile of H.264/AVC standard. In addition, it presents a competitive performance relative to the MV-HEVC standard. These results also suggest that current stereo image coding standards may benefit from the proposed linear prediction scheme for disparity compensation, as an extension to the omnipresent block-matching solution.     

Human visual system and segment-based disparity estimation

In this paper, an efficient segment-based disparity estimation algorithm is proposed, based on wavelet transform and human visual system. The core idea is to estimate disparity not only from the left and right images but also from their decomposed sub-bands up to a certain level. The stereo image pair is divided into the segments of homogeneous color. Instead of assigning a disparity value to each pixel inside a segment, a disparity plane is assigned to each segment and the stereo matching problem is formulated as an energy minimization problem in the segmented domain. The optimal disparity plane labeling is approximated by applying belief propagation, which assigns the corresponding disparity plane to each segment. The obtained disparity maps are merged into a single disparity map using the human visual system model. Experiments with stereo image pairs show the validity of the proposed method.     

基于块的背景重构算法研究及应用

针对背景静止的立体视频压缩编码,提出了一种新的背景重构和前景提取的视频分割算法。首先在利用帧差法得到左右通道前景运动区域的基础上,分别对前景运动区域进行外接矩形块的初始化,并以初始块为单位进行左右通道的背景重构。然后,对左右通道视频中的每一帧序列均与重构的背景图像做差处理并得到运动前景。实验结果表明,此算法可以达到精确的分割效果,且视频分割后可以显著减小立体视频匹配时间,进而减少数据传输量和存储空间。