Optimal-correlation-based reconstruction for distributed compressed video sensing

Distributed compressed video sensing (DCVS) is a framework that integrates both compressed sensing and distributed video coding characteristics to achieve a low-complexity video coding. However, how to design an efficient joint reconstruction by leveraging more realistic signal models is still an open challenge. In this paper, we present a novel optimal-correlation-based reconstruction method for compressively sampled videos from multiple measurement vectors. In our method, the sparsity is mainly exploited through inter-signal correlations rather than the traditional frequency transform, wherein the optimization is not only over the signal space to satisfy data consistency but also over all possible linear correlation models to achieve minimum-l₁-norm correlation noise. Additionally, a two-phase Bregman iterative based algorithm is outlined for solving the optimization problem. Simulation results show that our proposal can achieve an improved reconstruction performance in comparison to the conventional approaches, and especially, offer a 0.7–9.9 dB gain in the average PSNR for DCVS.     

Video compressed sensing reconstruction based on structural group sparsity and successive approximation estimation model

The existing video compressed sensing (CS) algorithms for inconsistent sampling ignore the joint correlations of video signals in space and time, and their reconstruction quality and speed need further improvement. To balance reconstruction quality with computational complexity, we introduce a structural group sparsity model for use in the initial reconstruction phase and propose a weight-based group sparse optimization algorithm acting in joint domains. Then, a coarse-to-fine optical flow estimation model with successive approximation is introduced for use in the interframe prediction stage to recover non-key frames through alternating optical flow estimation and residual sparse reconstruction. Experimental results show that, compared with the existing algorithms, the proposed algorithm achieves a peak signal-to-noise ratio gain of 1–3 dB and a multi-scale structural similarity gain of 0.01–0.03 at a low time complexity, and the reconstructed frames not only have good edge contours but also retain textural details.     

Correlation estimation from compressed images

This paper addresses the problem of correlation estimation in sets of compressed images. We consider a framework where the images are represented under the form of linear measurements due to low complexity sensing or security requirements. We assume that the images are correlated through the displacement of visual objects due to motion or viewpoint change and the correlation is effectively represented by optical flow or motion field models. The correlation is estimated in the compressed domain by jointly processing the linear measurements. We first show that the correlated images can be efficiently related using a linear operator. Using this linear relationship we then describe the dependencies between images in the compressed domain. We further cast a regularized optimization problem where the correlation is estimated in order to satisfy both data consistency and motion smoothness objectives with a Graph Cut algorithm. We analyze in detail the correlation estimation performance and quantify the penalty due to image compression. Extensive experiments in stereo and video imaging applications show that our novel solution stays competitive with methods that implement complex image reconstruction steps prior to correlation estimation. We finally use the estimated correlation in a novel joint image reconstruction scheme that is based on an optimization problem with sparsity priors on the reconstructed images. Additional experiments show that our correlation estimation algorithm leads to an effective reconstruction of pairs of images in distributed image coding schemes that outperform independent reconstruction algorithms by 2–4 dB.     

CVS中基于视频运动特征的多假设-双稀疏重构算法

针对目前视频压缩感知重构算法对不同特征的视频序列重构质量参差不齐的问题,结合双稀疏对轮廓、细节的高清晰重构以及多假设算法对高频噪声有效抑制的优点,本文提出一种基于视频运动特征的多假设-双稀疏重构算法（VF-MH-DSR）.基本思路是基于每个视频组（GOP）的运动特征,采取相应的多假设-双稀疏重构策略.首先给出一种观测域多维度参考帧的多假设重构算法（MD-MRF-MH）及其最优相似块个数设置方案;然后给出一种像素域多假设参考帧的重构算法（PD-MRF-MH）及一种高性能双匹配准则;最后介绍了视频信号运动特征判定方案及多假设-双稀疏重构的具体实现方案.仿真实验表明,本文所提多假设-双稀疏重构算法相对于目前较好的多假设预测重构算法2sMHR及组稀疏重构算法SSIM-InterF-GSR,重构性能平均提升了1.98dB和0.84dB.     

Correlation based universal image/video coding loss recovery

Coding artifacts are annoying in highly compressed signals. Most of the existing artifact reduction methods are designed for one specific type of artifacts, codecs, and bitrates, which are complex and exclusive for one type of artifact reduction. Since both the compressed image/video and the coding error contain information of the original signal, they are highly correlated. Therefore, we try to recover some lost data based on the correlation between the compressed signal and the coding error, and introduce a novel and universal artifact reduction method. Firstly, according to the spatial correlation among pixels, a pixel-adaptive anisotropic filter is designed to reconstruct the distorted signal. Next, a globally optimal filter is designed to further recover the coding loss. Experimental results demonstrate that within an extensive range of bitrates, the proposed method achieves about 0.8 dB, 0.45 dB, 0.3 dB, and 0.2 dB on average of PSNR improvement for JPEG, MPEG4, H.264/AVC, and HEVC compressed signals, respectively.     

视频压缩感知中基于结构相似的帧间组稀疏表示重构算法研究

基于视频帧内图像的非局部相似性和帧间信号的相关性,本文提出了一种基于结构相似的帧间组稀疏表示重构算法（SSIM-InterF-GSR）,有效地提高了视频压缩感知的重构性能.在SSIM-InterF-GSR算法中,提出以结构相似度（SSIM）作为相似块匹配准则,在当前帧和参考帧内搜索匹配块生成相似块组,以相似块组的稀疏性作为正则项重构当前帧.同时,还提出了阶梯递减匹配块个数调整方案用于SSIM-InterF-GSR重构算法的迭代过程.仿真结果表明,相比于目前最好的视频压缩感知重构算法（Up-Se-AWEN-HHP）,本文算法获得了更好的重构质量,最多可提升4~5dB.     

帧间自适应压缩感知算法在视频编码中的应用

阐述了压缩感知的理论框架,分析了视频信号帧间相关性特点,提出了一种帧间自适应压缩感知的视频编码算法。本方法中,利用视频差值信号的特点建立自适应感知模型,自适应的选择稀疏域和重构域对信号进行压缩感知恢复,在空域稀疏度较强的情况下选择空域作为稀疏域和重构域,在空域稀疏度较差的情况下选择小波域作为稀疏域和重构域。用测试视频进行了仿真分析,结果表明该算法能够取得较好的效果。     

An automatic region-based video sequence codec based on fractal compression

In order to improve the performance of fractal video coding, we explore a novel fractal video sequences codec with automatic region-based functionality. To increase the quality of decoding image, intra frame coding, deblocking loop filter and sub-pixel block matching are applied to the codec. An efficient searching algorithm is used to increase the compression ratio and encoding speed. Automatic region-based fractal video sequences coding reduces coding stream greatly. Experimental results indicate that the proposed algorithm is more robust, and provides much less encoding time and bitrate while maintaining the quality of decompression image than the conventional CPM/NCIM method and other related references. We compare the proposed algorithm with three algorithms in Refs. [24], [25], [26], and the results of all these four algorithms are compared with H.264. The bitrate of the proposed algorithm is decreased by 0.11% and the other algorithms are increased by 4.29%, 6.85% and 11.62%, respectively. The average PSNR degradations of the four algorithms are 0.71 dB, 0.48 dB, 0.48 dB and 0.75 dB. So the bitrate of the proposed algorithm is decreased and the other algorithms are increased. At the meantime the compression time is reduced greatly, about 79.19% on average. The results indicate that, on average, the proposed automatic region-based fractal video sequences coding system can save compression time 48.97% and bitrate 52.02% with some image quality degradation in comparison with H.264, since they are all above 32 dB and the human eyes are insensitive to the differences.     

Free-viewpoint depth image based rendering

In 3D TV research, one approach is to employ multiple cameras for creating a 3D multi-view signal with the aim to make interactive free-viewpoint selection possible in 3D TV media. This paper explores a new rendering algorithm that enables to compute a free-viewpoint between two reference views from existing cameras. A unique property is that we perform forward warping for both texture and depth simultaneously. Advantages of our rendering are manyfold. First, resampling artifacts are filled in by inverse warping. Second, disocclusions are processed while omitting warping of edges at high discontinuities. Third, our disocclusion inpainting approach explicitly uses depth information. We obtain an average PSNR gain of 3 dB and 4.5 dB for the ‘Breakdancers’ and ‘Ballet’ sequences, respectively, compared recently published results. Moreover, experiments are performed using compressed video from surrounding cameras. The overall system quality is dominated by rendering quality and not by coding.     

SIFT-flow-based color correction for multi-view video

During the multi-view video acquisition, color variation across the views tends to be incurred due to different camera positions, orientations, and local lighting conditions. Such color variation will inevitably deteriorate the performance of the follow-up multi-view video processing, such as multi-view video coding (MVC). To address this problem, an effective color correction algorithm, called the SIFT flow-based color correction (SFCC), is proposed in this paper. First, the SIFT-flow technique is used to establish point-to-point correspondences across all the views of the multi-view video. The average color is then computed based on those identified common corresponding points and used as the reference color. By minimizing the energy of the difference yielded between the color of those identified common corresponding points in each view with respect to the reference color, the color correction matrix for each view can be obtained and used to correct its color. Experimental results have shown that the proposed SFCC algorithm is able to effectively eliminate the color variation inherited in multi-view video. By further exploiting the developed SFCC algorithm as a pre-processing for the MVC, extensive simulation results have shown that the coding efficiency of the color-corrected multi-view video can be greatly improved (on average, 0.85 dB, 1.27 dB and 1.63 dB gain for Y, U, and V components, respectively), compared with that of the original multi-view video without color correction.     

Exploiting quantization and spatial correlation in virtual-noise modeling for distributed video coding

Aiming for low-complexity encoding, video coders based on Wyner–Ziv theory are still unsuccessfully trying to match the performance of predictive video coders. One of the most important factors concerning the coding performance of distributed coders is modeling and estimating the correlation between the original video signal and its temporal prediction generated at the decoder.One of the problems of the state-of-the-art correlation estimators is that their performance is not consistent across a wide range of video content and different coding settings. To address this problem we have developed a correlation model able to adapt to changes in the content and the coding parameters by exploiting the spatial correlation of the video signal and the quantization distortion.In this paper we describe our model and present experiments showing that our model provides average bit rate gains of up to 12% and average PSNR gains of up to 0.5 dB when compared to the state-of-the-art models. The experiments suggest that the performance of distributed coders can be significantly improved by taking video content and coding parameters into account.     

Wyner–Ziv-based bidirectionally decodable video coding

In this paper, we propose a novel Wyner–Ziv-based video compression scheme which supports encoding a new type of inter frame called ‘M-frame’. Different from traditional multi-hypothesis inter frames, the M-frame is specially compressed with its two neighbor frames as reference at the encoder, but can be identically reconstructed by using any one of them as prediction at the decoder. Based on this, the proposed Wyner–Ziv-based bidirectionally decodable video compression scheme supports decoding the frames in a video stream in both temporal order and reverse order. Unlike the other schemes which support reverse playback, our scheme achieves the reversibility with low extra cost of storage and bandwidth. In error-resilient test, our scheme outperforms H.264 based schemes up to 3.5 dB at same bit rate. The proposed scheme also provides more flexibility for stream switching.     

基于自适应PCA稀疏基底的分布式视频压缩感知重构

为了提高分布式视频压缩感知（Distributed Video Compressive Sensing,DVCS）的率失真性能,本文提出利用自适应稀疏基底进行联合重构。提出算法利用帧间运动信息形成样本数据矩阵,再利用主成份分析（Principle Components Analysis,PCA）训练出其显著主成份构成稀疏字典,该稀疏字典不仅可根据视频时空统计特征自适应变化而且可有效地抑制噪声。仿真实验表明,本文所提出的联合重构算法可有效地改善主客观视频重构质量,能够以一定的计算复杂度为代价提高DVCS系统的率失真性能。     

Efficient reconstruction of compressively sensed images and videos using non-iterative method

Compressed sensing is widely applied for compression and reconstruction of images and videos by projecting the pixel values to smaller dimensional measurements. These measurements are reconstructed at the receiver using various reconstruction procedures. Greedy algorithms are often used for such recovery. These solve the least squares problem to find the best match with minimum error. This is a time consuming and complex process, giving rise to a trade-off between reconstruction performance and algorithmic performance. This work proposes a non-iterative method, viz., non-iterative pseudo inverse based recovery algorithm (NIPIRA), for reconstruction of compressively sensed images and videos with small complexity and time consumption, provided the reconstruction quality is maintained. NIPIRA gives a minimum PSNR of 32 dB for very few measurements (M/N = 0.3125) and accuracy of above 97%. There is more than 92% of decrease in elapsed time compared with other iterative algorithms. NIPIRA is tested for its performance with respect to many other objective measures as well. The complexity of NIPIRA is s times less than existing recovery algorithms.     

基于边信息的分布式视频压缩感知的残差重构

压缩感知(Compressed Sensing,CS)结合了视频信号的变换和信息压缩过程,为简化编码算法提供了一种新的解决思路.把分布式视频编码(DVC)和CS结合在一起,构建简单的视频编码框架,并利用原始视频帧与边信息之间的相关性进行残差重构,提出了一种基于边信息的分布式视频压缩感知编解码方案.此方法对关键帧采用传统的帧内编、解码;对非关键帧CS进行随机观测提取观测向量,解码端利用优化的边信息和传输的CS观测向量进行联合重构.实验结果表明,该方法在运动较平滑的序列中比参考方案的恢复质量提高了4 ～6 dB.     

Saliency-guided compressive sensing approach to efficient laser range measurement

The acquisition of laser range measurements can be a time consuming process for situations where high spatial resolution is required. As such, optimizing the acquisition mechanism is of high importance for many range measurement applications. Acquiring such data through a dynamically small subset of measurement locations can address this problem. In such a case, the measured information can be regarded as incomplete, which necessitates the application of special reconstruction tools to recover the original data set. The reconstruction can be performed based on the concept of sparse signal representation. Recovering signals and images from their sub-Nyquist measurements forms the core idea of compressive sensing (CS). A new saliency-guided CS-based algorithm for improving the reconstruction of range image from sparse laser range measurements has been developed. This system samples the object of interest through an optimized probability density function derived based on saliency rather than a uniform random distribution. Particularly, we demonstrate a saliency-guided sampling method for simultaneously sensing and coding range image, which requires less than half the samples needed by conventional CS while maintaining the same reconstruction performance, or alternatively reconstruct range image using the same number of samples as conventional CS with a 16 dB improvement in signal-to-noise ratio. For example, to achieve a reconstruction SNR of 30 dB, the saliency-guided approach required 30% of the samples in comparison to the standard CS approach that required 90% of the samples in order to achieve similar performance.     

一种自适应采样率视频压缩感知方案

为了进一步提高视频压缩感知方案的重构图像质量,提出了一种新的自适应采样方案.在该方案中,根据不同图像块的稀疏度自适应分配采样率.在对各图像块分类判决时,首先判断图像块在离散余弦变换域的稀疏度,其次结合该图像块与其参考帧之间的时域相关性,确定图像块的分类.实验结果表明,与现有的自适应采样率分配方案相比,该算法可获得0.5 dB左右的峰值信噪比增益.     

Reconstruction algorithm based on multi-reference frames hypothesis optimization for compressive sensing

In multi-hypothesis based distributed compressed video sensing systems,the quality of the multi-hypothesis set has important influence on the reconstruction performance of decoder.However,the acquiring of the hypothesis set has not been concerned in existing works.A reconstruction algorithm based on multi-reference frames hypothesis optimization (MRHO) was proposed.This algorithm expanded the selection of hypothesis vectors by increasing the number of reference frames.The quality of the prediction set was improved by hypotheses optimization selection under the same size with the original hypothesis set.Simulation results show that the proposed MRHO algorithm effectively improves the reconstructed quality of the distributed compressed video sensing scheme.     

融合解析模型和综合模型的压缩感知算法

如何利用更多的图像先验知识来提高图像的重构质量是压缩感知的一个关键问题.本文将综合稀疏模型与近几年提出的Cosparse解析模型结合,利用图像在综合字典和解析字典下的稀疏性提出了一种融合两种稀疏先验的图像重构算法,并利用交替方向乘子法(ADMM)求解对应的复杂优化问题.为进一步提高算法性能,该算法还充分利用了图像中任意位置图像块的稀疏性.实验结果表明,本文算法能有效提高图像重构质量.     

基于二项分布改进的宽带压缩频谱检测方案

宽带压缩频谱检测存在依赖稀疏度先验信息和信号重构时延较高的问题.因此,本文提出了一种高效可靠的宽带压缩频谱检测方案.首先,推导出了基于二项分布精确置信区间改进的稀疏度估计模型.其次,利用稀疏度估计上下界改进了稀疏度自适应匹配追踪算法.最后,提出了一种宽带压缩频谱检测方案.仿真结果表明,本文所提出方法可以同时精确的估计信号稀疏度的上下界,提高了频谱检测的效率和可靠性,加快了算法的收敛速度.