Robust spatio-temporal error concealment for packet- lossy H.264 video transmission

1 Introduction With the ubiquitous use of Internet and the deployment of next generation of networks, video communications are increa- singly becoming the major services in demand. Unlike data transmission, video communication is essentially time-sensitiv…     

基于最小像素跨度准则的空域错误隐藏方法

提出了一种新的基于最小像素跨度的错误隐藏方法,利用丢失块边界的像素,在无需进行边缘估计的情况下能够有效地恢复出丢失块中可能存在的边缘。本方法主要包括以下三个步骤:1)在最小像素跨度准则下将待恢复像素分为有争议像素和无争议像素;2)对有争议像素和无争议像素使用不同方法分别恢复;3)对整个丢失块做滤波处理以消除上述过程中的噪点。试验表明本方法有效地保留边缘,并在 PSNR 上优于其它传统方法 1～2dB。     

Hybrid error concealment method combining exemplar-based image inpainting and spatial interpolation

This paper proposes an efficient error concealment method for the reconstruction of pixels that are lost in video communication. The proposed method is developed by combining exemplar-based image inpainting for patch reconstruction and spatial interpolation for pixel reconstruction using adaptive threshold by local complexity. By exemplar-based image inpainting, regions with regular structures are reconstructed. For complex regions with irregular structures, just one pixel is reconstructed using the proposed spatial interpolation method. The proposed spatial interpolation method performs reconstruction by selecting adaptively directional interpolation or neighbor interpolation based on gradient information. Simulation results show that the proposed hybrid method performs reconstruction with significantly improved subjective quality compared with the previous spatial error concealment and image inpainting methods. The proposed method also gives substantial improvements of PSNR compared with the previous methods.     

一种改进的H.264空域错误隐藏算法

本文提出了一种改进的H.264Intra帧的错误隐藏算法,该算法引入方向模板对错误块内的边界方向和幅度进行估计,然后利用邻域边界信息内插错误块内的像素。实验结果表明本算法与参考软件中的方法相比,在具有较好隐藏效果的同时又没有增加运算量。     

On entropy-constrained vector quantization using gaussian mixture models

A flexible and low-complexity entropy-constrained vector quantizer (ECVQ) scheme based on Gaussian mixture models (GMMs), lattice quantization, and arithmetic coding is presented. The source is assumed to have a probability density function of a GMM. An input vector is first classified to one of the mixture components, and the Karhunen-Lo`eve transform of the selected mixture component is applied to the vector, followed by quantization using a lattice structured codebook. Finally, the scalar elements of the quantized vector are entropy coded sequentially using a specially designed arithmetic coder. The computational complexity of the proposed scheme is low, and independent of the coding rate in both the encoder and the decoder. Therefore, the proposed scheme serves as a lower complexity alternative to the GMM based ECVQ proposed by Gardner, Subramaniam and Rao [1]. The performance of the proposed scheme is analyzed under a high-rate assumption, and quantified for a given GMM. The practical performance of the scheme was evaluated through simulations on both synthetic and speech line spectral frequency (LSF) vectors. For LSF quantization, the proposed scheme has a comparable performance to [1] at rates relevant for speech coding (20-28 bits per vector) with lower computational complexity.     

基于模糊聚类和网格变形的自适应时-空差错掩盖方法

当视频传输中出现不能恢复的误码或丢包时,在接收端就要使用差错掩盖技术对损坏的视频进行掩藏。该文提出了一种基于模糊聚类和网格变形的自适应差错掩盖方法,使得错误恢复效果提高。该方法首先用基于时-空的模糊聚类块匹配方法补偿平移运动场景中的块丢失,然后如果检测出丢失块所在的空间有旋转、缩放等复杂运动,则用基于网格变形的方法进行补偿。实验结果表明其PSNR值比Luigi(2001)的BMA-MBW方法平均提高约1.5dB,比传统的基于块匹配的方法平均提高约3.5dB。     

Spatial error concealment with sequence-aligned texture modeling and adaptive directional recovery

A spatial error concealment technique based on the sequence-aligned texture modeling and the adaptive directional recovery is proposed in this work. The sequence alignment technique captures the local variation and the global trend of image textures with surrounding uncorrupted pixels, and provides the best texture model under a given cost function. With the derived texture model, geometric interpolation is used to recover lost pixels adaptively based on pixel locations. There are four candidate modes of pixel sequences to recover lost pixels, and one of them is selected for the concealment purpose. The selection criterion is based on the texture pattern modes of surrounding uncorrupted blocks. The pixel sequences used for error concealment can be obtained from the computation of the decoder or the side information from the encoder. Extensive experimental results are given to demonstrate that our error concealment technique outperforms several benchmark methods in both objective and subjective tests.     

一种基于局部可信视差的立体图像误码掩盖算法

为解决立体图像传输的差错问题,提出了一种基于局部可信视差的掩盖方法。首先,考虑到存在丢失块,并结合像素间色彩空间相似程度和几何空间距离接近程度等因素,设计了基准点偏置的窗口用于自适应权重的视差匹配;然后根据视差连续性原则和左右一致性约束,得出局部的可信视差;最后采用"胜者为王"(Winner-Takes-All)策略估计丢失块的视差,并根据此视差提取相应块来进行误码掩盖.实验结果表明,与其他掩盖算法相比,该算法在计算复杂度相当的情况下,无论在重建图像的PSNR还是主观质量上均具有较好的效果.     

基于GMM的AMR-NB与G.729A之间的LSP参数转码方法

通过对高斯混合模型的数学分析,建立并实现了一种基于高斯混合模型的、AMR-NB与G.729A之间的LSP参数转码算法,并通过实验对高斯混合模型在转码中的一些实际应用问题进行了分析和讨论.实验结果表明,提出的基于高斯混合模型的LSP参数转码算法在合成语音质量不次于DTE方法的同时,LSP参数转码复杂度下降了78%以上.     

一种基于快速搜索的视频时域差错隐藏算法

为了克服视频传输中因传输差错引起的视频质量下降,提出一种基于快速搜索的边框匹配时域隐藏算法(CSBM)。该算法针对时域差错隐藏的运动矢量恢复问题,利用边框匹配算法改善被恢复的物体边缘模糊的情况;采用基于中心偏置的快速搜索样式得到最小边界匹配差值的候选运动矢量,减少了解码器差错恢复的计算复杂度。实验结果显示,针对不同性质的序列,该算法与边界匹配算法(SMA)、棱形搜索的边界匹配法(DSSM)等典型差错隐藏方法相比,平均搜索点数可减少12.5~19个点,亮度分量的峰值信噪比(PSNR)能改善0.93~1.55 dB,证明该算法能获得更好的差错隐藏效果,并减少了运算量。     

Double Gaussian mixture model for image segmentation with spatial relationships

In this paper, we present a finite mixture model based on a Gaussian distribution for image segmentation. There are four advantages to the proposed model. First, compared with the standard Gaussian mixture model (GMM), the proposed model effectively incorporates spatially relationships between the pixels using a Markov random field (MRF). Second, the proposed model is similar to GMM, but has a simple representation and is easier to implement than some existing models based on MRF. Third, the contextual mixing proportion of the proposed model is explicitly modelled as a probabilistic vector and can be obtained directly during the inference process. Finally, the expectation maximization algorithm and gradient descent approach are used to maximize the log-likelihood function and infer the unknown parameters of the proposed model. The performance of the proposed model at image segmentation is compared with some state-of-the-art models on various synthetic noisy grayscale images and real-world color images.     

基于时空的混合高斯背景建模的运动目标检测

本文针对摄像机固定下的复杂背景环境,提出一种基于时空的自适应混合高斯背景建模方法,克服经典混合高斯模型（Gaussian Mixture Model,GMM）中只考虑单个像素的独立性而忽略相邻像素间的空间域相关性。首先采用混合高斯模型对每个像素在时间域上进行学习,然后利用相邻像素的自信息对背景及前景目标进行二次聚类,以修正错误的判断。实验结果表明,与经典混合高斯背景算法相比,本文提出的方法目标检测结果更加完整,具有更强的鲁棒性和很好的应用前景。     

Packet video error concealment with Gaussian mixture models.

In this paper, Gaussian mixture modeling is applied to error concealment for block-based packet video. A Gaussian mixture model for video data is obtained offline and is thereafter utilized online in order to restore lost blocks from spatial and temporal surrounding information. We propose estimators on closed form for missing data in the case of varying available neighboring contexts. Our error concealment strategy increases peak signal-to-noise ratio compared to previously proposed schemes. Examples of improved subjective visual quality by means of the proposed method are also supplied.     

Image denoising algorithm based on even step-length generalized cross validation model

Generalized cross validation (GCV) is a significant mean square error (MSE) estimator. It is widely used for image denoising because it can provide an optimal denoising threshold for these wavelet coefficients of noise image. However, the computational complexity of GCV is higher than that of the universal threshold denoising algorithm. In this study, an efficient and fast image denoising algorithm is proposed based on even step-length (ESL) GCV model. In ESL-GCV model, only the thresholds on even points are calculated from four to the maximum wavelet coefficient. In addition, the ESL-GCV model is optimized using the integer wavelet transform (IWT). These experimental results show that the IWT-based ESL-GCV model can provide lower computational complexity and the better peak signal-to-noise ratio (PSNR) than those of the traditional GCV. The proposed algorithm has important theoretical and practical value for image denoising in the future.     

Hybrid Temporal Error Concealment Methods for Block-Based Compressed Video Transmission

Boundary matching algorithm (BMA) and decoder motion vector estimation (DMVE) are two well-known temporal error concealment methods using the matching-based approach. In these two methods, the motion vector of each missing block is estimated by choosing one among candidate motion vectors which minimizes a sum of absolute differences (SAD) between boundary pixels of the corrupted macroblock. In general, the performance of DMVE is better than that of BMA. However, depending on the location or pattern of the corrupted block, BMA produces higher visual quality than DMVE. In this paper, we propose two types of hybrid error concealment methods; switching method and blending method. The switching method chooses one of two results obtained by BMA and DMVE based on the normalized SAD values. In the blending method, the weighted sum of the results concealed by the aforementioned two methods is utilized to improve the performance of error concealment. In order to reduce blocking artifacts further, the modified overlapped-block motion compensation is adaptively applied to the concealed blocks. Simulation results show that the proposed methods outperform other techniques in terms of subjective visual quality as well as PSNR performance.     

Motion Compensated Three-Dimensional Frequency Selective Extrapolation for improved error concealment in video communication

During transmission of video data over error-prone channels the risk of getting severe image distortions due to transmission errors is ubiquitous. To deal with image distortions at decoder side, error concealment is applied. This article presents Motion Compensated Three-Dimensional Frequency Selective Extrapolation, a novel spatio-temporal error concealment algorithm. The algorithm uses fractional-pel motion estimation and compensation as initial step, being followed by the generation of a model of the distorted signal. The model generation is conducted by an enhanced version of Three-Dimensional Frequency Selective Extrapolation, an existing error concealment algorithm. Compared to this existent algorithm, the proposed one yields an improvement in concealment quality of up to 1.64 dB PSNR. Altogether, the incorporation of motion compensation and the improved model generation extends the already high extrapolation quality of the underlying Frequency Selective Extrapolation, resulting in a gain of more than 3 dB compared to other well-known error concealment algorithms.     

A robust channel estimator for DS-CDMA systems under multipath fading channels

This paper addresses the problem of channel estimation for direct-sequence code-division multiple-access (DS-CDMA) systems with time-varying multipath fading channels. The multipath fading channels are modeled as autoregressive (AR) models. A method is first proposed to convert the time-varying regression model due to the time-varying nature of users' information symbols into a time-invariant one. Then, a polynomial approach is proposed to obtain the minimum mean square error (MMSE) estimator. The uncertainty of the channel model and decision errors of the DS-CDMA detector are taken into consideration in the design of the MMSE estimator. Compared with the Kalman estimator, the computational complexity of the proposed algorithm is much lower. The simulation results show that the proposed estimator provides a comparable estimation performance with the Kalman estimator and is robust for fast-fading channels.     

A fast image super-resolution algorithm using an adaptive Wiener filter.

A computationally simple super-resolution algorithm using a type of adaptive Wiener filter is proposed. The algorithm produces an improved resolution image from a sequence of low-resolution (LR) video frames with overlapping field of view. The algorithm uses subpixel registration to position each LR pixel value on a common spatial grid that is referenced to the average position of the input frames. The positions of the LR pixels are not quantized to a finite grid as with some previous techniques. The output high-resolution (HR) pixels are obtained using a weighted sum of LR pixels in a local moving window. Using a statistical model, the weights for each HR pixel are designed to minimize the mean squared error and they depend on the relative positions of the surrounding LR pixels. Thus, these weights adapt spatially and temporally to changing distributions of LR pixels due to varying motion. Both a global and spatially varying statistical model are considered here. Since the weights adapt with distribution of LR pixels, it is quite robust and will not become unstable when an unfavorable distribution of LR pixels is observed. For translational motion, the algorithm has a low computational complexity and may be readily suitable for real-time and/or near real-time processing applications. With other motion models, the computational complexity goes up significantly. However, regardless of the motion model, the algorithm lends itself to parallel implementation. The efficacy of the proposed algorithm is demonstrated here in a number of experimental results using simulated and real video sequences. A computational analysis is also presented.     

灰度共生矩阵纹理特征选块的可逆图像水印

针对当前灰度共生矩阵(GLCM)的4个主要特 征参数与图像子块纹理复杂度之间没有准确的数学关系和隐蔽 图像遮蔽性上不足的问题,提出一种GLCM纹理特征选块的可逆图像水印。首先把原宿主 图像分成 128×128大小的子块;然后利用均方误差(MSE)给 4个纹理特征参数赋予权值,建立特征参数与图像子块复杂度 的数学关系;最后计算得出各个子块的复杂度,选择复杂度最大和次大的子块,采用基于预 测误差对扩展 的可逆算法进行数据隐藏。提出的特征参数与纹理复杂度之间的数学关系,能够准确计算图 像子块复杂度, 嵌入水印后的自然图像和医学图像平均峰值信噪比(PSNR)值较 现有方法分别高出2.65%和0.93%左右。本文算 法能准 确反映子块内部的纹理复杂度并具有更好的隐蔽性,适用于医学、军事和卫星等领域。     

Error concealment for video transmission with dual multiscale Markov random field modeling

A novel error concealment algorithm based on a stochastic modeling approach is proposed as a post-processing tool at the decoder side for recovering the lost information incurred during the transmission of encoded digital video bitstreams. In our proposed scheme, both the spatial and the temporal contextual features in video signals are separately modeled using the multiscale Markov random field (MMRF). The lost information is then estimated using maximum a posteriori (MAP) probabilistic approach based on the spatial and temporal MMRF models; hence, a unified MMRF-MAP framework. To preserve the high frequency information (in particular, the edges) of the damaged video frames through iterative optimization, a new adaptive potential function is also introduced in this paper. Comparing to the existing MRF-based schemes and other traditional concealment algorithms, the proposed dual MMRF (DMMRF) modeling method offers significant improvement on both objective peak signal-to-noise ratio (PSNR) measurement and subjective visual quality of restored video sequence.