H.264编码模式选择的研究

研究H.264的编码模式选择方案,根据视频序列的时域相关性和空域相关性的关系,模式选择包括帧内预测和帧间预测,帧间预测基于率失真函数,在一帧图像的编码中则可以采用提前终止预测方向的方法降低编码器端的复杂度.     

基于HVS和时域感知失真的感兴趣区域搜索

视频质量评价模型对于感知视频编码有着重要意义.观察者对于视频场景中不同区域有不同的视觉兴趣性,而度量不同区域人眼感兴趣程度,对于构建高性能视频质量评价模型非常重要.我们在研究中发现,时域失真和时域波动分布是影响视觉兴趣性最重要的特征因素,因此定量度量视频序列的时域感知失真和时域失真波动,并根据这两个参量,采用自适应阈值判断的算法,标定出可能的兴趣像素点;同时对选中的可能兴趣点,进行空域连通分析.根据连通区域面积大小,确定1～5个可能的感兴趣区域.并根据聚类算法,确定最终的区域.最后结果证明了算法的可行性.     

一种新的空时域滤波小目标检测方法

为了提高红外图像序列中弱小目标的信噪比和检测概率,同时考虑检测算法实时性,提出了一种新的基于空时域滤波的小目标检测方法.首先,以第一帧图像为参考帧,对各帧图像进行运动补偿;然后,对运动补偿后的各帧图像在空域进行方差加权信息熵滤波,对空域滤波后图像采取双向隔帧差分的时域滤波;最后通过检测差分图像中的"凸包",抑制背景和噪声,检测小目标.文中给出了实验结果与分析,并与其他方法作了比较.实验结果表明,上述方法能大幅度的提高目标的信噪比,检测小目标,且实时性好.     

光电图像序列运动弱目标实时检测算法

针对光电探测图像序列中的运动弱小目标实时检测问题,提出了一种基于时空域融合滤波的弱目标检测算法。算法在空域上利用形态学Tophat滤波抑制背景增强目标,在时域上通过改进的帧间差分方法增强运动目标,两者融合后经自适应门限分割与航迹关联确认目标。实际录取数据分析结果表明,算法全面考虑运动弱小目标在时域与空域方面的特性,能更有效地从复杂背景中检测低信噪比运动弱小目标,减小了虚警率,抗噪声干扰能力强。     

一种视频图像序列中运动对象的分割与跟踪算法

本文提出了一种视频图像序列中运动对象的分割与跟踪算法。该算法通过Canny算子检测出差帧图像的边缘信息,并结合当前帧与背景帧的边缘图像,提取出运动对象。在后续帧中通过建立前帧感兴趣运动对象与当前帧中各运动对象的帧间向量来跟踪当前帧中感兴趣的视频对象。实验结果表明,该算法可行,而且由于该算法简单、计算复杂度小,能很好地满足实时监控系统中对感兴趣运动对象的提取与跟踪。     

结构相似度的立体视频错误隐藏

针对立体视频传输中右视点整帧丢失,提出了一种基于结构相似度(Structural Similarity,SSIM)的立体视频右视点整帧丢失错误隐藏算法。首先,提出了时域SSIM和视间SSIM的概念。然后,根据视频序列的时域相关性,将前一时刻右视点图像宏块的预测方式作为丢失图像宏块的预测方式。接着,将前一时刻右视点图像以宏块为单位进行时域和视间匹配,求取其以像素为单位的时域SSIM映射图和视间SSIM映射图。最后,计算并比较前一时刻右视点图像每个宏块的时域SSIM和视间SSIM值,得到每个宏块的预测方式,将其预测方式作为丢失帧中宏块的预测方式,从而使用运动补偿预测或者视差补偿预测的方法进行恢复。实验结果表明,与传统的算法和Pang的算法相比,PSNR值分别提高了2.76 dB和3.43 dB,且本文算法主观效果较好。     

基于L1范数优化路径的视频稳像算法

目的 为解决外包装行业对产品外观进行检测时,采集视频图像存在抖动失真的问题.方法 提出一种基于L1范数优化路径的视频稳像算法,采用SURF算法和FREAK算法对视频序列帧中的特征点进行检测和描述;然后,使用KNN算法和RANSAC算法对相邻帧间的特征点进行匹配和筛选;最后,通过L1范数优化算法对序列帧进行校正和去黑边处理,得到稳像视频.结果 在处理前景无运动和前景有运动的2类视频时,稳像前后视频的平均PSNR值分别提升了5.094 dB和4.273 dB,并且相对于常用的特征匹配算法,文中算法能显著降低相机路径的主动平滑因子.结论 文中算法能够有效地解决视频抖动失真的问题,提高抖动视频的质量,具有一定的鲁棒性.     

基于双流网络的视频图像去噪算法

该文针对传统视频图像去噪算法因阈值不准确导致伪影等时域模糊问题,提出了一种基于双流网络的视频图像去噪算法。该算法统筹考虑视频图像在时间通道与空间通道的关系属性,通过效仿人体视觉过程,对视频帧序列中的时序信息与空间信息进行内容理解,以帧间图像绝对误差均值MAD的方差来表征视频序列在时空网络中的动静属性,并由此计算出动态可变的去噪阈值,使采用的时空维度去噪算法更科学、更精准。     

H.264帧内预测与帧间块模式选择快速算法

针对H.264帧间预测时的多子块模式选择和帧内预测的多模式选择问题,提出了一种实用的基于统计门限的模式快速选择算法,通过门限设定有效限定了编码模式的选择范围.多组视频序列的实验结果表明,基于统计门限的帧内预测模式选择算法在Ⅰ帧的PSNR值有微小变化的情况下,大大降低了编码时间,Ⅰ帧的编码时间降低17%～23%,PSNR下降不到0.4%.基于统计门限的帧间块模式选择算法使得平均PSNR值下降了0.047dB(PSNR最大下降不到1%),但是相对全模式预测时,编码器速度提高了20%～30%.     

基于空间连续生成对抗网络的视频帧间图像生成

针对低帧率视频播放不流畅以及使用传统方法提高视频帧率造成的边缘模糊问题,本文提出一种基于空间连续生成对抗网络(SC-GAN)的视频帧间图像生成方法。首先本文使用自编码器作为判别器,引入Wasserstein距离表示真实样本与生成样本损失分布的差异,替代传统生成对抗网络直接匹配数据分布的方式,其次利用生成器与判别器之间的平衡参数稳定训练过程,有效避免了模型崩溃的问题,最后利用连续视频帧图像在空间上的连续性,通过Adam在相邻两帧之间找到一个最优值,将其映射到图像空间,得到生成的帧间图像。为了说明生成的帧间图像的真实性,本文采用PSNR和SSIM对帧间图像进行了评估,评估结果证明生成的帧间图像具有较高的真实度,同时验证了本文提出的基于SC-GAN的视频帧间图像生成方法的可行性和有效性。     

Intra mode decision and frame layer Lagrange multiplier adjustment scheme for H.264 based on structural similarity

H.264 takes rate distortion optimisation (RDO) technique to perform intra and inter mode decision and achieves higher coding efficiency, but the objective distortion metric such as mean square error (MSE) is employed in traditional RDO framework, which cannot acquire optimal subjective quality. In this paper, structural similarity (SSIM)-based subjective distortion is applied to RDO-based intra mode decision in H.264 I frame video coding, and a linear SSIM distortion model is firstly proposed and SSIM-based rate distortion cost function for intra mode decision is defined. Furthermore, a content adaptive frame layer Lagrange multiplier adjustment scheme is proposed to balance the tradeoff between rate and SSIM distortion better. Experimental results show that, the proposed method encodes image structural information more effectively and thus acquires better perceptual quality and subjective RDO performance compared with objective distortion-based RDO method. Under the same perceptual quality, our scheme achieves about 8·03% I frame bit rate reduction on average for various sequences over MSE-based RDO employed in JM reference software.     

基于Dirichlet过程的无线视频码率变化识别算法

研究了通过对终端视频帧质量的聚类分析来识别无线视频传输中码率变化的方法,以便为无线视频传输过程中视频码率自适应调整提供参考依据.针对经典模糊C均值(FCM)算法和K均值(K-means)算法需要设定聚类数目的问题,提出一种基于荻利克雷过程(DP)的FCM算法——DP-FCM算法.该算法将Dirichlet过程和FCM算法相结合,由视频帧信息权重峰值信噪比(IWPSNR)值使用DP过程混合模型模拟估计出聚类数目,然后进行FCM模糊聚类,通过设定合理的阈值,合并聚类结果相似项,完成视频帧的聚类,从而实现视频传输码率变化的识别.以LIVE视频库为试验数据源,对该算法进行了性能测试.试验结果表明,DP-FCM算法能够在无需设定聚类数目的前提下实现视频传输码率变化的分类识别.     

Defining Embedding Distortion for Intra Prediction Mode-Based Video Steganography

In this paper, an effective intra prediction mode-based video strganography is proposed. Secret messages are embedded during the intra prediction of the video encoding without causing large embedding impact. The influence on the sum of absolute difference (SAD) in intra prediction modes (IPMs) reversion phenomenon is sharp when modifying IPMs. It inspires us to take the SAD prediction deviation (SPD) to define the distortion function. What is more, the mapping rule between IPMs and the codewords is introduced to further reduce the SPD values of each intra block. Syndrome-trellis code (STC) is used as the practical embedding implementation. Experimental results demonstrate that our proposed steganographic scheme presents high undetectability compared with existing IPMs-based steganographic approaches. It also outperforms these schemes on stego video quality.     

基于背景图像的视频图像压缩感知重构关键技术研究

目的针对视频序列相邻帧图像之间的相关性,提出了基于帧间差分的背景图像的视频图像重构算法。方法利用前2帧的重构图像求得背景图像,作为下一帧图像重构的先验知识。结果明显提高了重构图像的质量,同时也减少了重构图像所需时间。结论将帧间差分法与压缩感知重构算法相结合得到的视频重构算法,在视频图像重构的质量和速度上都有明显优势。     

High-Movement Human Segmentation in Video Using Adaptive N-Frames Ensemble

A wide range of camera apps and online video conferencing services support the feature of changing the background in real-time for aesthetic, privacy, and security reasons. Numerous studies show that the Deep-Learning (DL) is a suitable option for human segmentation, and the ensemble of multiple DL-based segmentation models can improve the segmentation result. However, these approaches are not as effective when directly applied to the image segmentation in a video. This paper proposes an Adaptive N-Frames Ensemble (AFE) approach for high-movement human segmentation in a video using an ensemble of multiple DL models. In contrast to an ensemble, which executes multiple DL models simultaneously for every single video frame, the proposed AFE approach executes only a single DL model upon a current video frame. It combines the segmentation outputs of previous frames for the final segmentation output when the frame difference is less than a particular threshold. Our method employs the idea of the N-Frames Ensemble (NFE) method, which uses the ensemble of the image segmentation of a current video frame and previous video frames. However, NFE is not suitable for the segmentation of fast-moving objects in a video nor a video with low frame rates. The proposed AFE approach addresses the limitations of the NFE method. Our experiment uses three human segmentation models, namely Fully Convolutional Network (FCN), DeepLabv3, and Mediapipe. We evaluated our approach using 1711 videos of the TikTok50f dataset with a single-person view. The TikTok50f dataset is a reconstructed version of the publicly available TikTok dataset by cropping, resizing and dividing it into videos having 50 frames each. This paper compares the proposed AFE with single models and the Two-Models Ensemble, as well as the NFE models. The experiment results show that the proposed AFE is suitable for low-movement as well as high-movement human segmentation in a video.     

基于编码模式的H.264/AVC视频信息隐藏算法

本文提出一种基于编码模式的H.264/AVC信息隐藏方法,通过调制某些宏块的编码模式,分别在Ⅰ帧、P帧和B帧中嵌入隐秘信息.对帧内4x4预测模式的宏块,是通过调整宏块中某个4×4块的编码模式嵌入隐秘信息;对P帧和B帧其它类型的宏块,则是通过调整宏块的编码模式嵌入隐秘信息,模式调整后对宏块做了优化处理.在模式调制过程中引入率失真代价,取得了较好的率失真平衡,减小了隐秘信息嵌入后对视频质量和视频码流的影响.该算法可以实现隐秘信息的快速提取,满足视频实时处理的要求,实验仿真结果证明了该算法的有效性.     

A no reference quality metric to measure the blocking artefacts for video sequences

The need to measure video quality arises in the development of video equipment and in the delivery and storage of video and image information. In this paper, we propose a new perceptually significant video quality metric to estimate the effect of block coding for standards H.264 AVC and MPEG2. Our method operates in the spatial domain and does not require a high complexity of computation. We evaluate the performance of the proposed method by using three sequences CIF ‘common intermediate file’ with different compression rate. We compare it with Suthaharan’s and MSU’s techniques by using ‘LIVE’ and ‘IVP’ databases. Results indicate that the proposed method outperforms Suthaharan’s and MSU techniques in H264 coder. They also indicate that our method is more effective than MSU’s and Suthaharan’s techniques for the H.264 AVC standards with the Spearman Rank Order Correlation Coefficient.     

Intra mode selection in downsizing video transcoder based on H.264

An intra mode selection scheme is proposed in this work, which supports both downsizing transcoding and re‐quantization transcoding simultaneously. In the proposal, a total number of nonzero coefficients in precoded frame is used as criterion and a thresholding method is applied to select intra macroblock mode in re‐encoder. To calculate this threshold, which is related to re‐quantization parameter (denoted as Q_r), we propose a Th_I‐Q_r model which includes direct method and percentage I16MB method. In the former, an exponent model is proposed to describe relationship between the threshold and the Q_r; while in the latter, the threshold Th_I is converted into percentage of macroblocks with I16MB mode in the downsized frame (denoted as per_16), and relationship between the per_16 and the Q_r is also modeled as an exponent function. Then the two exponent models are all converted into linear regression model, and least square estimation is used to estimate the parameters of the models. Furthermore, if I4MB mode is selected for one macroblock, the intra prediction modes in precoded frame are utilized to select prediction mode for every 4 × 4 block of the macroblock in downsized frame to reduce computational complexity. We compared rate distortion performance and computational complexity of the proposed method with rate‐distortion optimization method. Simulation results demonstrate that on the precondition of compression performance of the proposal being close to the results of the rate‐distortion optimization method, the proposed method can save up to 30 and 80% in total encoding time and mode decision time, respectively. © 2009 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 19, 340–349, 2009     

Fast intra prediction mode decision for H.264/AVC video coding

Abstract

Intra coding is used for reducing the spatial redundancy in video coding. H.264 supports several macroblocks of predictions for intra coding such as luma block four 16×16 modes, nine 4×4 modes and chroma block four modes, which significantly improve intra coding efficiency, but increase the encoding complexity. In order to select the best mode, we need to calculate the cost of the various modes. In this paper, a fast intra prediction mode decision for H.264/AVC video coding is proposed. Based on Laplacian, this intra prediction mode decision detects edges and selects the best mode for the block. This mode decision can shorten the time to reduce the encoding time. The experimental results show that the proposed algorithm achieves an encoding time saving of 70% on average.     

一种基于视觉感知的H.264码率控制算法

针对目前视频编码标准H.264的码率控制算法未考虑人眼视觉感知、易导致编码后视频图像质量波动的不足,提出了一种基于视觉感知的H.264码率控制算法.首先,设计了像素域的恰可察觉失真模型.在此基础上,根据各帧的恰可察觉失真的大小进行帧层比特分配.其次,建立了基于结构相似度的率失真模型,并采用此模型设计了基本单元层（basic unit,BU）的比特分配方案.最后结合二次速率-量化模型得到量化参数.实验结果表明,该算法与目前H.264中典型的码率控制算法相比,错误率降低了0.2％.