No-reference quality assessment of HEVC video streams based on visual memory modelling

Providing adequate Quality of Experience (QoE) to end-users is crucial for streaming service providers. In this paper, in order to realize automatic quality assessment, a No-Reference (NR) bitstream Human-Vision-System-(HVS)-based video quality assessment (VQA) model is proposed. Inspired by discoveries from the neuroscience community, which suggest there is a considerable overlap between active areas of the brain when engaging in video quality assessment and saliency detection tasks, saliency maps are used in the proposed method to improve the quality assessment accuracy. To this end, saliency maps are first generated from features extracted from the HEVC bitstream. Then, saliency map statistics are employed to create a model of visual memory. Finally, a support vector regression pipeline learns an estimate of the video quality from the visual memory, saliency, and frame features. Evaluations on SJTU dataset indicate that the proposed bitstream based no-reference video quality assessment algorithm achieves a competitive performance.     

基于块编码视频的无参考质量评估

该文提出了一种适用于基于块编码视频的无参考质量评估方法。首先结合人类视觉的亮度掩盖和对比度掩盖特性提出了一个符合主观视觉感知的方块效应测度,然后根据滤波对方块效应的影响,给出了一种适合于使用不同压缩和处理算法的基于块编码重构视频的质量评估方法。实验表明该质量评估测度与主观质量评估有较好的一致性。     

基于半马尔科夫决策过程的视频传输拥塞控制算法

针对传输控制协议(TCP, transmission control protocol)的拥塞控制算法未能满足视频传输质量要求的问题,提出了一种基于半马尔科夫决策过程的视频传输拥塞控制算法。首先, 为克服目前基于峰值信噪比的视频质量评估方法实时性低的缺点,设计了一种可在线运行的无参考视频质量评估方法。其次,根据接收端视频质量的反馈,采用半马尔科夫决策过程对拥塞控制进行建模,并通过求解此模型得到拥塞控制参数的调整策略。仿真实验结果表明,与目前典型的拥塞控制算法相比,该算法不但具备更好的TCP友好性,而且有效地提高了解码后视频序列的主观和客观质量。     

注意力模型指导下的视频质量评价方法

在视频质量评价方法中,常常需要对人眼视觉系统做出合理的假设,其中注意力模型就是一个很重要的因素。提出了一种在注意力模型指导下的视频质量评价方法,在图像帧的质量评价中加入了显著性区域信息,使之更能符合人眼视觉特性,并兼顾了视频中的运动信息,在一定程度上提高了客观质量评价方法的性能。     

Optimized video compression with residual split attention and swin-block artifact contraction

Research in video compression has seen significant advancement in the last several years. However, the existing deep learning-based algorithms continue to be plagued by erroneous motion compression and ineffective motion compensation architectures, resulting in compression errors with a lower rate–distortion trade-off. To overcome these challenges, we present an end-to-end purely deep learning-based video compression method through a set of primary operations (e.g., motion estimation, motion compression, motion compensation, residual compression, and artifact contraction) differently. A deep residual attention split (DRAS) block is introduced for motion compression networks to pay more attention to certain image regions to create more effective features for the decoder while boosting the rate–distortion optimization (RDO) efficiency. A channel residual block (CRB) is proposed in motion compensation to yield a more accurate predicted frame, potentially improving the residual frame. To mitigate the compression errors, an artifact contraction module (ACM) by residual swin convolution UNet block is included in this model to improve the reconstruction quality. To improve the final frame, a buffer is added to fine-tune the previous reference frames. These modules combine with a loss function by assessing the trade-off and enhancing the decoded video quality. A comprehensive ablation study demonstrates the effectiveness of the proposed blocks and modules for video compression. Experimental results show the competitive performance of the proposed method on four benchmark datasets.     

基于自然场景统计的无参考图像质量评价算法

为了度量多种失真类型的图像质量,提出一种基于图像空域自然场景统计特征的无参考图像质量评价算法。该算法通过度量失真图像和原始图像在统计规律上的偏差,对失真图像质量做出评价。与现有无参考图像质量评价算法相比,该算法不需要使用原始图像及其失真图像进行训练,也不需要知道图像的失真类型,是一种更具实际意义的通用型无参考图像质量评价算法。同时,考虑到人眼观察图像时感兴趣区域的影响,该算法加入了视觉显著性区域提取的过程。实验结果表明,该算法对于人的主观感知具有较好的一致性。     

基于可变块运动矢量的边信息生成算法*

在分布式视频编码系统中,针对图像中细节丰富的区域易造成严重的块效应,提出了一种基于可变块运动矢量的边信息生成算法。根据前后相邻关键帧对应块的相关性,将像素块分为保留块和运动块。对保留块直接作保留处理,对运动块中的像素块继续进行分割并计算子块的初始运动矢量,最后将所有对应块的运动矢量进行加权自适应运动补偿得到改进的边信息。实验结果表明,对于运动较剧烈复杂的视频序列,该算法能够提高边信息生成质量, 并且使得改进后的边信息PSNR值提高了0.98~1.33 dB。     

基于梯度结构相似度的无参考模糊图像质量评价

在对模糊图像边缘膨胀后进行边缘膨胀块提取的 基础上,提出一种基于梯 度结构相似度(GSIM)的无参考模糊图像质量评价方法(NRGSIM)。首先,将原始模糊图像经过低 通滤波生成再模糊图像;之后,将原始模糊图像生成的边缘膨胀图像进行8×8分块,并将子块 划分为边缘膨胀块和平滑块;然后,计算原始模糊图像和再模糊图像中所有对应到边缘膨胀 图 中边缘膨胀块的相应子块的GSIM;最后,平均得到整幅图像的模糊值。在4个数据 库上实验结果表明,本文方法评价结果合理、稳定,更加符合人类视觉特性,与主观评分有 较好的一致性,而且计算简单,取得了很好的评价效果,LIVE2数据库上的SROCC指标达到0.964。     

移动多媒体广播无参考视频质量客观评价

重点研究了基于解压域的移动多媒体广播无参考视频质量客观评价方法,对不同测试序列在不同压缩码率下的受损图像进行了块效应和模糊度的算法分析,并利用线性加权的方法估计视频质量客观评价值.试验结果表明,采用的无参考客观评价值与全参考的PSNR值具有较强相关性,可用于终端用户的视频质量评价.     

Improved side information generation algorithm for Wyner-Ziv video coding简

Side information has a significant influence on the rate-distortion（RD） performance of distributed video coding（DVC）. In the conventional motion compensated frame interpolation scheme, all blocks adopt the same side-information generation method regardless of the motion intensity inequality at different regions. In this paper, an improved method is proposed. The image blocks are classified into two modes, fast motion and slow motion, by simply computing the discrete cosine transformation（DCT） coefficients at the encoder. On the decoder, it chooses the direct interpolation and refined motion compensated interpolation correspondingly to generate side information. Experimental results show that the proposed method, without increasing the encoder complexity, can increase the average peak signal-to-noise ratio（PSNR） by up to 1~ 2 dB compared with the existing algorithm. Meanwhile, the proposed algorithm significantly improves the subjective quality of the side information.     

Sparsity-based no-reference image quality assessment for automatic denoising

In image and video denoising, a quantitative measure of genuine image content, noise, and blur is required to facilitate quality assessment, when the ground truth is not available. In this paper, we present a no-reference image quality assessment for denoising applications, which examines local image structure using orientation dominancy and patch sparsity. We propose a fast method to find the dominant orientation of image patches, which is used to decompose them into singular values. Combining singular values with the sparsity of the patch in the transform domain, we measure the possible image content and noise of the patches and of the whole image. We show that the proposed method is useful to select parameters of denoising algorithms automatically in different noise scenarios such as white Gaussian and processed noise. Our objective and subjective results confirm the correspondence between the measured quality and the ground truth. We show that the proposed method rivals related state-of-the-art no-reference quality assessment approaches.     

Human motion quality assessment toward sophisticated sports scenes based on deeply-learned 3D CNN model

Video may be subject to various distortions during acquisition, processing, compression, storage, transmission, and reproduction, and it results in reduced visual quality. In complex sports scenes under big data environment, the human body's movements are even more so. The quality of human motion can intuitively affect the human visual experience. Therefore, it is necessary to determine an intelligent quality assessment model to evaluate human motion in complex motion scenarios under big data environment. It can be used to dynamically monitor and adjust video quality, and it can be used for algorithms and parameter settings in motion image processing systems. With the popularity of deep learning, convolutional neural networks have become a very important method in the field of computer vision research. Based on the 2D-CNN algorithm, we propose a 3D convolutional neural network model for human motion quality assessment in complex motion scenarios. The model captures the pose characteristics, motion trajectory, video brightness and contrast in time and space. The model feeds back the reference and distorted video pairs into the network, with each output layer acting as a feature map. The local similarity between the feature maps obtained from the reference video and the distorted video is then calculated and combined to obtain a global image quality score. Experiments show that the model can achieve competitive performance in big data environment for video quality assessment.     

Multi-layer and Multi-scale feature aggregation for DIBR-Synthesized image quality assessment

Depth-Image-Based Rendering (DIBR) is one of the main fundamental techniques for generating new viewpoints in 3D video applications such as multi-viewpoint video (MVV), free viewpoint video (FVV) and virtual reality (VR). Due to the imperfections of color images, depth maps or texture restoration techniques, several types of distortions occur in synthesized views. However, most of related works evaluated the quality of DIBR-synthesized views by only detecting a specific type of distortion, such as stretching, black holes, blurring, etc., which were unable to accurately evaluate the quality of DIBR-synthesized views. In this paper, a new no-reference image quality assessment method is proposed to evaluate the quality of DIBR-synthesized images by combining multi-layer and multi-scale features of images. To be specific, the distortions introduced by different stages of virtual viewpoint synthesis are first analyzed, and then multi-layer and multi-scale features are extracted to estimate the degree of texture and structure distortions. As a result, individual quality scores associated with two types of distortions (e.g., structural distortion and texture distortion) are aggregated to an overall image quality. Experimental results on two publicly available DIBR datasets show that the method has better performance than the state-of-the-art models.Index Terms: image quality assessment, DIBR-synthesized image, distortion correction, BIQA.     

基于高阶深层时空信息的自媒体视频质量评价

自媒体视频中普遍存在的失真扭曲现象,给视频质量评价带来了新的挑战。视觉感知理论研究表明,人类视觉系统中存在感知迭代机制,即视频的感知评价是一个正反向迭代修正的过程。受此启发,本文将该机制引入视频质量评价,提出了基于高阶深层时空信息的质量评价方法。具体而言,本文提出了二阶协方差聚合来进行高阶空域信息的提取,引入快速迭代GRU结构进行深层时域信息建模,而后通过特征层的池化聚合以及多层感知机回归视频得分。实验结果表明,预测结果与人类主观质量评分具有较好的一致性,明显优于现有无参考质量评价算法。      

融合空时感知特性的无参考视频质量评估算法

本文通过简化视频质量评估中人眼感知模型的复杂性,提出了一种新的无参考视频质量评估模型.首先通过分别抽取视频的空间域和时间域特征,然后按照视频局部块、视频帧、视频段等从细到粗的不同粒度,模拟人眼感知特性进行多重加权汇聚,最终得到整段视频的特征向量描述.本方法以支持向量回归器为评估模型训练工具,通过有监督的视频样本库训练,以无参考方式完成未知视频的质量评估.实验结果表明,该评估算法的性能不但要优于当前已知最经典的无参考评估算法Video BLLINDS,而且与部分参考评估算法相当.     

No-reference Stereoscopic Image Quality Assessment Using Binocular Self-similarity and Deep Neural Network

Quality assessment of three-dimensional (3D) images is more challenging than that of 2D images. The quality of 3D visual experience is one of the most challenging areas of human binocular perception and is affected by multiple factors such as asymmetric stereo image/video compression, depth perception, visual discomfort, and single view quality. In this paper, we propose a new no-reference quality assessment method for stereoscopic images based on Binocular Self-similarity (BS) and Deep Neural Networks (DNN). To be more specific, a BS index is defined and computed according to binocular rivalry and suppression based on the depth image-based rendering technique. Then, a DNN is trained in an opinion unaware way to predict local quality. Binocular integration (BI) index is calculated by using the trained DNN, accounting for binocular integration behaviors. Finally, the final quality score of stereoscopic image is obtained by combining the BS and BI indexes together. Experimental results on four public 3D image quality assessment databases demonstrate that compared with existing methods, the proposed method can achieve high consistency with subjective perception on stereoscopic images with both symmetric and asymmetric distortions.     

A New Approach to Slow Motion Effect for Digital TV Broadcasting Services

Slow motion replay of video sequences is a commercial feature for home video players, but also a special effect used in video production. At present, most techniques to perform digital slow motion are based on interpolation for generating additional intermediate frames. Mainly, interpolation is done through linear or cubic spline functions or by motion estimation/compensation applied either pixel by pixel, or by partitioning frames into blocks. This approach has drawbacks: above all, interpolation bears to fading effects. Purpose of this paper is to present a novel technique combining fractals and wavelets to obtain higher performance in slow motion display of digital video sequences. Active scene detection and video processing are used to reduce computational cost and improve visual quality. This method produces better results in terms of objective perceptual video quality comparing to state-of-the-art techniques.     

自适应联合相邻运动矢量的运动补偿插值算法

帧率上采样作为一种视频后处理技术,通过对原始视频插值得到高帧率视频,满足人们对高帧率视频的需求.传统基于重叠块的插值补偿算法会带来块效应或鬼影现象.为了解决这一问题,人们提出联合匹配块及其相邻块预测的方法,但是复杂度高,效果不明显.基于以上几点,本文提出一种自适应联合相邻运动矢量的运动补偿插值算法(Joint Motion-Compensated In-terpolation Algorithm Using Adjacent Block Motion Vectors Adaptively,AJ-MCI).在该算法中,将运动矢量矫正和运动补偿插值算法相结合,同时运动补偿模块中自适应地联合相邻匹配块,以最大限度刻画真实运动轨迹.实验结果表明,本文算法能很好的提升视频的主客观质量,同时保持较低计算复杂度.     

A dynamic search pattern motion estimation algorithm using prioritized motion vectors

Motion estimation is one of the critical parts in video compression standards with a high computational load. Many motion estimation algorithms have been developed to reduce the number of search points compared to a full-search algorithm without losing the quality considerably. Most of them use fixed search patterns in their first step which may suffer from trapping into local minima or searching unnecessary blocks due to inappropriate size and type of search patterns. In this paper, a new dynamic search pattern using motion vectors of spatial and temporal neighboring blocks is proposed. The motion vectors of neighboring blocks are prioritized, in order to efficiently use of halfway stop technique. The simulation results indicate that proposed algorithm is very close to the full-search algorithm in quality, compared to other rivals. Moreover, the average number of searches is often less than other algorithms.     

No-Reference Image Quality Assessment Method Based on Visual Parameters

Recent studies on no-reference image quality assessment (NR-IQA) methods usually learn to evaluate the image quality by regressing from human subjective scores of the training samples. This study presented an NR-IQA method based on the basic image visual parameters without using human scored image databases in learning. We demonstrated that these features comprised the most basic characteristics for constructing an image and influencing the visual quality of an image. In this paper, the definitions, computational method, and relationships among these visual metrics were described. We subsequently proposed a no-reference assessment function, which was referred to as a visual parameter measurement index (VPMI), based on the integration of these visual metrics to assess image quality. It is established that the maximum of VPMI corresponds to the best quality of the color image. We verified this method using the popular assessment database—image quality assessment database (LIVE), and the results indicated that the proposed method matched better with the subjective assessment of human vision. Compared with other image quality assessment models, it is highly competitive. VPMI has low computational complexity, which makes it promising to implement in real-time image assessment systems.