Objective and subjective evaluation of High Dynamic Range video compression

A number of High Dynamic Range (HDR) video compression algorithms proposed to date have either been developed in isolation or only-partially compared with each other. Previous evaluations were conducted using quality assessment error metrics, which for the most part were developed for qualitative assessment of Low Dynamic Range (LDR) videos. This paper presents a comprehensive objective and subjective evaluation conducted with six published HDR video compression algorithms. The objective evaluation was undertaken on a large set of 39 HDR video sequences using seven numerical error metrics namely: PSNR, logPSNR, puPSNR, puSSIM, Weber MSE, HDR-VDP and HDR-VQM. The subjective evaluation involved six short-listed sequences and two ranking-based subjective experiments with hidden reference at two different output bitrates with 32 participants each, who were tasked to rank distorted HDR video footage compared to an uncompressed version of the same footage. Results suggest a strong correlation between the objective and subjective evaluation. Also, non-backward compatible compression algorithms appear to perform better at lower output bit rates than backward compatible algorithms across the settings used in this evaluation.     

Review of objective video quality metrics and performance comparison using different databases

The use of video-based applications has increased in recent years owing to the development of video technology as well as the widespread use of the Internet. Thus the evaluation of perceptual video quality has become very important and numerous video quality assessment (VQA) metrics have been developed over the past years. In this paper, we give a classification and a short review of objective VQA metrics, with a focus on the full reference metrics. With the aim of conducting a reliable test of the VQA metrics performances, we made two databases, each of them including 90 distorted video sequences. We carried out a subjective quality evaluation on these databases and the data were made available to the research community. Furthermore, we compared the performance of nine different, freely available, objective VQA metrics by using three different databases in different resolutions: LIVE Video Quality Database (768×432 resolution) and our two newly created databases for progressively scanned videos, i.e. ETFOS CIF Video Quality (ECVQ) database and ETFOS VGA Video Quality (EVVQ) database. Five different distortion types were used and the total number of 330 video sequences was evaluated. A comparison of metrics was done with respect to accuracy, monotonicity, stability, as well as complexity vs. accuracy criteria. The results show that the resolution, the content of the sequence and the distortion type have a significant influence on the performances of VQA metrics. Metrics that generally achieve a high correlation with subjective results for all databases and all distortion types are MOtion-based Video Integrity Evaluation (MOVIE) and Foveated Mean Squared Error (FMSE), but MOVIE has significantly higher complexity than FMSE. An exception is the distortion caused by IP transmission for which none of the analyzed metrics has shown satisfying accuracy and stability.     

MPEG-4框架下的视频对象分割及其关键技术分析?

在简要介绍MPEG-4引入的视频对象概念及其应用背景的基础上,系统地研究了视频对象分割所涉及的关键技术。首先,按照视频是否压缩将现有算法划分为像素域和压缩域两类,详细讨论了所涉及到的初始对象分割、对象跟踪技术。其次,阐述了立体视频对象分割以及基于细胞神经网络体系结构的对象分割算法的研究进展。此外,对视频对象分割算法的客观性能评价进行了总结。最后,对本领域值得进一步研究的问题进行了预测和展望。     

Evaluation of segmentation techniques using region area and boundary matching information

Evaluation techniques play an important role while picking a suitable segmentation scheme out of a number of alternatives. In this paper, a novel supervised segmentation evaluation scheme is proposed that is designed by combining segment area and boundary information. Using the evaluation metric, a ranking of the popular segmentation algorithms is carried out. A comparative analysis with existing supervised metrics that are commonly used for grading segmentation schemes is performed. Experimental results indicate that the performance of the proposed measure is promising.     

Scalable 3D video streaming over P2P networks with playback length changeable chunk segmentation

3D video distribution over P2P networks has been thought as a promising way for 3D video entering home. The convergence of scalable 3D video coding and P2P streaming can provide diverse 3D experiences for heterogeneous clients with high distribution efficiencies. However, the conventional chunk segmentation and scheduling algorithms originally aiming at the non-scalable 2D video streaming are not very efficient for scalable 3D video streaming over P2P networks due to the particular data characteristics of scalable 3D video. Based on this motivation, this paper first presents a playback length changeable 3D video chunk segmentation (PLC3DCS) algorithm to provide different error resilience strengths to video and depth as well as layers with different importance levels in the 3D video transmission. Then, a hybrid-priority based chunk scheduling (HPS) algorithm is proposed to be tied in with the proposed chunk segmentation algorithm to further promote the overall 3D video P2P streaming performance. The simulation results show that the proposed PLC3DCS algorithm with the corresponding HPS can increase the success delivery rates of chunks with more important levels, and further improve the user’s quality of 3D experience.     

Methods for performance evaluation of VBR video traffic models

Models for predicting the performance of multiplexed variable bit rate video sources are important for engineering a network. However, models of a single source are also important for parameter negotiations and call admittance algorithms. In this paper we propose to model a single video source as a Markov renewal process whose states represent different bit rates. We also propose two novel goodness-of-fit metrics which are directly related to the specific performance aspects that we want to predict from the model. The first is a leaky bucket contour plot which can be used to quantify the burstiness of any traffic type. The second measure applies only to video traffic and measures how well the model can predict the compressed video quality     

Toward a generic evaluation of image segmentation.

Image segmentation plays a major role in a broad range of applications. Evaluating the adequacy of a segmentation algorithm for a given application is a requisite both to allow the appropriate selection of segmentation algorithms as well as to tune their parameters for optimal performance. However, objective segmentation quality evaluation is far from being a solved problem. In this paper, a generic framework for segmentation evaluation is introduced after a brief review of previous work. A metric based on the distance between segmentation partitions is proposed to overcome some of the limitations of existing approaches. Symmetric and asymmetric distance metric alternatives are presented to meet the specificities of a wide class of applications. Experimental results confirm the potential of the proposed measures.     

Temporal stabilization of Video Object Segmentation for 3D-TV applications

Our aim is to insert depth information into an existing 2D video sequence to provide content for 3D-TV applications, which we try to achieve through segmentation of the objects in the given 2D video sequence. To this effect, we present a method for temporal stabilization of video object segmentation algorithms for 3D-TV applications. First, two quantitative measures to evaluate temporal stability without ground-truth are discussed. Then, a pseudo-3D curve evolution method, which spatio-temporally stabilizes the estimated segmentation of a video object is introduced. Temporal stability is achieved by re-distributing existing object segmentation errors such that they will be less disturbing when the scene is rendered and viewed in 3D. Our starting point is the hypothesis that if making segmentation errors is inevitable, these errors should be made in a temporally consistent way for 3D-TV applications. This hypothesis is supported by the experiments, which show that there is significant improvement in segmentation quality both in terms of the objective quantitative measures and in terms of the viewing comfort in subjective perceptual tests. Therefore, it is possible to increase the perceptual object segmentation quality without increasing the actual segmentation accuracy.     

A wrapper-based approach to image segmentation and classification.

The traditional processing flow of segmentation followed by classification in computer vision assumes that the segmentation is able to successfully extract the object of interest from the background image. It is extremely difficult to obtain a reliable segmentation without any prior knowledge about the object that is being extracted from the scene. This is further complicated by the lack of any clearly defined metrics for evaluating the quality of segmentation or for comparing segmentation algorithms. We propose a method of segmentation that addresses both of these issues, by using the object classification subsystem as an integral part of the segmentation. This will provide contextual information regarding the objects to be segmented, as well as allow us to use the probability of correct classification as a metric to determine the quality of the segmentation. We view traditional segmentation as a filter operating on the image that is independent of the classifier, much like the filter methods for feature selection. We propose a new paradigm for segmentation and classification that follows the wrapper methods of feature selection. Our method wraps the segmentation and classification together, and uses the classification accuracy as the metric to determine the best segmentation. By using shape as the classification feature, we are able to develop a segmentation algorithm that relaxes the requirement that the object of interest to be segmented must be homogeneous in some low-level image parameter, such as texture, color, or grayscale. This represents an improvement over other segmentation methods that have used classification information only to modify the segmenter parameters, since these algorithms still require an underlying homogeneity in some parameter space. Rather than considering our method as, yet, another segmentation algorithm, we propose that our wrapper method can be considered as an image segmentation framework, within which existing image segmentation algorithms may be executed. We show the performance of our proposed wrapper-based segmenter on real-world and complex images of automotive vehicle occupants for the purpose of recognizing infants on the passenger seat and disabling the vehicle airbag. This is an interesting application for testing the robustness of our approach, due to the complexity of the images, and, consequently, we believe the algorithm will be suitable for many other real-world applications.     

On the differentiated QoE enforcement between competing scalable video flows over wireless networks

In differentiated quality‐of‐experience enforcement (QoE) for video transmission over wireless networks, accurate video quality metrics play a crucial role in the designing process of optimal resource assignment algorithms. Many cross‐layer optimization‐based rate‐allocation strategies, which consider different objective functions (congestion level, total packet loss, etc.), have been developed for this purpose. The main contributions of the proposed work are twofold. Firstly, an optimal resource assignment framework is being developed in which, based on some network‐specific constraints and by incorporating appropriate video quality metrics, the total weighted QoE of some competing scalable video sources is being optimized based on cross‐layer optimization techniques. Secondly, these optimal rates can be used for differentiated QoE enforcement between multiple competing scalable video sources. The resulting optimal rates can be considered as rate feedbacks for online rate adaptation of a moderate scalable video encoder such as H.264/MPEG‐4 advanced video coding. The aforementioned weight parameters are selected based on the importance of each video sequence's quality and can be associated with some previous service level agreement‐based prices. Some numerical analysis have been presented to validate the theoretical results and to verify the claims. Copyright © 2011 John Wiley & Sons, Ltd.     

Efficient PFD-Based Networking and Buffering Models for Improving Video Quality over Congested Links

Video traffic over the Internet becomes increasingly popular and is expected to comprise the largest proportion of the traffic carried by wired and wireless networks. On the other hand, videos are usually compressed by exploiting spatial and temporal redundancy for the reason of increasing the number of video streams that can be simultaneously carried over links. Unfortunately, receiving high-quality video streaming over the Internet remains a challenge due to the packet loss encountered in the congested wired and wireless links. In addition, the problem is more apparent in wireless links due to not only employing limited system capacity, but also some of the major drawbacks of wireless networks, out of which the bandwidth limitations and link asymmetry which refers to the situation where the forward and reverse paths of a transmission have different channel capacities. Therefore, the wireless hops may be congested which result in dropping many video frames. Additionally, as a result of compressing videos, dependencies among frames and within a frame arise. Consequently, the overall video quality tends to be degraded dramatically. The main challenge is to support the growth of video traffic while keeping the perceived quality of the delivered videos high. In this paper, we extend our previous work concerning improving video traffic over wireless networks through professionally studying the dependencies between video frames and their implications on the overall network performance. In other words, we propose very efficient network and buffer models proportionately to novel algorithms that aim to minimize the cost of aforementioned possible losses by selectively discarding frames based on their contribution to picture quality, namely, partial and selective partial frame discarding policies considering the dependencies between video frames. The performance metrics that are employed to evaluate the performance of the proposed algorithms include the rate of non-decodable frames, peak signal-to-noise ratio, frameput, average buffer occupancy, average packet delay, as well as jitter. Our results are so promising and show significant improvements in the perceived video quality over what is relevant in the current literature. We do not end up to this extent, but rather the effect of producing different bit-stream rates by the FFMPEG codecs on aforementioned performance metrics has been extensively studied.     

Video fusion performance evaluation based on structural similarity and human visual perception

In order to evaluate different video fusion algorithms in temporal stability and consistency as well as in spatial information transfer, a novel objective video fusion quality metric is proposed with the structural similarity (SSIM) index and the perception characteristics of human visual system (HVS) in this paper. Firstly, for each frame, two sub-indices, i.e., the spatial fusion quality index and the temporal fusion quality index, are defined by the weighted local SSIM indices. Secondly, for the current frame, an individual-frame fusion quality measure is obtained by integrating the above two sub-indices. Lastly, the proposed global video fusion metric is constructed as the weighted average of all the individual-frame fusion quality measures. In addition, according to the perception characteristics of HVS, some local and global spatial-temporal information, such as local variance, pixel movement, global contrast, background motion and so on, is employed to define the weights in the proposed metric. Several sets of experimental results demonstrate that the proposed metric can evaluate different video fusion algorithms accurately, and the evaluation results coincide with the subjective results well.     

A new sharpness metric based on local kurtosis,edge and energy information

Sharpness metrics that use the whole frequency spectrum of the image cannot separate the sharpness information from the scene content. The sharpness metrics that use spatial gradients of the edges work only for comparisons among versions of the same image. We have developed a content independent, no-reference sharpness metric based on the local frequency spectrum around the image edges. In this approach, we create an edge profile by detecting edge pixels and assigning them to 8×8 pixel blocks. Then we compute sharpness using the average 2D kurtosis of the 8×8 DCT blocks. However, average kurtosis is highly sensitive to asymmetry in the DCT, e.g. different amounts of energy and edges in the x and y directions, therefore causing problems with different content and asymmetric sharpness enhancement. Thus we compensate the kurtosis using spatial edge extent information and the amount of vertical and horizontal energy in the DCT. The results show high correlation with subjective quality for sharpness-enhanced video and high potential to deal with asymmetric enhancement. For compressed, extremely sharpened and noisy video, the metric correlates with subjective scores up to the point where impairments become strongly noticeable in the subjective quality evaluation. The metric can be used by itself as a control variable for high-quality image capture and display systems, high-quality sharpness enhancement algorithms, and as a key component of a more general overall quality metric.     

一种基于立体视觉显著性的多视点视频比特分配方法

针对多视点立体视频压缩编码,提出了一种基于立 体视觉显著性的比 特分配方法。研究综合利用多视点立体视频数据中场景的运动、深度以及深度边缘信息提取 人眼感兴趣区 域(ROI)的方法;然后根据ROI的划分结果优化区域比特分配。实验结果表 明,本文提出的算法能有效提 高ROI区域的编码性能,同时整体视频的率失真性能有一定程度的提高。     

Evaluation of objective quality metrics for multidimensional video scalability

Multidimensional video scalability refers to the possibility that a video sequence can be adapted according to given conditions of video consumption by adjusting one or more of its features such as frame size, frame rate, and spatial quality. An important issue in implementing an adaptive video distribution scheme using scalability is how to maximize the quality of experience for the delivered contents, which raises a more fundamental issue, that is, how to estimate perceived quality of scalable video contents. This paper evaluates existing state-of-the-art objective quality metrics, including both generic image/video metrics and ones particularly developed for scalable videos, on the problem of quality assessment of multidimensional video scalability. It is shown that, on the whole, some recently developed metrics targeting scalability perform best. The results are thoroughly discussed in relation to the nature of the problem in comparison to what has been reported in existing studies for other problems.     

The impact of interference-aware routing metrics on video streaming in Wireless Mesh Networks

Wireless Mesh Networks form a wireless backbone that provides ubiquitous Internet access and support of multimedia services. In this scenario, traffic crosses multi-hop paths, through mesh routers and gateways, causing high levels of interference. To address this problem, the use of schemes that introduce routing metrics that take into account the characteristics of the interference have been made to improve the application performance. Given the diversity of interference-aware routing metrics for Wireless Mesh Networks, it is necessary to assess the impact of employing these routing metrics on multimedia traffic performance, and in particular, on video streaming. This paper seeks to fill this gap, by using simulation to evaluate the video streaming performance when the most relevant interference-aware routing metrics are used. The degree of video quality can be evaluated from two perspectives, the network viewpoint and the standpoint of the user perception. At the network level, video streaming quality is assessed through IP measures, that is, throughput, delay, jitter and routing overhead. At the user level, ‘Quality of Experience’ metrics are employed to measure the user perception with regard to the video quality. The evaluation of the performance takes account of outdoor and indoor environments. The results of the simulation study have shown that routing metrics based on the information that detects interference using accurate measures achieve a better network and user perception performance. However, depending on the environment (i.e., whether it is indoor or outdoor), all the routing metrics result in a different performance being achieved. Although interference-aware routing metrics affect the performance of both the network and the user levels, there are some cases where they have less impact on the user level, because the user perception parameters are less influenced by the behaviour of the network.     

客观的图象质量测度及其在分割评价中的应用

图象分割是图象分析中的关健步骤。对图象分割的评价是研究分割技术性能的有效手段,其中分割质量测度起着重要作用。本文介绍一类新的客观质量测度并对其性能进行了比较研究。另外为了验证新测度的有效性,本文提出了一种比较合理全面的分割技术分类法。在此基础上选取各类技术中有代表性的算法借助新的质量测度进行了实际评价比较并取得了有意义的结果。     

M2OVQA: Multi-space signal characterization and multi-channel information aggregation for quality assessment of compressed omnidirectional videos

Considering the high requirements for omnidirectional video compression, we propose an objective quality evaluation method to assess quality loss in encoding omnidirectional videos. According to characteristics of 360° videos, we consider multi-space signal characterization (MSSC) to fully characterize the distortions of video signals from spatial/image domains to frequency domains and from image content to motion information, and further consider multi-channel information aggregation (MCIA) to fuse scores from multiple projection planes and temporal divided groups. The main innovation of our method is to establish a universal framework in bridging the connection between typical quality assessment and 360° quality assessment to measure 360° video quality effectively and efficiently. Experimental results show that our method outperforms state-of-the-art 2D quality metrics and quality metrics for omnidirectional images.     

OBJECTIVE IMAGE QUALITY MEASURES AND THEIR APPLICATIONS IN SEGMENTATION EVALUATION

Image segmentation is a critical step of image analysis. Segmentation evaluation is an effective procedure for studying the performance of segmentation techniques, in which quality measure plays an important role. This paper presents a group of new objective quality measures for segmentation evaluation and compares their performances. In addition, to verify the effectiveness of these new measures, an appropriate classification of segmentation is proposed. According to this classification, several representative algorithms from different categories are selected for comparison testing. Some valuable results are obtained and presented.