Structural similarity quality metrics in a coding context: exploring the space of realistic distortions

Perceptual image quality metrics have explicitly accounted for human visual system (HVS) sensitivity to subband noise by estimating just noticeable distortion (JND) thresholds. A recently proposed class of quality metrics, known as structural similarity metrics (SSIM), models perception implicitly by taking into account the fact that the HVS is adapted for extracting structural information from images. We evaluate SSIM metrics and compare their performance to traditional approaches in the context of realistic distortions that arise from compression and error concealment in video compression/transmission applications. In order to better explore this space of distortions, we propose models for simulating typical distortions encountered in such applications. We compare specific SSIM implementations both in the image space and the wavelet domain; these include the complex wavelet SSIM (CWSSIM), a translation-insensitive SSIM implementation. We also propose a perceptually weighted multiscale variant of CWSSIM, which introduces a viewing distance dependence and provides a natural way to unify the structural similarity approach with the traditional JND-based perceptual approaches.     

Human visual weighted progressive image transmission

A progressive image transmission scheme which combines transform coding with the human visual system (HVS) model is developed. The adaptive transform coding of W.H. Chen and C.H. Smith (1977) is utilized to classify an image into four equally populated subblocks based on their AC energies. The modulation transfer function (MTF) of the HVS model is obtained experimentally, based on processing a number of test images. A simplified technique for incorporating the MTF into the discrete cosine transform (DCT) domain is utilized. In the hierarchical image buildup, the image is first reconstructed from the DC coefficients of all subblocks. Further transmission hierarchy of transform coefficients and consequent image buildup are dependent on their HVS weighted variances. The HVS weighted reconstructed images are compared to the ones without any weighting at several stages. The HVS weighted progressive image transmission results in perceptually higher quality images compared to the unweighted scheme     

Eye-movement-controlled transform image coders

An image-coding scheme which combines transform coding with a human visual system (HVS) model is described. The system includes an eye tracker to pick up the point of regard of a single viewer. One can then utilize that the acuity of the HVS is less in the peripheral vision than in the central part of the visual field. A model of the decreasing acuity of the HVS, which can be applied to a wide class of transform coders is described. Such a coding system has large potential for data compression.In this paper, we have incorporated the model into four different transform coders, one from each of the main classes of transform coders. Two of the coders are block-based decomposition schemes, the discrete cosine transform-based JPEG coder and a lapped transform scheme. The two others are subband-based decomposition schemes, a wavelet based and a wavelet packet-based scheme.     

A human visual system-driven image segmentation algorithm

This paper presents a novel image segmentation algorithm driven by human visual system (HVS) properties. Segmentation quality metrics, based on perceptual properties of HVS with respect to segmentation, are integrated into an energy function. The energy function encodes the HVS properties from both region-based and boundary-based perspectives, where the just-noticeable difference (JND) model is employed when calculating the difference between the image contents. Extensive experiments are carried out to compare the performances of three variations of the presented algorithm and several representative segmentation and clustering algorithms available in the literature. The results show superior performance of our approach.     

A weighted full-reference image quality assessment based on visual saliency

In full reference image quality assessment (IQA), the images without distortion are usually employed as reference, while the structures in both reference images and distorted images are ignored and all pixels are equally treated. In addition, the role of human visual system (HVS) is not taken account into subjective IQA metric. In this paper, a weighted full-reference image quality metric is proposed, where a weight imposed on each pixel indicates its importance in IQA. Furthermore, the weights can be estimated via visual saliency computation, which can approximate the subjective IQA via exploiting the HVS. In the experiments, the proposed metric is compared with several objective IQA metrics on LIVE release 2 and TID 2008 database. The results demonstrate that SROCC and PLCC of the proposed metric are 0.9647 and 0.9721, respectively,which are higher than other methods and it only takes 427.5 s, which is lower than that of most other methods.     

Perceptually lossless coder for volumetric medical image data

With the development of modern imaging techniques, every medical examination would result in a huge volume of image data. Analysis, storage and/or transmission of these data demands high compression without any loss of diagnostically significant data. Although, various 3-D compression techniques have been proposed, they have not been able to meet the current requirements. This paper proposes a novel method to compress 3-D medical images based on human vision model to remove visually insignificant information. The block matching algorithm applied to exploit the anatomical symmetry remove the spatial redundancies. The results obtained are compared with those of lossless compression techniques. The results show better compression without any degradation in visual quality. The rate-distortion performance of the proposed coders is compared with that of the state-of-the-art lossy coders. The subjective evaluation performed by the medical experts confirms that the visual quality of the reconstructed image is excellent.     

Low bit-rate compression of underwater image based on human visual system

Image plays an irreplaceable role compared with the text and sound in the underwater data collection and transmission researches. However, it suffers from the limited bandwidth of the underwater acoustic communication which cannot afford the large image data. Compressing the image data before transmission is an inevitable process in the underwater image communication. As usual, the natural image compression methods are directly applied to the underwater scene. As we all know, underwater image has different degradation from the natural one due to the optical transmission property. Low illumination in underwater will cause more seriously blurring and color fading than that in the air. It is a great challenge to decrease the bit-rate of the underwater image while preserving the compressed image quality as much as possible. In this paper, the Human Visual System (HVS) is taken into account during the compressing and the evaluating stages for the underwater image communication. We present a new methodology for underwater image compression. Firstly, by taking the human visual system into account, the chrominance perception operator is proposed in this paper to neglect the imperceptible chrominance shift which is widely exited in the underwater imaging to improve the image compression rate. Secondly, depth of field(DOF) of underwater image is usually shallow and most of the usable image has targets in it. An ROI extraction algorithm based on Boolean map detection is then used for the underwater image compression so as to reduce the bitrate of the compressed image. Furthermore, the underwater image is grainy and low contrast, that means the degradation happens in some regions of the image would not be perceived. Just notice difference(JND) sensing algorithm based on the spatial and frequency domain masking feature of HVS is also considered in the image processing. By combining the three aspects above, hybrid wavelet and asymmetric coding are used together to promote the underwater image compression, so that the image can have better quality and less redundancy. Experiments show that the proposed method can make full use of the inherent characteristics of underwater images, and maximize the visual redundancy of underwater images without reducing the visual perception quality of reconstructed images.     

A fast and low memory image coding algorithm based on lifting wavelet transform and modified SPIHT

Due to its excellent rate–distortion performance, set partitioning in hierarchical trees (SPIHT) has become the state-of-the-art algorithm for image compression. However, the algorithm does not fully provide the desired features of progressive transmission, spatial scalability and optimal visual quality, at very low bit rate coding. Furthermore, the use of three linked lists for recording the coordinates of wavelet coefficients and tree sets during the coding process becomes the bottleneck of a fast implementation of the SPIHT. In this paper, we propose a listless modified SPIHT (LMSPIHT) approach, which is a fast and low memory image coding algorithm based on the lifting wavelet transform. The LMSPIHT jointly considers the advantages of progressive transmission, spatial scalability, and incorporates human visual system (HVS) characteristics in the coding scheme; thus it outperforms the traditional SPIHT algorithm at low bit rate coding. Compared with the SPIHT algorithm, LMSPIHT provides a better compression performance and a superior perceptual performance with low coding complexity. The compression efficiency of LMSPIHT comes from three aspects. The lifting scheme lowers the number of arithmetic operations of the wavelet transform. Moreover, a significance reordering of the modified SPIHT ensures that it codes more significant information belonging to the lower frequency bands earlier in the bit stream than that of the SPIHT to better exploit the energy compaction of the wavelet coefficients. HVS characteristics are employed to improve the perceptual quality of the compressed image by placing more coding artifacts in the less visually significant regions of the image. Finally, a listless implementation structure further reduces the amount of memory and improves the speed of compression by more than 51% for a 512×512 image, as compared with that of the SPIHT algorithm.     

Image coding by block prediction of multiresolution subimages

The redundancy of the multiresolution representation has been clearly demonstrated in the case of fractal images, but it has not been fully recognized and exploited for general images. Fractal block coders have exploited the self-similarity among blocks in images. We devise an image coder in which the causal similarity among blocks of different subbands in a multiresolution decomposition of the image is exploited. In a pyramid subband decomposition, the image is decomposed into a set of subbands that are localized in scale, orientation, and space. The proposed coding scheme consists of predicting blocks in one subimage from blocks in lower resolution subbands with the same orientation. Although our prediction maps are of the same kind of those used in fractal block coders, which are based on an iterative mapping scheme, our coding technique does not impose any contractivity constraint on the block maps. This makes the decoding procedure very simple and allows a direct evaluation of the mean squared error (MSE) between the original and the reconstructed image at coding time. More importantly, we show that the subband pyramid acts as an automatic block classifier, thus making the block search simpler and the block matching more effective. These advantages are confirmed by the experimental results, which show that the performance of our scheme is superior for both visual quality and MSE to that obtainable with standard fractal block coders and also to that of other popular image coders such as JPEG.     

Quality measurement and use of pre-processing in image compression

Traditional quality measures for image coding, such as the peak signal-to-noise ratio, assume that the preservation of the original image is the desired goal. However, pre-processing images prior to encoding, designed to remove noise or unimportant detail, can improve the overall performance of an image coder. Objective image quality metrics obtained from the difference between the original and coded images cannot properly assess this improved performance. This paper proposes a new methodology for quality metrics that differentially weighs the changes in the image due to pre-processing and encoding. These new quality measures establish the value of pre-processing for image coding and quantitatively determine the performance improvement that can be thus achieved by JPEG and wavelet coders.     

Model-based digital halftoning

Digital halftoning is the process of generating a pattern of pixels with a limited number of colors that, when seen by the human eye, is perceived as a continuous-tone image. Digital halftoning is used to display continuous-tone images in media in which the direct rendition of the tones is impossible. The most common example of such media is ink or toner on paper, and the most common rendering devices for such media are, of course, printers. Halftoning works because the eye acts as a spatial low-pass filter that blurs the rendered pixel pattern, so that it is perceived as a continuous-tone image. Although all halftoning methods rely at least implicitly, on some understanding of the properties of human vision and the display device, the goal of model-based halftoning techniques is to exploit explicit models of the display device and the human visual system (HVS) to maximize the quality of the displayed images. Based on the type of computation involved, halftoning algorithms can be broadly classified into three categories: point algorithms (screening or dithering), neighborhood algorithms (error diffusion), and iterative algorithms [least squares and direct binary search (DBS)]. All of these algorithms can incorporate HVS and printer models. The best halftone reproductions, however, are obtained by iterative techniques that minimize the (squared) error between the output of the cascade of the printer and visual models in response to the halftone image and the output of the visual model in response to the original continuous-tone image.     

A multi-factors approach for image quality assessment based on a human visual system model

In this paper, a multi-factor full-reference image quality index is presented. The proposed visual quality metric is based on an effective Human Visual System model. Images are pre-processed in order to take into account luminance masking and contrast sensitivity effects. The proposed metric relies on the computation of three distortion factors: blockiness, edge errors and visual impairments, which take into account the typical artifacts introduced by several classes of coders. A pooling algorithm is used in order to obtain a single distortion index. Results show the effectiveness of the proposed approach and its consistency with subjective evaluations.     

一种基于小波变换的低比特率混合图像编码方法

本文提出了一种基于小波变换的低比特率图像编码方法,利用人的视觉特性对高频高活跃度系数进行调制,为了保护图像能量,对高频高能系数进行标量量化后采用自适应算术编码,采用高分辨率级子图像矢量分类信息由低分辨率级子图像矢量自动产生,提高了压缩比。实验表明采用此编码方法在高压缩比的情况下获得了较好的图像质量。     

Adaptive Block-size Transform based Just-Noticeable Difference model for images/videos

In this paper, we propose a novel Adaptive Block-size Transform (ABT) based Just-Noticeable Difference (JND) model for images/videos. Extension from 8×8 Discrete Cosine Transform (DCT) based JND model to 16×16 DCT based JND is firstly performed by considering both the spatial and temporal Human Visual System (HVS) properties. For still images or INTRA video frames, a new spatial selection strategy based on the Spatial Content Similarity (SCS) between a macroblock and its sub-blocks is proposed to determine the transform size to be employed to generate the JND map. For the INTER video frames, a temporal selection strategy based on the Motion Characteristic Similarity (MCS) between a macroblock and its sub-blocks is presented to decide the transform size for the JND. Compared with other JND models, our proposed scheme can tolerate more distortions while preserving better perceptual quality. In order to demonstrate the efficiency of the ABT-based JND in modeling the HVS properties, a simple visual quality metric is designed by considering the ABT-based JND masking properties. Evaluating on the image and video subjective databases, the proposed metric delivers a performance comparable to the state-of-the-art metrics. It confirms that the ABT-based JND consists well with the HVS. The proposed quality metric also is applied on ABT-based H.264/Advanced Video Coding (AVC) for the perceptual video coding. The experimental results demonstrate that the proposed method can deliver video sequences with higher visual quality at the same bit-rates.     

Color image denoising with wavelet thresholding based on human visual system model

Recent research in transform-based image denoising has focused on the wavelet transform due to its superior performance over other transform. Performance is often measured solely in terms of PSNR and denoising algorithms are optimized for this quantitative metric. The performance in terms of subjective quality is typically not evaluated. Moreover, human visual system (HVS) is often not incorporated into denoising algorithm. This paper presents a new approach to color image denoising taking into consideration HVS model. The denoising process takes place in the wavelet transform domain. A Contrast Sensitivity Function (CSF) implementation is employed in the subband of wavelet domain based on an invariant single factor weighting and noise masking is adopted in succession. Significant improvement is reported in the experimental results in terms of perceptual error metrics and visual effect.     

A wavelet visible difference predictor

We describe a model of the human visual system (HVS) based on the wavelet transform. This model is largely based on a previously proposed model, but has a number of modifications that make it more amenable to potential integration into a wavelet based image compression scheme. These modifications include the use of a separable wavelet transform instead of the cortex transform, the application of a wavelet contrast sensitivity function (CSF), and a simplified definition of subband contrast that allows one to predict the noise visibility directly from the wavelet coefficients. Initially, we outline the luminance, frequency, and masking sensitivities of the HVS and discuss how these can be incorporated into the wavelet transform. We then outline a number of limitations of the wavelet transform as a model of the HVS, namely the lack of translational invariance and poor orientation sensitivity. In order to investigate the efficacy of this wavelet based model, a wavelet visible difference predictor (WVDP) is described. The WVDP is then used to predict visible differences between an original and compressed (or noisy) image. Results are presented to emphasize the limitations of commonly used measures of image quality and to demonstrate the performance of the WVDP. The paper concludes with suggestions on how the WVDP can be used to determine a visually optimal quantization strategy for wavelet coefficients and produce a quantitative measure of image quality.     

基于视觉特性和小波分解的数字水印隐藏方法

本文提出了一种隐藏数字水印的新方法,该方法所隐藏的不是传统的序列码或比特流,而是将水印作为一幅二值图像来处理;并结合人眼视觉模型（ＨＶＳ）和图像的ＤＷＴ多尺度分解来隐藏水印。实验表明这种新方法在降低原始图像变换后视觉失真和提取的被隐藏水印图像失真两方面都达到较好的效果,鲁棒性也较好,这是一种很有发展前景的数字水印隐藏新方法。     

基于人眼视觉特性的NSCT医学图像自适应融合

医学图像融合对于临床诊断具有重要的应用价值。针对多模态医学图像特性,本文提出一种基于人类视觉特性的医学图像自适应融合方法。首先,对经配准的源图像进行非间隔采样轮廓变换（（Nonsubsampled Coutour-let,NSCT）多尺度分解,得到低频子带和若干高频方向子带;其次,根据低频子带集中了大部分源图像能量和决定图像轮廓的特点,采用区域能量与平均梯度相结合的方法进行融合;根据人眼对图像对比度及边缘、纹理的高敏感度,在高频子带系数的选取时提出区域拉普拉斯能量、方向对比度与脉冲耦合神经网络（Pulse Coupled Neural Network,PCNN）相结合的融合策略;进而,提出了把与人类视觉高度一致的加权结构相似度（Weighted Structure Similarity,WSSIM）作为图像融合目标函数,自适应地获取各子带的最优权值;最后,对灰度图像和彩色图像进行了大量融合比较实验,并对不同融合方法进行分析对比。实验结果表明：本文算法不仅可以有效保留源图像的信息,而且可以使融合图像灰度级更分散,更好地保留了图像边缘信息,具有更好的视觉效果。     

基于DWT和视觉加权的图像质量评价方法研究

提出了一种新的基于小波变换和视觉加权的图像质量客观评价方法(WVWPSNR,wavelet and vision weighted peak signal noise ratio)。该方法将图像的DWT和HVS特性相结合,利用子图分解以及视觉加权处理实现图像质量的客观评价。对多幅分别经过JPEG压缩、JPEG2000压缩、White Noise(白噪声)、Gaussian Blur(高斯模糊)、FastFading降质的图像进行测试,实验结果表明该方法可靠、有效,与主观评价结果更接近。尤其是该方法对JPEG压缩图像的质量评价性能较好。