Multi-scale fidelity measure for image fusion quality assessment

Image fusion is considered an effective enhancing methodology widely included in high-quality imaging systems. Nevertheless, like other enhancing techniques, output quality assessment is made within small sample subjective evaluation studies which are very limited in predicting the human-perceived quality of general image fusion outputs. Simple, blind, universal and perceptual-like methods for assessing composite image quality are still a challenge, partially solved only in particular applications. In this paper, we propose a fidelity measure, called MS-Q_W with two major characteristics related to natural image statistics framework: A multi-scale computation and a structural similarity score. In our experiments, we correlate the scores of our measure with subjective ratings and state of the art measures included in the 2015 Waterloo IVC multi-exposure fusion (MEF) image database. We also use the measure to rank correctly the classical general fusion methods included in the Image Fusion Toolbox for medical, infra-red and multi-focus image examples. Moreover, we study the scores variability and statistical discrimination power with the TNO night vision database using the Friedman test. Finally, we define a new leave one out procedure based on our fidelity measure that selects the best subset of images (within a collection of distorted and unregistered cell phone type images) that provides a defect-free composite output. We exemplify the procedure with the fusion of a collection of images from Latour and Van Dongen paintings suffering from glass highlights and speckle noise, among other artifacts. The proposed multiscale quality measure MS-Q_W demonstrates improvement over the previous single-scale similarity measures towards a fidelity assessment between quantitative image fusion quality metrics and human perceptual qualitative scores.     

Analysis of neural coding through quantization with an information-based distortion measure

We discuss an analytical approach through which the neural symbols and corresponding stimulus space of a neuron or neural ensemble can be discovered simultaneously and quantitatively, making few assumptions about the nature of the code or relevant features. The basis for this approach is to conceptualize a neural coding scheme as a collection of stimulus-response classes akin to a dictionary or 'codebook', with each class corresponding to a spike pattern 'codeword' and its corresponding stimulus feature in the codebook. The neural codebook is derived by quantizing the neural responses into a small reproduction set, and optimizing the quantization to minimize an information-based distortion function. We apply this approach to the analysis of coding in sensory interneurons of a simple invertebrate sensory system. For a simple sensory characteristic (tuning curve), we demonstrate a case for which the classical definition of tuning does not describe adequately the performance of the cell studied. Considering a more involved sensory operation (sensory discrimination), we also show that, for some cells in this system, a significant amount of information is encoded in patterns of spikes that would not be discovered through analyses based on linear stimulus-response measures.     

A measure for the evaluation of multi-focus image fusion at feature level

Multimedia Tools and Applications - Most multi-focus image fusion evaluation methods are based on focus detection and measuring the similarity between the fused image and the whole source images...     

A novel similarity based quality metric for image fusion

A novel objective quality metric for image fusion is presented. The interest of our metric lies in the fact that the redundant regions and the complementary/conflicting regions are treated respectively according to the structural similarity between the source images. The experiments show that the proposed measure is consistent with human visual evaluations and can be applied to evaluate image fusion schemes that are not performed at the same level.     

MTFA as a measure for computer display screen image quality

Computer display screen image quality is known to vary widely. Attempts have been made the world over at defining objective scales to better assess image sharpness and clarity. The experiments reported here were stimulated by the recent US adoption of MTFA≥5 as a minimum quality criteria. A series of ten hard-copy photographs and seven retrievable display screens were developed for a monochrome, cathode ray tube terminal that differed only in objective image quality. This series of display screen images was presented to 22 office workers, who were asked to provide a subjective image quality judgement in the form of a numerical scale ranking. The subjects were instructed to match the display screen with photographs that they estimated to be most comparable in image quality. Finally, the subjects were asked to specify what they considered to be the minimum acceptable visual quality. It was found that the subjects could accurately rank low and minimum quality display images and that their subjective visual quality judgement corresponded well with the objective MTFA quality scale. The minimum acceptable display quality specified by the office worker group corresponded to MTFA≥7, which is substantially higher than the US criteria and would disquality 40% of the video display terminals currently marketed in Sweden.     

No-reference image quality assessment using fusion metric

This paper presents a fusion featured metric for no-reference image quality assessment of natural images. Natural images exhibit strong statistical properties across the visual contents such as leading edge, high dimensional singularity, scale invariance, etc. The leading edge represents the strong presence of continuous points, whereas high singularity conveys about non-continuous points along the curves. Both edges and curves are equally important in perceiving the natural images. Distortions to the image affect the intensities of these points. The change in the intensities of these key points can be measured using SIFT. However, SIFT tends to ignore certain points such as the points in the low contrast region which can be identified by curvelet transform. Therefore, we propose a fusion of SIFT key points and the points identified by curvelet transform to model these changes. The proposed fused feature metric is computationally efficient and light on resources. The neruofuzzy classifier is employed to evaluate the proposed feature metric. Experimental results show a good correlation between subjective and objective scores for public datasets LIVE, TID2008, and TID2013.

     

Novel image fusion scheme based on dependency measure for robust multispectral palmprint recognition

Multispectral palmprint is considered as an effective biometric modality to accurately recognize a subject with high confidence. This paper presents a novel multispectral palmprint recognition system consisting of three functional blocks namely: (1) novel technique to extract Region of Interest (ROI) from the hand images acquired using a contact less sensor (2) novel image fusion scheme based on dependency measure (3) new scheme for feature extraction and classification. The proposed ROI extraction scheme is based on locating the valley regions between fingers irrespective of the hand pose. We then propose a novel image fusion scheme that combines information from different spectral bands using a Wavelet transform from various sub-bands. We then perform the statistical dependency analysis between these sub-bands to perform fusion either by selection or by weighted fusion. To effectively process the information from the fused image, we perform feature extraction using Log-Gabor transform whose feature dimension is reduced using Kernel Discriminant Analysis (KDA) before performing the classification by employing a Sparse Representation Classifier (SRC). Extensive experiments are carried out on a CASIA multispectral palmprint database that shows the strong superiority of our proposed fusion scheme when benchmarked with contemporary state-of-the-art image fusion schemes.     

Multifocus image fusion using focus measure of fractional differential and NSCT

In this paper, a new multifocus image fusion scheme based on fractional differential and NSCT is proposed. Firstly, in virtue of the properties of fractional differential, a novel focus measure in nonsubsampled contourlet transform (NSCT) domain is presented and used to determine which coefficient is from the focused region. Then, based on the imaging principle of the multifocus image and the focus measure, a new selection principle for the lowpass subbands coefficients is developed. Meanwhile, focus measure maximum choosing scheme, namely select the coefficient with maximum focus measure value as the corresponding coefficient of the fused image, is applied to the high-frequency subbands. Finally, the inverse NSCT is employed to reconstruct the fused image and a pleasing fused result is generated. The experimental results show that the proposed method outperforms the conventional multifocus image fusion methods in both subjective and objective qualities.     

Structural and functional medical image fusion using an adaptive Fourier analysis

Multimedia Tools and Applications - Medical image fusion helps in meaningfully combining, the information provided by various imaging sensors, targeting the clinical details of human organs. It is...     

The Hermite transform as an efficient model for local image analysis: An application to medical image fusion

The Hermite transform is introduced as an image representation model that can be used to tackle the problem of fusion in multimodal medical imagery. This model includes some important properties of human visual perception, such as local orientation analysis and the Guassian derivative model of early vision. Local analysis is achieved by windowing the image with a Gaussian function, then a local expansion into orthogonal polynomials takes place at every window position. Expansion coefficients are called Hermite coefficients and it is shown that they can be directly obtained by convolving the image with Gaussian derivative filters, in agreement with psychophysical insights of human visual perception. A compact representation can be obtained by locally steering the Hermite coefficients towards the direction of local maximum energy. Image fusion is achieved by combining the steered Hermite coefficients of both source images with the method of verification of consistency. Fusion results are compared with a competitive wavelet-based technique, proving that the Hermite transform provides better reconstruction of relevant image structures.     

基于图像距离测度的置乱效果评价法

提出了一种修改的图像距离进行图像置乱效果评价的方法.针对传统欧氏距离不能真实刻画置乱图像间差异程度的不足,考虑到图像中任意位置灰度信息与其周围邻域灰度存在紧密相关,将一种具有黎曼度量特性的图像距离进行修改并用来构造一种描述图像置乱效果好坏的评价函数.通过大量的实验得出,该评价方法是能够保持主客观评价的一致性.     

可见光与红外图像融合质量评价指标分析

目的 客观评价作为图像融合的重要研究领域,是评价融合算法性能的有力工具。目前,已有几十种不同类型的评价指标,但各应用领域包括可见光与红外图像融合,仍缺少统一的选择依据。为了方便比较不同融合算法性能,提出一种客观评价指标的通用分析方法并应用于可见光与红外图像融合。方法 将可见光与红外图像基准数据集中的客观评价指标分为两类,分别是基于融合图像的评价指标与基于源图像和融合图像的评价指标。采用Kendall相关系数分析融合指标间的相关性,聚类得到指标分组;采用Borda计数排序法统计算法的综合排序,分析单一指标排序和综合排序的相关性,得到一致性较高的指标集合;采用离散系数分析指标均值随不同算法的波动程度,选择充分体现不同算法间差异的指标;综合相关性分析、一致性分析及离散系数分析,总结具有代表性的建议指标集合。结果 在13对彩色可见光与红外和8对灰度可见光与红外两组图像源中,分别统计分析不同图像融合算法的客观评价数据,得到可见光与红外图像融合的建议指标集(标准差、边缘保持度),作为融合算法性能评估的重要参考。相较于现有方法,实验覆盖20种融合算法和13种客观评价指标,并且不依赖主观评价结果。结论...     

Content-based image quality metric using similarity measure of moment vectors

In this paper, the similarity of moment vectors between the test and the reference image blocks together with the result from the block classification are used in the formulation of an image quality metric (IQM). First, the reference and the test images are divided into non-overlapping 8×8 blocks and transformed into moment domain using Discrete Tchebichef Transform. The moment features are then used in two operations: the local quality index calculation and the image content (block) classification. The local quality index is obtained from the similarity measure of moment vectors between the reference and the test image blocks. Next, the content of each reference image block is classified into three types: “plain”, “edge” and “texture”, based on its moment energy level and moment energy distribution. The local quality indices obtained from all the image blocks are then averaged based on the block types to obtain three mean quality scores for each test image. The performance of these three mean quality scores and their combinations are studied using the LIVE database. The results show that the performance of the metric is significantly improved by combining the mean quality scores from the edge and texture image region. The best combination (the proposed metric) is then compared with five other IQMs using the LIVE database and four other independent databases. The results show that the proposed metric performs comparatively well for all the databases.     

Towards an intelligent multi-sensor satellite image analysis based on blind source separation using multi-source image fusion

In this paper we propose a new approach for land cover classification using blind sources separation (BSS) and satellite image fusion methods simultaneously. Satellite image pixels are represented by radiometric values where each pixel is considered as a mixture of several independent sources. The BSS methods were developed in order to extract maximum information from different statistical features such as spatial correlation and local high order statistics. The statistical independence of the sources can be obtained through the joint approximate diagonalization of eigen-matrix in two dimensions (JADE-2D) algorithm. A reduction of spatial correlation can be obtained using the second order blind identification in two dimensions (SOBI-2D) algorithm. Non-Gaussianity can be measured using the fast-independent component analysis in two dimensions (Fast-ICA-2D) algorithm. These algorithms allow extraction of features by estimating the source images, mixing and un-mixing the matrix. These source images will be used by our framework as secondary knowledge, which is useful for a supervised classification.     

基于双目融合的无参考立体图像质量评价

针对对称失真和非对称失真图像的评价问题,提出了一种基于双目融合的无参考立体图像质量评价方法。首先,分别将立体图像的左、右视点图像分解成拉普拉斯金字塔序列,利用图像平均梯度和区域能量确定各层融合系数,在双目加权模型的基础上逐层融合两序列并重构合成图像。然后,提取左、右视点图像、合成图像的多尺度多方向频域变换特征和对比度、熵、能量、逆差分矩特征。最后,将特征参数作为支持向量回归模型的输入进行训练并预测图像质量。在LIVE 3D phaseⅠ和LIVE 3D phaseⅡ图像库上作相关性分析,其Pearson线性相关系数和Spearman等级相关系数均分别达到0.96和0.95以上。结果表明,本文方法对立体图像质量的预测结果与主观评价值具有较高的一致性。     

基于曲波活性测度的SAR与多光谱图像融合

曲波变换是一种更适合于图像处理的多尺度几何分析方法,具有很强的方向性。结合活性测度将其应用于合成孔径雷达(SAR)图像和多光谱(MS)图像融合可以更好地表示图像中的有用特征。首先,对多光谱图像和SAR图像的R、G、B三波段分别进行曲波变换,粗尺度系数采用3×3窗口系数活性测度进行融合,细节尺度直接取大,对粗尺度和细节尺度系数重构后得到最终融合结果。采用熵、平均梯度、信噪比和扭曲程度对融合结果进行评价。实验结果表明,基于曲波活性测度的融合方法在保持MS光谱信息和提高空间分辨率上都有较好的结果。     

An improved multi-focus image fusion algorithm based on multi-scale weighted focus measure

Applied Intelligence - This paper focuses on developing an improved multi-focus image fusion (MFIF) algorithm. Existing spatial domain algorithms dependent on the obtained fusion decision map still...     

Multi-focus image fusion using a morphology-based focus measure in a quad-tree structure

Finite depth-of-field poses a problem in light optical imaging systems since the objects present outside the range of depth-of-field appear blurry in the recorded image. Effective depth-of-field of a sensor can be enhanced considerably without compromising the quality of the image by combining multi-focus images of a scene. This paper presents a block-based algorithm for multi-focus image fusion. In general, finding a suitable block-size is a problem in block-based methods. A large block is more likely to contain portions from both focused and defocused regions. This may lead to selection of considerable amount of defocused regions. On the other hand, small blocks do not vary much in relative contrast and hence difficult to choose from. Moreover, small blocks are more affected by mis-registration problems. In this work, we present a block-based algorithm which do not use a fixed block-size and rather makes use of a quad-tree structure to obtain an optimal subdivision of blocks. Though the algorithm starts with blocks, it ultimately identifies sharply focused regions in input images. The algorithm is simple, computationally efficient and gives good results. A new focus-measure called energy of morphologic gradients is introduced and is used in the algorithm. It is comparable with other focus measures viz.energy of gradients, variance, Tenengrad, energy of Laplacian and sum modified Laplacian. The algorithm is robust since it works with any of the above focus measures. It is also robust against pixel mis-registration. Performance of the algorithm has been evaluated by using two different quantitative measures.