Multimodal image/video fusion rule using generalized pixel significance based on statistical properties of the neighborhood

Image fusion has been receiving increasing attention in the research community with the aim of investigating general formal solutions to a wide spectrum of applications such as multifocus, multiexposure, multispectral ( $IR$ -visible) and multimodal medical (CT and MRI) image and video fusion. While there exist many fusion techniques for each of these applications, it is difficult to formulate a common fusion technique that works equally well for all these applications. This is mainly because of the different characteristics of the images involved in various applications and the correspondingly different requirements on the fused image. In this work, we propose a common generalized fusion framework for all these classes, based on the statistical properties of local neighborhood of a pixel. As the eigenvalue of the unbiased estimate of the covariance matrix of an image block depends on the strength of edges in that block, we propose to employ it to compute a quantity we call as the significance of a pixel. This generalized pixel significance in turn can be used as a measure of the useful information content in that block, and hence can be used in the fusion process. Several data sets were fused to compare the results with various recently published methods. The analysis shows that for all the image types into consideration, the proposed methods improve the quality of the fused image, both visually and quantitatively, by preserving all the relevant information.