Pan-sharpening using a guided filter

Pan-sharpening aims to integrate the spatial details of a high-resolution panchromatic (Pan) image with the spectral information of low-resolution multispectral (MS) images to produce high-resolution MS images. The key is to appropriately estimate the missing spatial details of the MS images while preserving their spectral contents. However, many existing methods extract the spatial details from the Pan image without fully considering the structures of the MS images, resulting in spectral distortion due to redundant detail injection. A guided filter can transfer the structures of the MS images into the intensity component or the low-pass approximation of the Pan image. Using the guided filter, we propose two novel pan-sharpening methods to reduce the redundant details among the MS and Pan images. Specifically, we extract the missing spatial details of the MS images by minimizing the difference between the Pan image and its corresponding filtering output, with the help of the MS images. Two different ways of using the MS images as guided images lead to two proposed methods, which can be grouped into component substitution (CS) family. Extensive experimental results over three data sets collected by different satellite sensors demonstrate the effectiveness of the proposed methods.     

The multiscale directional bilateral filter and its application to multisensor image fusion

In this paper, a novel multiscale geometrical analysis called the multiscale directional bilateral filter (MDBF) which introduces the nonsubsampled directional filter bank into the multiscale bilateral filter is proposed. Through combining the characteristic of preserving edge of the bilateral filter with the ability of capturing directional information of the directional filter bank, the MDBF can better represent the intrinsic geometrical structure of images. The MDBF, which is a multiscale, multidirectional and shift-invariant image decomposition scheme, is used to fuse multisensor images in this paper. The source images are first decomposed into the directional detail subbands and the approximation subbands via the MDBF. Then, the directional detail subbands and the approximation subbands are fused according to the given fusion rule, respectively. Finally, the inverse MDBF is applied to the fused subbands to obtain the fused image. Experimental results over visible and infrared images and medical images demonstrate the superiority of our method compared with conventional methods in terms of visual inspection and objective measures.     

基于 MTF 和变分的全色与多光谱图像融合模型

Pan-sharpening将高分辨率图像全色(Panchromatic, Pan)波段的空间细节注入多光谱(Multispectral, MS)波段, 以生成同时具有高光谱和高空间分辨率的多光谱图像. 为改善融合效果, 需要考虑多光谱和全色波段的调制传输函数(Modulation transfer function, MTF). 本文提出了一个新的基于MTF和变分的Pan-sharpening模型. 该模型的能量泛函包括两项, 第1项为细节注入项, 基于高通滤波器从Pan波段中提取细节信息并注入融合图像;第2项为光谱保真项, 基于MTF设计多孔小波的低通滤波器以保持MS波段的多光谱信息. 在QuickBird、IKONOS和GeoEye数据集上的融合结果表明, 该模型可以生成同时具有高空间和高光谱质量的融合图像, 融合效果优于AWLP、IHS_BT、HPM-CC-PSF、NAWL、快速变分等算法.     

基于双边滤波的多尺度图像融合

提出一种彩色图像和近红外图像的融合算法,先给出一种改进核函数的双边滤波器,然后应用该双边滤波器和高斯滤波器相结合多次作用于彩色图像和近红外图像,得到低频图像、边缘图像、细节图像等不同尺度的图像,在不同尺度图像融合的权重选择上,提出一种自适应的权重选择法。该方法综合考虑参与融合的各像素点的邻域来确定各点权重,不仅抑制了融合图像噪声,还使融合图像色彩更贴近原彩色图像。实验表明该方法得到的融合图像具有符合人眼观测习惯的颜色信息,此外融合图像的纹理、边缘等细节信息比原彩色图像更丰富。     

基于多尺度边缘保持分解与PCNN的医学图像融合

在医学图像融合过程中,传统多尺度分析方法多采用线性滤波器,由于无法保留图像边缘特征导致分解阶段的强边缘处出现模糊,从而产生光晕。为提高融合图像的视觉感知效果,通过结合多尺度边缘保持分解方法与脉冲耦合神经网络（PCNN）,提出一种新的图像融合方法。对源图像进行加权最小二乘滤波分解得到图像的基础层和细节层,采用高斯滤波器对基础层进行二次分解得到低频层和边缘层,将分解过程中每级边缘层和细节层叠加构建高频层,并引入非下采样方向滤波器组进行方向分析。在此基础上,利用改进的空间频率以及区域能量激励PCNN融合高频层和低频层,通过逆变换得到最终的融合图像。实验结果表明,该方法能够突出医学图像的边缘轮廓并增强图像细节,可将更多的显著特征从源图像分离并转移到融合图像中。     

基于边缘增强与光谱特性保持的Pan-sharpening融合模型

为生成兼具高光谱质量与高空间质量的融合图像,本文提出了一种新的Pan-sharpening变分融合模型.通过拟合退化后的全色（Panchromatic,Pan）波段图像与低分辨率多光谱（Multispectral,MS）波段图像间的线性关系得到各波段MS图像的权重系数,计算从Pan图像抽取的空间细节;基于全色波段图像的梯度定义加权函数,增强了图像的强梯度边缘并对因噪声而引入的虚假边缘进行了抑制,有效地保持了全色波段图像中目标的几何结构;基于MS波段传感器的调制传输函数定义低通滤波器,自适应地限制注入空间细节的数量,显著降低了融合MS图像的光谱失真;针对Pan-sharpening模型的不适定性问题,引入L1正则化能量项,保证了数值解的稳定性.采用Split Bregman数值方法求解能量泛函的最优解,提高了算法的计算效率.QuickBird、IKONOS和GeoEye-1数据集上的实验结果表明,模型的综合融合性能优于MTF-CON、AWLP、SparseFI、TVR和MTF-Variational等算法.     

一种改进的多光谱双边滤波图像融合算法

针对双边滤波图像融合算法核函数复杂、人为设计参数较多、计算时间较长的问题,文章提出一种改进的双边滤波融合算法实现彩色图像和近红外图像的融合。文章简化了双边滤波算法中的核函数,取消了人为设计参数;在对彩色图像进行双边滤波融合时,取彩色图像与近红外图像中相应像素间差异性大的作为相似度项,避免了融合图像的纹理、边缘被平滑掉。通过对实际采集的图像应用几种融合方法进行测试,表明文章提出的改进双边滤波融合方法同Eric.P等人的方法相比速度提高6倍多,且人为设计参数较少,得到的融合图像色彩同源图像色彩接近,图像细节更清晰,具有一定应用价值。     

基于联合双边滤波器的Kinect深度图像滤波算法

针对Kinect镜头采集的深度图像一般有噪声和黑洞现象,直接应用于人体动作跟踪和识别等系统中效果差的问题,提出一种基于联合双边滤波器的深度图像滤波算法。算法利用联合双边滤波原理,将Kinect镜头同一时刻采集的深度图像和彩色图像作为输入,首先,用高斯核函数计算出深度图像的空间距离权值和RGB彩色图像的灰度权值;然后,将这两个权值相乘得到联合滤波权值,并利用快速高斯变换替换高斯核函数,设计出联合双边滤波器;最后,用此滤波器的滤波结果与噪声图像进行卷积运算实现Kinect深度图像滤波。实验结果表明,所提算法应用在人体动作识别和跟踪系统后,可显著提高在背景复杂场景中的抗噪能力,识别正确率提高17.3%,同时所提算法的平均耗时为371ms,远低于同类算法。所提算法保持了联合双边滤波平滑保边的优点,由于引入彩色图像作为引导图像,去噪的同时也能对黑洞进行修补,因此该算法在Kinect深度图像上的去噪和修复效果优于经典的双边滤波算法和联合双边滤波算法,且实时性强。     

A systematic investigation of geostatistical image fusion for the improvement of the spectral fidelity and spatial detail in Landsat MS imagery

This is the first systematic investigation into the assumptions of image fusion using regression Kriging (RK) – a geostatistical method – illustrated with Landsat MS (multispectral) and SPOT (Satellite Pour l’Observation de la Terre) panchromatic images. The efficiency of different linear regression and Kriging methods in the fusion process is examined by visual and quantitative indicators. Results indicate a trade-off between spectral fidelity and spatial detail preservation for the GLS (generalized least squares regression) and OLS (ordinary least squares regression) methods in the RK process: OLS methods preserve more spatial detail, while GLS methods retain more spectral information from the MS images but at a greater computational cost. Under either OK (ordinary Kriging) or UK (universal Kriging) with either OLS or GLS, the spherical variogram improves spatial details from the panchromatic image, while the exponential variogram maintains more spectral information from the MS image. Overall, RK-based fusion methods outperform conventional fusion approaches from both the spectral and spatial point of view.     

Panchromatic sharpening of remote sensing images using a multiscale Kalman filter

This paper presents a solution to the problem of enhancing the spatial resolution of multispectral images with high-resolution panchromatic observations. The proposed method exploits the undecimated discrete wavelet transform, which is an octave bandpass representation achieved from a conventional discrete wavelet transform by omitting all decimators and up-sampling the wavelet filter bank, and the vector multiscale Kalman filter, which is used to model the injection process of wavelet details. Kalman modelization is exploited by spatial detail analysis at coarser scales in which multispectral and panchromatic representations are known. Results are presented and discussed on very-high resolution images acquired by Quickbird satellite systems. Fusion simulations on spatially degraded data and fusion tests at the full scale reveal that an accurate and reliable PAN-sharpening is achieved by the proposed method.     

基于核与局部信息的多维度模糊聚类图像分割算法

在以聚类分析为背景的图像分割算法中,引入局部信息是为了在保留图像细节的同时尽可能地减少噪声.在模糊C均值算法基础上,提出了一种基于核与局部信息的多维度模糊聚类分析方法来权衡图像中的噪声和细节.该算法引入2个基于局部信息的图像变体,即平滑和锐化处理后的图像,使之与原始图像一起构成多维度的灰度值向量来替换原始单维的灰度值; 再利用核方法提高其鲁棒性; 最后添加一个邻域隶属度差异惩罚项很好地修正和增强了最终的分割效果.在人工合成图片的去噪实验中,所提方法取得了近99%的分割正确率,优于Nystrom归一化分割(NNcut)和基于模糊局部信息C均值(FLICM)算法;同时在自然图片和医学图片的对比实验以及参数调控实验中,展现出了其在处理图像噪声和细节时灵活、稳定、健壮且易于调控的特点.     

Image denoising using bilateral filter and Gaussian scale mixtures in shiftable complex directional pyramid domain

For image denoising, the main challenge is how to preserve the information-bearing structures such as edges and textures to get satisfactory visual quality when improving the signal-to-noise-ratio (SNR). Edge-preserving image denoising has become a very intensive research topic. In this paper, we describe a method for removing noise from digital images, based on bilateral filter and Gaussian scale mixtures (GSM) in shiftable complex directional pyramid (also named Pyramidal Dual-Tree Directional Filter Bank, PDTDFB) domain. Firstly, the noisy image is decomposed into different subbands of frequency and orientation responses using a PDTDFB transform. Secondly, the bilateral filter, which is a nonlinear filter that does spatial averaging without smoothing edges, is applied on the approximation subband. Finally, the distribution of detail subbands of PDTDFB coefficients is modeled with GSM, and the statistical model is then used to obtain the denoised detail coefficients from the noisy image decomposition by Bayes least squares estimator. Extensive experimental results demonstrate that our method can obtain better performances in terms of both subjective and objective evaluations than those state-of-the-art denoising techniques. Especially, the proposed method can preserve edges very well while removing noise.     

Improved bilateral filter for suppressing mixed noise in color images

It is a challenging problem to suppress mixed noise in color images. The traditional bilateral filter can excellently reduce additive noise without destroying image edges and details, but it fails to remove impulsive noise. This paper presents an improved bilateral filtering method, which can simultaneously suppress both impulsive and additive noise. The proposed solution first introduces a new weighting function to the bilateral filtering mechanism, which is experimentally more effective than the traditional Gaussian kernel. Then, either the current pixel or the vector median, instead of always the current pixel itself, is chosen as the base to take part in the bilateral filtering action, which is determined by whether the current pixel is a possible impulse or not. The experimental results show that the proposed solution can simultaneously remove impulsive and additive noise while preserving edge structures, and outperforms other vector filtering methods in terms of both objective evaluation and subjectively visual quality.     

形态学滤波和改进PCNN的NSST域多光谱与全色图像融合

目的 全色图像的空间细节信息增强和多光谱图像的光谱信息保持通常是相互矛盾的,如何能够在这对矛盾中实现最佳融合效果一直以来都是遥感图像融合领域的研究热点与难点。为了有效结合光谱信息与空间细节信息,进一步改善多光谱与全色图像的融合质量,提出一种形态学滤波和改进脉冲耦合神经网络（PCNN）的非下采样剪切波变换（NSST）域多光谱与全色图像融合方法。方法 该方法首先分别对多光谱和全色图像进行非下采样剪切波变换;对二者的低频分量采用形态学滤波和高通调制框架（HPM）进行融合,将全色图像低频子带的细节信息注入到多光谱图像低频子带中得到融合后的低频子带;对二者的高频分量则采用改进脉冲耦合神经网络的方法进行融合,进一步增强融合图像中的空间细节信息;最后通过NSST逆变换得到融合图像。结果 仿真实验表明,本文方法得到的融合图像细节信息清晰且光谱保真度高,视觉效果上优势明显,且各项评价指标与其他方法相比整体上较优。相比于5种方法中3组融合结果各指标平均值中的最优值,清晰度和空间频率分别比NSCT-PCNN方法提高0.5%和1.0%,光谱扭曲度比NSST-PCNN方法降低4.2%,相关系数比NSST-PCNN方法提高1.4%,信息熵仅比NSST-PCNN方法低0.08%。相关系数和光谱扭曲度两项指标的评价结果表明本文方法相比于其他5种方法能够更好地保持光谱信息,清晰度和空间频率两项指标的评价结果则展示了本文方法具有优于其他对比方法的空间细节注入能力,信息熵指标虽不是最优值,但与最优值非常接近。结论 分析视觉效果及各项客观评价指标可以看出,本文方法在提高融合图像空间分辨率的同时,很好地保持了光谱信息。综合来看,本文方法在主观与客观方面均具有优于亮度色调饱和度（IHS）法、主成分分析（PCA）法、基于非负矩阵分解（CNMF）、基于非下采样轮廓波变换和脉冲耦合神经网络（NSCT-PCNN）以及基于非下采样剪切波变换和脉冲耦合神经网络（NSST-PCNN）5种经典及现有流行方法的融合效果。     

Fast multi-scale joint bilateral texture upsampling

We present a new approach using a multi-scale joint bilateral filter for upsampling the synthesized texture generated by optimization-based methods. Our method is based on the following motivation: if the available exemplar texture is used as a priority to upsample the synthesized texture, a high resolution result that prevents image blurring can be obtained. Our multi-scale joint bilateral upsampling applies a spatial filter on each multi-scale layer of the synthesized texture, and jointly applies a similar range filter on exemplar texture, which guides the interpolation from low to high resolution, by magnifying and combining the upsampled information; the details of the upsampled texture are progressively enhanced, and the image blurring artifacts can be effectively avoided. We offer an accelerated joint bilateral filter, which enables our upsampling method to interactively generate a large texture. In addition, we propose a detail-aware texture optimization approach that incorporates image detail in texture optimization to improve the quality of the synthesized texture, on which the multi-scale joint bilateral filter works to generate a more convincing result. We show results for upsampling image and video textures and compare them to traditional upsampling methods, by this demonstrating that with low computational and memory costs, our method achieves better results.     

基于卷积神经网络模型的医学图像融合

提出了一种新的基于卷积神经网络（CNN）和加权最小二乘法（WLS）的医学图像融合算法。算法主要步骤如下：利用滚动导向滤波（RGF）和高斯滤波（GF）构成的混合多尺度分解工具将源图像分解为基础层和一系列细节层,从而能够更好地保留尺度信息和边缘信息。基于卷积神经网络给出基础层融合规则,该规则能够更好地提取图像特征,使融合图像能够很好继承源图像结构信息、能量信息和强度信息。利用绝对值取大规则和加权最小二乘法优化策略,对细节层进行融合,使融合图像中包含更多的视觉细节信息和具有更高对比度。实验结果表明所提算法在视觉评价和客观评价方面与其他算法相比具有较好的优势,且在急性中风、致命性中风和脑膜瘤这三类疾病图像融合效果更为突出。     

PAN‐sharpening of very high resolution multispectral images using genetic algorithms

A novel image fusion method is presented, suitable for sharpening of multispectral (MS) images by means of a panchromatic (PAN) observation. The method is based on redundant multiresolution analysis (MRA); the MS bands expanded to the finer scale of the PAN band are sharpened by adding the spatial details from the MRA representation of the PAN data. As a direct, unconditioned injection of PAN details gives unsatisfactory results, a new injection model is proposed that provides the optimum injection by maximizing a global quality index of the fused product. To this aim, a real‐valued genetic algorithm (GA) has been defined and tested on Quickbird data. The optimum GA injection is driven by an index function capable of measuring different types of possible distortions in the fused images. Fusion tests are carried out on spatially degraded data to objectively compare the proposed scheme to the most promising state‐of‐the‐art image fusion methods, and on full‐resolution image data to visually assess the performance of the proposed genetic image fusion method.     

联合双边图像插值

传统的图像插值算法及其改进算法都存在边缘模糊、边缘锯齿或局部扭曲变形等问题。提出联合双边图像插值,首先采用传统插值算法对原始图像进行插值,将其结果作为导引图像对原始图像进行联合双边上采样,并给出其核函数方差的自适应确定算法。实验结果表明,与传统插值算法相比,该算法明显改进插值图像质量,与已有的绝大多数非线性改进算法相比还提高了插值速度。     

A novel adaptive fast IHS transform fusion method driven by regional spectral characteristics for Gaofen-2 imagery

ABSTRACT

With the increasing diversity of applications based on the Gaofen-2 satellite imagery, broadly applicable methods to generate high quality fused images is a significant problem to investigate. To obtain an image with high spatial and spectral resolutions from given panchromatic (Pan) images and multispectral (MS) images, most existing fusion algorithms adopt a unified strategy for the whole image. However, regions have distinct characteristics that impact the spatial and spectral resolution processing, on account of their varying regional features. In this article, to satisfy the diverse needs of different regions, a novel fast IHS (Intensity-Hue-Saturation) transform fusion method driven by regional spectral characteristics is proposed to fuse Gaofen-2 imagery. First, by the fast IHS transform framework, the original intensity component is obtained from the upsampled MS imagery. Then, numerous independent regions of upsampled MS imagery are generated by a novel superpixel merging strategy, and the spectral characteristics of these regions are utilized for generating a fusion factor. Next, to acquire a new fused intensity component, the fusion factor is applied to guide the injection of details in the fusion procedure. This fusion factor adapts the method to meet the spatial and spectral resolution needs for each region. Finally, the difference between the new fused intensity component and the original one is regarded as the detail that needs to be injected; these are added equally to the different bands of the upsampled MS imagery to yield the final fused multispectral image. In comparison with other classical algorithms, the visual and statistical analysis reveal that our proposed method can provide better results in improving spatial detail and preserving spectral information.     

基于小数目标尺度的图像混合滤波算法

为了在有效去除可见光图像噪声的同时最大限度地保持图像的边缘、纹理等细节,将已有的目标尺度改进为小数目标尺度以便更精确地反映局部目标结构的大小,提出了基于小数目标尺度的自适应高斯滤波和基于小数目标尺度的自适应中值滤波的混合滤波算法。前者通过小数目标尺度来自适应地控制高斯核的尺度和滤波的模板大小,后者利用小数目标尺度自适应地筛选出脉冲噪声点并进行中值滤波,并弥补前者在抑制脉冲噪声方面的不足。理论分析和仿真实验结果均表明,所提出的算法不仅可以去除各种类型的点状噪声,而且在图像细节的保护和信噪比方面优于其他几类传统算法。