面向可见光—近红外图像融合的植被与天空概率模板生成

目的 近红外(near-infrared，NIR)图像在夜视和去雾等方面发挥着重要作用，RGB-NIR图像融合是一种常见且有效的处理方式。在实际图像处理过程中，图像的不同对象区域因特性不同需要进行差异化处理，但是现有图像融合算法在植被和天空图像区域存在明显不足。对此，提出RGB-NIR联合图像的植被和天空区域概率模板生成算法。方法 以植被为感兴趣区域，基于RGB图像各通道比值和扩展归一化植被指数(normalized difference vegetation index，NDVI)两种特征，提出RGB-NIR联合图像的植被区域概率模板生成算法。以天空为感兴趣区域，基于透射率图引导的局部熵和扩展NDVI两种特征，结合像素高度信息，提出RGB-NIR联合图像的天空区域概率模板生成算法。两种算法生成的植被和天空的概率模板在RGB-NIR图像融合过程中利用概率模板对权重矩阵进行修正，可明显改善融合效果。结果 检测植被的模板生成算法与传统NDVI进行比较，在对比度和鲁棒性方面有更大优势；与语义分割进行比较，在准确度和纹理细节上有更好表现。检测天空的模板生成算法与当前的概率模板天空检测算法相比，准确率更高，边缘过渡更平滑；与当前的二值模板天空检测算法相比，在检测效果相当的情况下能保留更多细节信息，并且对小物体的划分更为准确。以本文检测算法修正后的图像融合结果在保持细节增强效果的同时，视觉感观更为自然，在定量指标上也更占优势。结论 本文提出的概率模板生成算法结果准确、性能鲁棒，能有效提升RGB-NIR图像融合的效果，特别是在涉及权重的图像融合中能更好地结合与应用。

Vegetation and sky probability mask generation for visible and near-infrared image fusion

Objective The imaging mechanism of near-infrared (NIR) images is different from that of visible images. It can receive the infrared radiation emitted by the object and convert it into grayscale values. Therefore, stronger infrared radiation in the scene yields higher grayscale value in the NIR image and its adaptability to harsh environments (e.g., fog, haze) is better than the visible light imaging. To take advantage of NIR images, RGB-NIR image fusion is a common and effective processing method, which has been widely used in various image vision applications, including recognition, detection and surveillance. Multiple objects will have different imaging results in the same image in terms of their reflection and infrared radiation features, and the same object will have different appearances in visible and NIR images as well. For example, the vegetation part appears as low gray-scale values in the RGB image, but high gray-scale values in the NIR image. In addition, current image fusion algorithms have been challenging in specific regions like vegetation and sky. Therefore, an accurate and robust region detection method is necessary for regional-based processing. However, most algorithms are concerned of single image only and cannot meet the requirements for RGB-NIR image-fused region detection. Method We develop a probability-mask generation method from vegetation and sky regions based on RGB-NIR image pairs. The vegetation region: 1) to preserve high contrast and smooth transition, we obtain the ratio of multiple channels of RGB images with the extended normalized difference vegetation index (NDVI). 2) To avoid the extreme case that the red channel is of value minimum or maximum, we use the relationship between NIR and luminance instead of red channel. 3) To get the detection result, we integrate the ratio-guided and the NDVI-extended into the probability mask of vegetation. The sky-region: 1) the local-entropy feature of the RGB image is calculated and a transmission map is for guidance. 2) The guided feature and the extended NDVI is combined, and the results with the height of pixels is enhanced, according to the prior that the sky basically has a great probability of appearing in the upper part of the natural-scene image. 3) The result is a probability mask and considered as the sky detection result. The vegetation and sky detection based algorithms produce corresponding probability masks. We can incorporate them into RGB-NIR image fusion algorithms to improve image quality. The original algorithm uses the Laplacian-Gaussian pyramid and the weight map for multi-scale fusion. We modify the weight map of the NIR image by multiplying it with the vegetation and sky probability masks, and then replace the original NIR weight map with the modified one. The rest of the fusion algorithm remains unchanged. Result Our algorithm is evaluated on a public dataset containing outdoors images, including country, field, forest, indoor, mountain, old-building, street, urban, and water. To express the health of vegetation in the field of remote sensing, we compare the proposed vegetation detection algorithm with the traditional NDVI. The experimental results of image fusion indicate that our image fusion algorithm can perform better by incorporating the region masks both quantitatively and qualitatively, and produce more realistic and natural images perceptually. Moreover, we analyze the difference between the probability mask and the binary mask when applied to image fusion in the same way. The results show that selected probability mask makes the fused images more colorful and rich in details. Conclusion Our probability mask generation algorithm of vegetation and sky is potential to high accuracy and robustness. Specifically, the detected areas in result images are accurate with clear details and smooth transition, and small objects can be segmented properly. Moreover, our algorithm is beneficial to improving the performance of RGB-NIR image fusion, especially on weight map, making the results have enhanced details and natural colors. It is easy-used without complicated calculations. It is worthy note that our algorithm is more suitable for natural scenes generally.