A wavelet-based image fusion tutorial

The objective of image fusion is to combine information from multiple images of the same scene. The result of image fusion is a new image which is more suitable for human and machine perception or further image-processing tasks such as segmentation, feature extraction and object recognition. Different fusion methods have been proposed in literature, including multiresolution analysis. This paper is an image fusion tutorial based on wavelet decomposition, i.e. a multiresolution image fusion approach. We can fuse images with the same or different resolution level, i.e. range sensing, visual CCD, infrared, thermal or medical. The tutorial performs a synthesis between the multiscale-decomposition-based image approach (Proc. IEEE 87 (8) (1999) 1315), the ARSIS concept (Photogramm. Eng. Remote Sensing 66 (1) (2000) 49) and a multisensor scheme (Graphical Models Image Process. 57 (3) (1995) 235). Some image fusion examples illustrate the proposed fusion approach. A comparative analysis is carried out against classical existing strategies, including those of multiresolution.     

Dynamic clustering using particle swarm optimization with application in image segmentation

A new dynamic clustering approach (DCPSO), based on particle swarm optimization, is proposed. This approach is applied to image segmentation. The proposed approach automatically determines the “optimum” number of clusters and simultaneously clusters the data set with minimal user interference. The algorithm starts by partitioning the data set into a relatively large number of clusters to reduce the effects of initial conditions. Using binary particle swarm optimization the “best” number of clusters is selected. The centers of the chosen clusters is then refined via the K-means clustering algorithm. The proposed approach was applied on both synthetic and natural images. The experiments conducted show that the proposed approach generally found the “optimum” number of clusters on the tested images. A genetic algorithm and random search version of dynamic clustering is presented and compared to the particle swarm version.     

MFF-GAN: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion

Multi-focus image fusion is an enhancement method to generate full-clear images, which can address the depth-of-field limitation in imaging of optical lenses. Most existing methods generate the decision map to realize multi-focus image fusion, which usually lead to detail loss due to misclassification, especially near the boundary line of the focused and defocused regions. To overcome this challenge, this paper presents a new generative adversarial network with adaptive and gradient joint constraints to fuse multi-focus images. In our model, an adaptive decision block is introduced to determine whether source pixels are focused or not based on the difference of repeated blur. Under its guidance, a specifically designed content loss can dynamically guide the optimization trend, that is, force the generator to produce a fused result of the same distribution as the focused source images. To further enhance the texture details, we establish an adversarial game so that the gradient map of the fused result approximates the joint gradient map constructed based on the source images. Our model is unsupervised without requiring ground-truth fused images for training. In addition, we release a new dataset containing 120 high-quality multi-focus image pairs for benchmark evaluation. Experimental results demonstrate the superiority of our method over the state-of-the-art in terms of both subjective visual effect and quantitative metrics. Moreover, our method is about one order of magnitude faster compared with the state-of-the-art.     

基于彩色图像融合新算法的隐藏武器检测研究

针对隐藏在衣服下的武器融合检测问题,提出了一种彩色可见光图像和红外图像融合的新算法。该算法在基于RGB、HSV和LAB彩色空间变换的基础上,使用了双树复数小波变换技术,该变换明显具有平移不变性、方向选择性、有限数据冗余性、完美重构性和较高计算效率等特点,在融合方面优于其他的小波变换。融合后的图像保持了可见光图像的高分辨率;保留了红外图像中检测出的隐藏武器;维持了可见光图像的自然色彩。该融合技术的可行性在实验中得到了验证。     

Multi-source unsupervised domain adaptation for object detection

Domain adaptation for object detection has been extensively studied in recent years. Most existing approaches focus on single-source unsupervised domain adaptive object detection. However, a more practical scenario is that the labeled source data is collected from multiple domains with different feature distributions. The conventional approaches do not work very well since multiple domain gaps exist. We propose a Multi-source domain Knowledge Transfer (MKT) method to handle this situation. First, the low-level features from multiple domains are aligned by learning a shallow feature extraction network. Then, the high-level features from each pair of source and target domains are aligned by the followed multi-branch network. After that, we perform two parts of information fusion: (1) We train a detection network shared by all branches based on the transferability of each source sample feature. The transferability of a source sample feature means the indistinguishable degree to the target domain sample features. (2) For using our model, the target sample features output by the multi-branch network are fused based on the average transferability of each domain. Moreover, we leverage both image-level and instance-level attention to promote positive cross-domain transfer and suppress negative transfer. Our main contributions are the two-stage feature alignments and information fusion. Extensive experimental results on various transfer scenarios show that our method achieves the state-of-the-art performance.     

一种改进的Harris角点图像拼接算法

主要研究图像拼接的优化问题,针对传统的Harris角点检测算法进行了改进。避免了角点响应函数中的k 值和检测过程中的阂值T的选取对角点检测带来的影响。在图像拼接过程中通过NCC对图像进行配准,采用加权 平均法对拼接后的图像进行融合并消除拼接缝隙,从而提高了运算的速率。为验证其有效性,该算法被应用于一个图 像拼接的实例中,仿真结果表明,该算法缩短了图像拼接所消耗的时间,提高了图像拼接的精度和效率,能够达到无缝 拼接的效果,具有良好的实用性。     

Fusing multiple image transformations and a thermal sensor with kinect to improve person detection ability

This paper proposes a novel approach to combine data from multiple low-cost sensors to detect people in a mobile robot. Robust detection of people is a key capability required for robots working in environments with people. Several works have shown the benefits of fusing data from complementary sensors. The Kinect sensor provides a rich data set at a significantly low cost, however, it has some limitations for its use on a mobile platform, mainly that people detection algorithms rely on images captured by a static camera. To cope with these limitations, this work is based on the fusion of Kinect and a thermical sensor (thermopile) mounted on top of a mobile platform. We propose the implementation of an evolutionary selection of sequences of image transformation to detect people through supervised classifiers. Experimental results carried out with a mobile platform in a manufacturing shop floor show that the percentage of wrong classified using only Kinect is drastically reduced with the classification algorithms and with the combination of the three information sources. Extra experiments are presented as well to show the benefits of the image transformation sequence idea here presented.     

Object's contour detection scheme based on image fusion

Three novel object's contour detection schemes based on image fusion are proposed in this paper. In these schemes an active contour model is applied to detect the object's contour edge. Since an object's contour in an infrared (IR) image is usually clearer than that in a visible image, the convergent active contour in a visible image is improved with that in an IR image. The first contour detection scheme is realized by revising the shape-preserving active contour model. The second scheme minimizes the B-spline L ₂ norm's square of the difference of the B-spline control point vectors in two modal images. Contour tracking and extraction experiments indicate that the first scheme outperforms the second one. Moreover, a third scheme based on the active contour and pixel-level image fusion is proposed for images with incomplete but complementary scene information. An example using contour extraction of a partially hidden tank proves its efficacy.     

Detection of human faces in color image sequences with arbitrary motions for very low bit-rate videophone coding

The problem of human face detection is a focus of interest in image analysis, image databases and video coding. A new multi-resolution method using color and motion information and shape model is developed to detect human faces in videophone QCIF sequences for efficient encoding. The method is based on color segmentation and multiresolution propagation of a geometrical model. A new measure of motion activity is proposed to validate the choice of candidates.     

结合LOG算子和小波变换的图像边缘检测方法

针对目前常用的边缘检测方法对噪声比较敏感,测得的边缘比较粗糙等现象,在传统边缘检测方法的基础上提出了一种基于数据融合的边缘检测方法。该方法分别采用改进的LOG算子和离散小波变换对原始图像进行边缘检测,建立相应的融合原则,并将两种方法检测出来的边缘进行数据融合,得到最终的图像边缘。实验证明,该方法结合了两种检测方法的优点,有效地提高了边缘检测准确性,获得了比较理想的边缘检测效果。     

Efficient greedy estimation of mixture models through a binary tree search

Unsupervised data clustering can be addressed by the estimation of mixture models, where the mixture components are associated to clusters in data space. In this paper we present a novel unsupervised classification algorithm based on the simultaneous estimation of the mixture’s parameters and the number of components (complexity). Its distinguishing aspect is the way the data space is searched. Our algorithm starts from a single component covering all the input space and iteratively splits components according to breadth first search on a binary tree structure that provides an efficient exploration of the possible solutions. The proposed scheme demonstrates important computational savings with respect to other state-of-the-art algorithms, making it particularly suited to scenarios where the performance time is an issue, such as in computer and robot vision applications. The initialization procedure is unique, allowing a deterministic evolution of the algorithm, while the parameter estimation is performed with a modification of the Expectation Maximization algorithm. To compare models with different complexity we use the Minimum Message Length information criteria that implement the trade-off between the number of components and data fit log-likelihood. We validate our new approach with experiments on synthetic data, and we test and compare to related approaches its computational efficiency in data-intensive image segmentation applications.     

Benchmarking and comparing multi-exposure image fusion algorithms

Multi-exposure image fusion (MEF) is an important area in computer vision and has attracted increasing interests in recent years. Apart from conventional algorithms, deep learning techniques have also been applied to MEF. However, although many efforts have been made on developing MEF algorithms, the lack of benchmarking studies makes it difficult to perform fair and comprehensive performance comparison among MEF algorithms, thus hindering the development of this field significantly. In this paper, we fill this gap by proposing a benchmark of multi-exposure image fusion (MEFB), which consists of a test set of 100 image pairs, a code library of 21 algorithms, 20 evaluation metrics, 2100 fused images, and a software toolkit. To the best of our knowledge, this is the first benchmarking study in the field of MEF. This paper also gives a literature review on MEF methods with a focus on deep learning-based algorithms. Extensive experiments have been conducted using MEFB for comprehensive performance evaluation and for identifying effective algorithms. We expect that MEFB will serve as an effective platform for researchers to compare the performance of MEF algorithms.     

基于图像的动目标检测技术

本文研究了光流估计的基本算法，从运动分割的角度的提出了使用光流分割方案来解决复杂背景下的动目标检测问题。文中分析了标准光流估计的缺陷及其原因，并使用鲁棒性估计技术提高了光流估计算法的实用性。最后给出了动目标检测算法的仿真结果，表明利用图像的运动特征来实现目标检测是一种很有效的方法。     

Browsing museum image collections on a multi-touch table

In this paper we present a multi-touch tabletop system for browsing image databases, conceived for museums and art gallery exhibitions. The system exploits an innovative image browsing paradigm and image retrieval functionalities to perform natural and intuitive user interaction: users can explore the image database by handling digital pictures by means of touch gestures or by the use of a predetermined set of physical objects; once one of these objects is placed on the table, it is automatically recognized and the associated function is triggered. The set of objects and the function associations can be dynamically configured. An innovative feature of our application is that users can interactively create and manipulate image clusters where images are grouped according to their pictorial similarity. This is achieved by placing one or more specific tangible objects on the table surface. The system has been evaluated on a collection of photos organized in groups according to the UNESCO picture categories. The usability tests, performed considering different user categories, show that users consider the application to be attractive and interesting.     

A hierarchical digital watermarking method for image tamper detection and recovery

In this paper, we present an efficient and effective digital watermarking method for image tamper detection and recovery. Our method is efficient as it only uses simple operations such as parity check and comparison between average intensities. It is effective because the detection is based on a hierarchical structure so that the accuracy of tamper localization can be ensured. That is, if a tampered block is not detected in level-1 inspection, it will be detected in level-2 or level-3 inspection with a probability of nearly 1. Our method is also very storage effective, as it only requires a secret key and a public chaotic mixing algorithm to recover a tampered image. The experimental results demonstrate that the precision of tamper detection and localization is 99.6% and 100% after level-2 and level-3 inspection, respectively. The tamper recovery rate is better than 93% for a less than half tampered image. As compared with the method in Celik et al. [IEEE Trans. Image Process. 11(6) (2002) 585], our method is not only as simple and as effective in tamper detection and localization, it also provides with the capability of tamper recovery by trading off the quality of the watermarked images about 5 dB.     

焦点区域的检测实现多聚焦图像融合

针对传统像素级图像融合方法割裂像素间联系的问题,提出一种基于区域检测的图像融合方法。首先,通过像素点在其邻域内的方差来区分是清晰点还是模糊点,其次,以当前像素为中心的浮动窗口统计相应窗口区域的清晰点数,消除清晰点数少的中心像素,以同样的方式统计局部窗口区域的清晰点数,清晰点数多的中心像素点设置成清晰点,由此得到清晰区域,最后,通过检测出的区域结果进行图像融合。实验结果表明提出的方法是有效的。     

一种基于多特征的视频人物聚类方法

检索一段视频中出现的人物并进行人物归类具有重要的研究意义和实用价值.本文提出一种基于多特征的视频人物检索聚类算法:先用一种结合人脸检测和物体跟踪的算法检测镜头人物,并提取人物衣服区域颜色以及声音作为人物特征,再用一种无监督模糊聚类方法对人物进行聚类,最后利用声音特征对聚类结果进行修正.该方法适用于人物数未知的条件下进行无监督的人物聚类.不同类型视频的试验证明该方法有效而实用.     

基于小波变换一致性检测的遥感影像融合

通过分析一些经典遥感影像融合方法的不足, 提出了一种基于小波变换一致性检测的遥感影像融合方法。其基本思想是在基于局部方差融合方法的基础上,采用“多数”原则对各个像素的8个邻域进行一致性检测,并进行相应调整,以保持融合后的高频细节分量具有一致性。通过仿真实验以及相应评价指标的衡量证明了该方法在遥感影像融合方面的有效性和优越性。     

基于小波的小尺寸物体的图像边缘提取方法

基于小波变换的“多分辨”特性，采用一种利用小波模极大值，找到图像中的边界点，从而实现对灰度图像的边界检测的方法，通过分析图像中是否有灰度突变特征，来判断边界的存在。在进行边界检测之前。首先要滤除原图像的寄生噪声的干扰。实验结果表明：基于小波变换的边界检测方法可对图像边界进行敏锐提取，从而得到小尺寸物体的图像的细微特征。