Image indexing and retrieval based on color histograms

While general object recognition is difficult, it is relatively easy to capture various primitive properties such as color distributions, prominent regions and their topological features from an image and use them to narrow down the search space when attempts to retrieving images by contents from an image database are made.In this paper, we present an image database in which images are indexed and retrieved based on color histograms. We first address the problems inherent in color histograms created by the conventional method, and then propose a new method to create histograms which are compact in size and insensitive to minor illumination variations such as highlight, shape, and etc. A powerful indexing scheme where each histogram of an image is encoded into a numerical key, and stored in a two-layered tree structure is introduced. This approach turns the problem of histogram matching, which is computation intensive, into index key search, so as to realize quick data access in a large scale image database. Two types of user interfaces, Query by user provided sample images, and Query by combination of the system provided templates, are provided to meet various user requests. Various experimental evaluations exhibit the effectiveness of the image database system.     

A probabilistic matching algorithm for computer vision

A model-based vision system attempts to find correspondences between features of an object model and features detected in an image for purposes of recognition, localization, or inspection. In this paper we pose the relational matching problem as a special case of the pattern complex recognition problem and propose a probabilistic model to describe the images of an object. This Bayesian approach allows us to make explicit statements of how an image is formed from a model, and hence define a natural matching cost that can be used to guide a heuristic search in finding the best observation mapping. Furthermore, we show that even though the nature of the feature matching problem is exponential, the use of the proposed algorithm keeps the size of the problem under control, by efficiently reducing the search space.     

Matching of dental X-ray images for human identification

Forensic dentistry involves the identification of people based on their dental records, mainly available as radiograph images. Our goal is to automate this process using image processing and pattern recognition techniques. Given a postmortem radiograph, we search a database of antemortem radiographs in order to retrieve the closest match with respect to some salient features. In this paper, we use the contours of the teeth as the feature for matching. A semi-automatic contour extraction method is used to address the problem of fuzzy tooth contours caused by the poor image quality. The proposed method involves three stages: radiograph segmentation, pixel classification and contour matching. A probabilistic model is used to describe the distribution of object pixels in the image. Results of retrievals on a database of over 100 images are encouraging.     

Dental biometrics: Human identification based on teeth and dental works in bitewing radiographs

This paper presents an enhanced dental identification method based on both the contours of teeth and dental works. To reduce the alignment error caused from unreliable contours, we propose a point-reliability measuring method and weigh each point based on its reliability when calculating the Hausdorff distance (HD) between the contours. For reducing the alignment error caused from incomplete tooth contours, we propose an outlier detection method to prune the outliers from each contour and realign the pruned contours. And for compensating the error when matching with the spatial feature of dental works due to imperfect alignment of the teeth in which they reside, we propose using an additional alignment-invariant frequency feature of dental works. Experimental results show that our method can achieve (1) 94.3% and 100% image retrieval accuracy of the top-1 and -5 retrievals, respectively, when matching with the weighted HD for the pruned contour of a single tooth; (2) 100% accuracy of top-2 (top 6%) image retrievals when matching with both contours of teeth and dental works.     

Latent fingerprint matching

Latent fingerprint identification is of critical importance to law enforcement agencies in identifying suspects: Latent fingerprints are inadvertent impressions left by fingers on surfaces of objects. While tremendous progress has been made in plain and rolled fingerprint matching, latent fingerprint matching continues to be a difficult problem. Poor quality of ridge impressions, small finger area, and large nonlinear distortion are the main difficulties in latent fingerprint matching compared to plain or rolled fingerprint matching. We propose a system for matching latent fingerprints found at crime scenes to rolled fingerprints enrolled in law enforcement databases. In addition to minutiae, we also use extended features, including singularity, ridge quality map, ridge flow map, ridge wavelength map, and skeleton. We tested our system by matching 258 latents in the NIST SD27 database against a background database of 29,257 rolled fingerprints obtained by combining the NIST SD4, SD14, and SD27 databases. The minutiae-based baseline rank-1 identification rate of 34.9 percent was improved to 74 percent when extended features were used. In order to evaluate the relative importance of each extended feature, these features were incrementally used in the order of their cost in marking by latent experts. The experimental results indicate that singularity, ridge quality map, and ridge flow map are the most effective features in improving the matching accuracy.     

Signature Detection and Matching for Document Image Retrieval

As one of the most pervasive methods of individual identification and document authentication, signatures present convincing evidence and provide an important form of indexing for effective document image processing and retrieval in a broad range of applications. However, detection and segmentation of free-form objects such as signatures from clustered background is currently an open document analysis problem. In this paper, we focus on two fundamental problems in signature-based document image retrieval. First, we propose a novel multiscale approach to jointly detecting and segmenting signatures from document images. Rather than focusing on local features that typically have large variations, our approach captures the structural saliency using a signature production model and computes the dynamic curvature of 2D contour fragments over multiple scales. This detection framework is general and computationally tractable. Second, we treat the problem of signature retrieval in the unconstrained setting of translation, scale, and rotation invariant nonrigid shape matching. We propose two novel measures of shape dissimilarity based on anisotropic scaling and registration residual error and present a supervised learning framework for combining complementary shape information from different dissimilarity metrics using LDA. We quantitatively study state-of-the-art shape representations, shape matching algorithms, measures of dissimilarity, and the use of multiple instances as query in document image retrieval. We further demonstrate our matching techniques in offline signature verification. Extensive experiments using large real-world collections of English and Arabic machine-printed and handwritten documents demonstrate the excellent performance of our approaches.     

ViT2CMH: Vision Transformer Cross-Modal Hashing for Fine-Grained Vision-Text Retrieval

In recent years, the development of deep learning has further improved hash retrieval technology. Most of the existing hashing methods currently use Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to process image and text information, respectively. This makes images or texts subject to local constraints, and inherent label matching cannot capture fine-grained information, often leading to suboptimal results. Driven by the development of the transformer model, we propose a framework called ViT2CMH mainly based on the Vision Transformer to handle deep Cross-modal Hashing tasks rather than CNNs or RNNs. Specifically, we use a BERT network to extract text features and use the vision transformer as the image network of the model. Finally, the features are transformed into hash codes for efficient and fast retrieval. We conduct extensive experiments on Microsoft COCO (MS-COCO) and Flickr30K, comparing with baselines of some hashing methods and image-text matching methods, showing that our method has better performance.     

Phase congruency induced local features for finger-knuckle-print recognition

Researchers have recently found that the finger-knuckle-print (FKP), which refers to the inherent skin patterns of the outer surface around the phalangeal joint of one’s finger, has high discriminability, making it an emerging promising biometric identifier. Effective feature extraction and matching plays a key role in such an FKP based personal authentication system. This paper studies image local features induced by the phase congruency model, which is supported by strong psychophysical and neurophysiological evidences, for FKP recognition. In the computation of phase congruency, the local orientation and the local phase can also be defined and extracted from a local image patch. These three local features are independent of each other and reflect different aspects of the image local information. We compute efficiently the three local features under the computation framework of phase congruency using a set of quadrature pair filters. We then propose to integrate these three local features by score-level fusion to improve the FKP recognition accuracy. Such kinds of local features can also be naturally combined with Fourier transform coefficients, which are global features. Experiments are performed on the PolyU FKP database to validate the proposed FKP recognition scheme.     

A Line Feature Matching Technique Based on an Eigenvector Approach

In this paper, we propose a new eigenvector-based line feature matching algorithm, which is invariant to the in-plane rotation, translation, and scale. First, in order to reduce the number of possible matches, we use a preliminary correspondence test that generates a set of finite candidate models, by restricting combinations of line features in the input image. This approach resolves an inherent problem relating to ordering and correspondence in an eigenvector/modal approach. Second, we employ the modal analysis, in which the Gaussian weighted proximity matrices for reference and candidate models are constructed to record the relative distance and angle information between line features for each model. Then, the modes of the proximity matrices of the two models are compared to yield the dissimilarity measure, which describes the quantitative degree of the difference between the two models. Experimental results for synthetic and real images show that the proposed algorithm performs matching of the line features with affine variation fast and efficiently and provides the degree of dissimilarity in a quantitative way.     

A system for human identification from X-ray dental radiographs

Forensic odontology is the branch of forensics that deals with human identification based on dental features. In this paper, we present a system for automating that process by identifying people from dental X-ray images. Given a dental image of a postmortem (PM), the proposed system retrieves the best matches from an antemortem (AM) database. The system automatically segments dental X-ray images into individual teeth and extracts the contour of each tooth. Features are extracted from each tooth and are used for retrieval. We developed a new method for teeth separation based on integral projection. We also developed a new method for representing and matching teeth contours using signature vectors obtained at salient points on the contours of the teeth. During retrieval, the AM radiographs that have signatures closer to the PM are found and presented to the user. Matching scores are generated based on the distance between the signature vectors of AM and PM teeth. Experimental results on a small database of dental radiographs are encouraging.     

基于层次匹配下多种特征融合的蕾丝花边检索方法

针对基于图像纹理特征的蕾丝花边检索方法效率低下问题,为提高蕾丝花边检索效率,提出一种基于层次匹配下多种特征融合的蕾丝花边检索方法。通过运用图像纹理特征标识图像,利用Canny算子处理纹理图像,得到彩色Canny图像及其灰度梯度共生矩阵GGCM,采用能量、梯度平均、灰度平均、相关等二次统计特征参数描述图像的纹理特征,将上述提取纹理特征结合形状特征和SURF特征进行逐层匹配,实现层次匹配下多种特征的融合,弥补单个匹配方法的不足,同时在蕾丝花边库中验证所提检索方法的正确率。实验结果表明,与其他匹配方法相比,该方法提取的纹理特征具有更强的纹理鉴别能力,能较好地实现蕾丝花边检索,有效地提高了检索方法的速率和准确率。     

Image Retrieval With Relevance Feedback Based on Graph-Theoretic Region Correspondence Estimation

This paper presents a graph-theoretic approach for interactive region-based image retrieval. When dealing with image matching problems, we use graphs to represent images, transform the region correspondence estimation problem into an inexact graph matching problem, and propose an optimization technique to derive the solution. We then define the image distance in terms of the estimated region correspondence. In the relevance feedback steps, with the estimated region correspondence, we propose to use a maximum likelihood method to re-estimate the ideal query and the image distance measurement. Experimental results show that the proposed graph-theoretic image matching criterion outperforms the other methods incorporating no spatially adjacent relationship within images. Furthermore, our maximum likelihood method combined with the estimated region correspondence improves the retrieval performance in feedback steps.     

Fast exact feature based data correspondence search with an efficient bit-parallel MCP solver

The problem of finding the optimal correspondence between two sets of geometric entities or features is known to be NP-hard in the worst case. This problem appears in many real scenarios such as fingerprint comparisons, image matching and global localization of mobile robots. The inherent complexity of the problem can be avoided by suboptimal solutions, but these could fail with high noise or corrupted data. The correspondence problem has an interesting equivalent formulation in finding a maximum clique in an association graph. We have developed a novel algorithm to solve the correspondence problem between two sets of features based on an efficient solution to the Maximum Clique Problem using bit parallelism. It outperforms an equivalent non bit parallel algorithm in a number of experiments with simulated and real data from two different correspondence problems. This article validates for the first time, to the best of our knowledge, that bit parallel optimization techniques can greatly reduce computational cost, thus making feasible the use of an exact solution in real correspondence search problems despite their inherent NP computational complexity.     

基于mean-shift全局立体匹配方法

针对图像全局立体匹配精度高、计算量大的问题,提出基于mean shift图像分割的全局立体匹配方法。首先,通过mean shift算法对图像进行分割,获取图像同质区域数量和区域的标号。在计算匹配代价时,根据像素所属的分割区域,对像素进行筛选,从而提高匹配代价计算速度;其次,在代价聚合前,将mean shift算法获取的同质区域数K值赋值给K-means聚类算法,对像素再次聚类,提高立体匹配精度和速度;最后通过TRW-S置信传播解决能量最小化问题。实验表明,该算法明显提高了匹配的准确性和速度,与单纯的全局匹配算法相比,具有更大的优势。     

Incremental User Identification Across Social Networks Based on User-Guider Similarity Index

Identifying accounts across different online social networks that belong to the same user has attracted extensive attentions. However, existing techniques rely on given user seeds and ignore the dynamic changes of online social networks, which fails to generate high quality identification results. In order to solve this problem, we propose an incremental user identification method based on user-guider similarity index (called CURIOUS), which efficiently identifies users and well captures the changes of user features over time. Specifically, we first construct a novel user-guider similarity index (called USI) to speed up the matching between users. Second we propose a two-phase user identification strategy consisting of USI-based bidirectional user matching and seed-based user matching, which is effective even for incomplete networks. Finally, we propose incremental maintenance for both USI and the identification results, which dynamically captures the instant states of social networks. We conduct experimental studies based on three real-world social networks. The experiments demonstrate the effectiveness and the efficiency of our proposed method in comparison with traditional methods. Compared with the traditional methods, our method improves precision, recall and rank score by an average of 0.19, 0.16 and 0.09 respectively, and reduces the time cost by an average of 81%.     

一种去除运动目标重影的图像镶嵌方法研究

针对图像镶嵌过程中如果图像序列中存在运动目标就会引起重影的问题,文中提出了一种新的去除运动目标重影的图像镶嵌方法。算法首先对视频图像进行了运动分割,在图像匹配阶段采用边缘特征点进行匹配,并由马尔可夫随机场模型生成运动目标的二值模板,剔除掉运动目标二值模板上的边缘点,从而保证图像匹配的准确率。在镶嵌阶段使用活动轮廓模型生成一条最优镶嵌线,产生的镶嵌线充分考虑了图像的边缘特征和梯度信息,确保了镶嵌后图像两边纹理差异更小。从连续视频帧中选取多帧图像进行实际的图像镶嵌,实验结果表明文中算法取得了较好的效果。     

Automatic Construction of Dental Charts for Postmortem Identification

Identification of deceased individuals based on dental characteristics is receiving increased attention, especially with the large volume of victims encountered in mass disasters. An important problem in automated dental identification is automatic classification of teeth into four classes (molars, premolars, canines, and incisors). An equally important problem is the construction of a dental chart, which is a data structure that guides tooth-to-tooth matching. Dental charts are the key for avoiding illogical comparisons that inefficiently consume the limited computational resources and may mislead decision making. Labeling of the teeth is a challenging task which has received inadequate attention in the literature. We tackle this composite problem using a two-stage approach. The first stage utilizes low computational cost, appearance-based features for assigning an initial class. The second stage applies a string matching technique, based on teeth neighborhood rules, to validate initial teeth-classes and, hence, to assign each tooth a number corresponding to its location in the dental chart. Based on a large test dataset of 507 bitewing and periapical films that contain 2027 teeth, the proposed approach achieves classification accuracy of 87%. Experimental results indicate that the proposed approach works very fast, and achieves high performance compared to other approaches suggested in the literature.     

3-D shape reconstruction in an active stereo vision system using genetic algorithms

The recovery of 3-D shape information (depth) using stereo vision analysis is one of the major areas in computer vision and has given rise to a great deal of literature in the recent past. The widely known stereo vision methods are the passive stereo vision approaches that use two cameras. Obtaining 3-D information involves the identification of the corresponding 2-D points between left and right images. Most existing methods tackle this matching task from singular points, i.e. finding points in both image planes with more or less the same neighborhood characteristics. One key problem we have to solve is that we are on the first instance unable to know a priori whether a point in the first image has a correspondence or not due to surface occlusion or simply because it has been projected out of the scope of the second camera. This makes the matching process very difficult and imposes a need of an a posteriori stage to remove false matching.In this paper we are concerned with the active stereo vision systems which offer an alternative to the passive stereo vision systems. In our system, a light projector that illuminates objects to be analyzed by a pyramid-shaped laser beam replaces one of the two cameras. The projections of laser rays on the objects are detected as spots in the image. In this particular case, only one image needs to be treated, and the stereo matching problem boils down to associating the laser rays and their corresponding real spots in the 2-D image. We have expressed this problem as a minimization of a global function that we propose to perform using Genetic Algorithms (GAs). We have implemented two different algorithms: in the first, GAs are performed after a deterministic search. In the second, data is partitioned into clusters and GAs are independently applied in each cluster. In our second contribution in this paper, we have described an efficient system calibration method. Experimental results are presented to illustrate the feasibility of our approach. The proposed method yields high accuracy 3-D reconstruction even for complex objects. We conclude that GAs can effectively be applied to this matching problem.