PCA-based image recombination for multimodal 2D + 3D face recognition

Most of the existing approaches of multimodal 2D + 3D face recognition exploit the 2D and 3D information at the feature or score level. They do not fully benefit from the dependency between modalities. Exploiting this dependency at the early stage is more effective than the later stage. Early fusion data contains richer information about the input biometric than the compressed features or matching scores. We propose an image recombination for face recognition that explores the dependency between modalities at the image level. Facial cues from the 2D and 3D images are recombined into a more independent and discriminating data by finding transformation axes that account for the maximal amount of variances in the images. We also introduce a complete framework of multimodal 2D + 3D face recognition that utilizes the 2D and 3D facial information at the enrollment, image and score levels. Experimental results based on NTU-CSP and Bosphorus 3D face databases show that our face recognition system using image recombination outperforms other face recognition systems based on the pixel- or score-level fusion.     

Multimodal Biometric Score Fusion Using Gaussian Mixture Model and Monte Carlo Method

Multimodal biometric fusion is gaining more attention among researchers in recent days. As multimodal biometric system consolidates the information from multiple biometric sources, the effective fusion of information obtained at score level is a challenging task. In this paper, we propose a framework for optimal fusion of match scores based on Gaussian Mixture Model (GMM) and Monte Carlo sampling based hypothesis testing. The proposed fusion approach has the ability to handle: 1) small size of match scores as is more commonly encountered in biometric fusion, and 2) arbitrary distribution of match scores which is more pronounced when discrete scores and multimodal features are present. The proposed fusion scheme is compared with well established schemes such as Likelihood Ratio (LR) method and weighted SUM rule. Extensive experiments carried out on five different multimodal biometric databases indicate that the proposed fusion scheme achieves higher performance as compared with other contemporary state of art fusion techniques.     

Confidence factor weighted Gaussian function induced parallel fuzzy rank-level fusion for inference and its application to face recognition

This paper proposes a novel approach for inference using fuzzy rank-level fusion and explores it application to face recognition using multiple biometric representations. Multiple representations of single biometric (trait) aim to increase the reliability or acceptance of a biometric system, as it exploits the underlying essential characteristics provided by different sensors. In this paper, we propose a new scheme for generating fuzzy ranks induced by a Gaussian function based on the confidence of a classifier. In contrast to the conventional ranking, this fuzzy ranking reflects some associations among the outputs (confidence factors) of a classifier. These fuzzy ranks, yielded by multiple representations of a face image, are fused weighted by the corresponding confidence factors of the classifier to generate the final ranks while recognizing a face. In many real-world applications, where multiple traits of a person are unavailable, the proposed method is highly effective. However, it can easily be extended to multimodal biometric systems utilizing multiple classifiers. The experimental results using different feature vectors of a face image employing different classifiers show that the proposed method can significantly improve recognition accuracy as compared to those from individual feature vectors and as well as some commonly used rank-level fusion methods.     

Score level fusion of multimodal biometrics using triangular norms

A multimodal biometric system that alleviates the limitations of the unimodal biometric systems by fusing the information from the respective biometric sources is developed. A general approach is proposed for the fusion at score level by combining the scores from multiple biometrics using triangular norms (t-norms) due to Hamacher, Yager, Frank, Schweizer and Sklar, and Einstein product. This study aims at tapping the potential of t-norms for multimodal biometrics. The proposed approach renders very good performance as it is quite computationally fast and outperforms the score level fusion using the combination approach (min, mean, and sum) and classification approaches like SVM, logistic linear regression, MLP, etc. The experimental evaluation on three databases confirms the effectiveness of score level fusion using t-norms.     

Combining classifiers: A theoretical framework

The problem of classifier combination is considered in the context of the two main fusion scenarios: fusion of opinions based on identical and on distinct representations. We develop a theoretical framework for classifier combination for these two scenarios. For multiple experts using distinct representations we argue that many existing schemes such as the product rule, sum rule, min rule, max rule, majority voting, and weighted combination, can be considered as special cases of compound classification. We then consider the effect of classifier combination in the case of multiple experts using a shared representation where the aim of fusion is to obtain a better estimate of the appropriatea posteriori class probabilities. We also show that the two theoretical frameworks can be used for devising fusion strategies when the individual experts use features some of which are shared and the remaining ones distinct. We show that in both cases (distinct and shared representations), the expert fusion involves the computation of a linear or nonlinear function of thea posteriori class probabilities estimated by the individual experts. Classifier combination can therefore be viewed as a multistage classification process whereby thea posteriori class probabilities generated by the individual classifiers are considered as features for a second stage classification scheme. Most importantly, when the linear or nonlinear combination functions are obtained by training, the distinctions between the two scenarios fade away, and one can view classifier fusion in a unified way.     

Likelihood ratio-based biometric score fusion

Multibiometric systems fuse information from different sources to compensate for the limitations in performance of individual matchers. We propose a framework for the optimal combination of match scores that is based on the likelihood ratio test. The distributions of genuine and impostor match scores are modeled as finite Gaussian mixture model. The proposed fusion approach is general in its ability to handle 1) discrete values in biometric match score distributions, 2) arbitrary scales and distributions of match scores, 3) correlation between the scores of multiple matchers, and 4) sample quality of multiple biometric sources. Experiments on three multibiometric databases indicate that the proposed fusion framework achieves consistently high performance compared to commonly used score fusion techniques based on score transformation and classification.     

A comprehensive overview of biometric fusion

The performance of a biometric system that relies on a single biometric modality (e.g., fingerprints only) is often stymied by various factors such as poor data quality or limited scalability. Multibiometric systems utilize the principle of fusion to combine information from multiple sources in order to improve recognition accuracy whilst addressing some of the limitations of single-biometric systems. The past two decades have witnessed the development of a large number of biometric fusion schemes. This paper presents an overview of biometric fusion with specific focus on three questions: what to fuse, when to fuse, and how to fuse. A comprehensive review of techniques incorporating ancillary information in the biometric recognition pipeline is also presented. In this regard, the following topics are discussed: (i) incorporating data quality in the biometric recognition pipeline; (ii) combining soft biometric attributes with primary biometric identifiers; (iii) utilizing contextual information to improve biometric recognition accuracy; and (iv) performing continuous authentication using ancillary information. In addition, the use of information fusion principles for presentation attack detection and multibiometric cryptosystems is also discussed. Finally, some of the research challenges in biometric fusion are enumerated. The purpose of this article is to provide readers a comprehensive overview of the role of information fusion in biometrics.     

Score normalization in multimodal biometric systems

Multimodal biometric systems consolidate the evidence presented by multiple biometric sources and typically provide better recognition performance compared to systems based on a single biometric modality. Although information fusion in a multimodal system can be performed at various levels, integration at the matching score level is the most common approach due to the ease in accessing and combining the scores generated by different matchers. Since the matching scores output by the various modalities are heterogeneous, score normalization is needed to transform these scores into a common domain, prior to combining them. In this paper, we have studied the performance of different normalization techniques and fusion rules in the context of a multimodal biometric system based on the face, fingerprint and hand-geometry traits of a user. Experiments conducted on a database of 100 users indicate that the application of min–max, z-score, and tanh normalization schemes followed by a simple sum of scores fusion method results in better recognition performance compared to other methods. However, experiments also reveal that the min–max and z-score normalization techniques are sensitive to outliers in the data, highlighting the need for a robust and efficient normalization procedure like the tanh normalization. It was also observed that multimodal systems utilizing user-specific weights perform better compared to systems that assign the same set of weights to the multiple biometric traits of all users.     

Group sparse representation based classification for multi-feature multimodal biometrics

Multimodal biometrics technology consolidates information obtained from multiple sources at sensor level, feature level, match score level, and decision level. It is used to increase robustness and provide broader population coverage for inclusion. Due to the inherent challenges involved with feature-level fusion, combining multiple evidences is attempted at score, rank, or decision level where only a minimal amount of information is preserved. In this paper, we propose the Group Sparse Representation based Classifier (GSRC) which removes the requirement for a separate feature-level fusion mechanism and integrates multi-feature representation seamlessly into classification. The performance of the proposed algorithm is evaluated on two multimodal biometric datasets. Experimental results indicate that the proposed classifier succeeds in efficiently utilizing a multi-feature representation of input data to perform accurate biometric recognition.     

An improved method of early diagnosis of smoking-induced respiratory changes using machine learning algorithms

The purpose of this study was to develop an automatic classifier to increase the accuracy of the forced oscillation technique (FOT) for diagnosing early respiratory abnormalities in smoking patients. The data consisted of FOT parameters obtained from 56 volunteers, 28 healthy and 28 smokers with low tobacco consumption. Many supervised learning techniques were investigated, including logistic linear classifiers, k nearest neighbor (KNN), neural networks and support vector machines (SVM). To evaluate performance, the ROC curve of the most accurate parameter was established as baseline. To determine the best input features and classifier parameters, we used genetic algorithms and a 10-fold cross-validation using the average area under the ROC curve (AUC). In the first experiment, the original FOT parameters were used as input. We observed a significant improvement in accuracy (KNN = 0.89 and SVM = 0.87) compared with the baseline (0.77). The second experiment performed a feature selection on the original FOT parameters. This selection did not cause any significant improvement in accuracy, but it was useful in identifying more adequate FOT parameters. In the third experiment, we performed a feature selection on the cross products of the FOT parameters. This selection resulted in a further increase in AUC (KNN = SVM = 0.91), which allows for high diagnostic accuracy. In conclusion, machine learning classifiers can help identify early smoking-induced respiratory alterations. The use of FOT cross products and the search for the best features and classifier parameters can markedly improve the performance of machine learning classifiers.     

A user-specific and selective multimodal biometric fusion strategy by ranking subjects

The recognition performance of a biometric system varies significantly from one enrolled user to another. As a result, there is a need to tailor the system to each user. This study investigates a relatively new fusion strategy that is both user-specific and selective. By user-specific, we understand that each user in a biometric system has a different set of fusion parameters that have been tuned specifically to a given enrolled user. By selective, we mean that only a subset of modalities may be chosen for fusion. The rationale for this is that if one biometric modality is sufficiently good to recognize a user, fusion by multimodal biometrics would not be necessary, we advance the state of the art in user-specific and selective fusion in the following ways: (1) provide thorough analyses of (a) the effect of pre-processing the biometric output (prior to applying a user-specific score normalization procedure) in order to improve its central tendency and (b) the generalisation ability of user-specific parameters; (2) propose a criterion to rank the users based solely on a training score dataset in such a way that the obtained rank order will maximally correlate with the rank order that is obtained if it were to be computed on the test set; and, (3) experimentally demonstrate the performance gain of a user-specific and -selective fusion strategy across fusion data sets at different values of "pruning rate" that control the percentage of subjects for whom fusion is not required. Fifteen sets of multimodal fusion experiments carried out on the XM2VTS score-level benchmark database show that even though our proposed user-specific and -selective fusion strategy, its performance compares favorably with the conventional fusion system that considers all information.     

Feature selection for support vector machine-based face-iris multimodal biometric system

Multimodal biometric can overcome the limitation possessed by single biometric trait and give better classification accuracy. This paper proposes face-iris multimodal biometric system based on fusion at matching score level using support vector machine (SVM). The performances of face and iris recognition can be enhanced using a proposed feature selection method to select an optimal subset of features. Besides, a simple computation speed-up method is proposed for SVM. The results show that the proposed feature selection method is able improve the classification accuracy in terms of total error rate. The support vector machine-based fusion method also gave very promising results.     

Structural hidden Markov models for biometrics: Fusion of face and fingerprint

The goal of this paper is threefold: (i) propose a novel face and fingerprint feature modeling using the structural hidden Markov models (SHMMs) paradigm, (ii) explore the use of some feature extraction techniques such as ridgelet transform, discrete wavelet transform with various classifiers for biometric identification, and (iii) determine the best method for classifier combination. The experimental results reported in both fingerprint and face recognition reveal that the SHMMs concept is promising since it has outperformed several state-of-the-arts classifiers when combined with the discrete wavelet transform. Besides, this study has shown that the ridgelet transform without principal components analysis (PCA) dimension reduction fits better with the support vector machines (SVMs) classifier than it does with the SHMMs in the fingerprint recognition task. Finally, these results also reveal a small improvement of the bimodal biometric system over unimodal systems; which suggest that a most effective fusion scheme is necessary.     

Likelihood ratio based features for a trained biometric score fusion

In this work, we present a novel trained method for combining biometric matchers at the score level. The new method is based on a combination of machine learning classifiers trained using the match scores from different biometric approaches as features. The parameters of a finite Gaussian mixture model are used for modelling the genuine and impostor score densities during the fusion step.Several tests on different biometric verification systems (related to fingerprints, palms, fingers, hand geometry and faces) show that the new method outperforms other trained and non-trained approaches for combining biometric matchers.We have tested some different classifiers, support vector machines, AdaBoost of neural networks, and their random subspace versions, demonstrating that the choice for the proposed method is the Random Subspace of AdaBoost.     

Robustness of multimodal biometric fusion methods against spoof attacks

In this paper, we address the security of multimodal biometric systems when one of the modes is successfully spoofed. We propose two novel fusion schemes that can increase the security of multimodal biometric systems. The first is an extension of the likelihood ratio based fusion scheme and the other uses fuzzy logic. Besides the matching score and sample quality score, our proposed fusion schemes also take into account the intrinsic security of each biometric system being fused. Experimental results have shown that the proposed methods are more robust against spoof attacks when compared with traditional fusion methods.     

结合并行融合的序列化多模态生物特征识别系统框架

针对多模态生物特征识别系统并行融合模式中使用方便性和使用效率方面的问题,在现有序列化多模态生物特征识别系统的基础上,提出了一种结合并行融合和序列化融合的多生物特征识别系统框架。框架中首先采用步态、人脸与指纹三种生物特征的不同组合方式以加权相加的得分级融合算法进行的识别过程;其次,利用在线的半监督学习技术提高弱特征的识别性能,从而进一步增强系统的使用方便性和识别可靠性。理论分析和实验结果表明,在此框架下,随使用时间的推移,系统能够通过在线学习提高弱分类器的性能,用户的使用方便性和系统的识别精度都得到了进一步提升。     

Face and palmprint pixel level fusion and Kernel DCV-RBF classifier for small sample biometric recognition

Recently, multi-modal biometric fusion techniques have attracted increasing atove the recognition performance in some difficult biometric problems. The small sample biometric recognition problem is such a research difficulty in real-world applications. So far, most research work on fusion techniques has been done at the highest fusion level, i.e. the decision level. In this paper, we propose a novel fusion approach at the lowest level, i.e. the image pixel level. We first combine two kinds of biometrics: the face feature, which is a representative of contactless biometric, and the palmprint feature, which is a typical contacting biometric. We perform the Gabor transform on face and palmprint images and combine them at the pixel level. The correlation analysis shows that there is very small correlation between their normalized Gabor-transformed images. This paper also presents a novel classifier, KDCV-RBF, to classify the fused biometric images. It extracts the image discriminative features using a Kernel discriminative common vectors (KDCV) approach and classifies the features by using the radial base function (RBF) network. As the test data, we take two largest public face databases (AR and FERET) and a large palmprint database. The experimental results demonstrate that the proposed biometric fusion recognition approach is a rather effective solution for the small sample recognition problem.     

Overview of the combination of biometric matchers

Biometric identity verification refers to technologies used to measure human physical or behavioral characteristics, which offer a radical alternative to passports, ID cards, driving licenses or PIN numbers in authentication. Since biometric systems present several limitations in terms of accuracy, universality, distinctiveness, acceptability, methods for combining biometric matchers have attracted increasing attention of researchers with the aim of improving the ability of systems to handle poor quality and incomplete data, achieving scalability to manage huge databases of users, ensuring interoperability, and protecting user privacy against attacks. The combination of biometric systems, also known as “biometric fusion”, can be classified into unimodal biometric if it is based on a single biometric trait and multimodal biometric if it uses several biometric traits for person authentication.The main goal of this study is to analyze different techniques of information fusion applied in the biometric field. This paper overviews several systems and architectures related to the combination of biometric systems, both unimodal and multimodal, classifying them according to a given taxonomy. Moreover, we deal with the problem of biometric system evaluation, discussing both performance indicators and existing benchmarks.As a case study about the combination of biometric matchers, we present an experimental comparison of many different approaches of fusion of matchers at score level, carried out on three very different benchmark databases of scores. Our experiments show that the most valuable performance is obtained by mixed approaches, based on the fusion of scores. The source code of all the method implemented for this research is freely available for future comparisons¹.After a detailed analysis of pros and cons of several existing approaches for the combination of biometric matchers and after an experimental evaluation of some of them, we draw our conclusion and suggest some future directions of research, hoping that this work could be a useful start point for newer research.     

Fusion of face and speech data for person identity verification

Biometric person identity authentication is gaining more and more attention. The authentication task performed by an expert is a binary classification problem: reject or accept identity claim. Combining experts, each based on a different modality (speech, face, fingerprint, etc.), increases the performance and robustness of identity authentication systems. In this context, a key issue is the fusion of the different experts for taking a final decision (i.e., accept or reject identity claim). We propose to evaluate different binary classification schemes (support vector machine, multilayer perceptron, C4.5 decision tree, Fisher's linear discriminant, Bayesian classifier) to carry on the fusion. The experimental results show that support vector machines and Bayesian classifier achieve almost the same performances, and both outperform the other evaluated classifiers.     

Monitoring the spatio-temporal dynamics of geometrid moth outbreaks in birch forest using MODIS-NDVI data

Defoliation caused by repeated outbreaks of cyclic geometrid moths is the most prominent natural disturbance factor in the northern-boreal birch forest. Evidence suggests that recent changes in outbreak distribution and duration can be attributed to climate warming. There is hence an immediate need for methods that can be applied to characterize the geographical distribution of outbreaks. Here we assess the reliability of MODIS (Moderate Resolution Imaging Spectroradiometer) 16-day NDVI data for generating time series of the distribution of defoliation caused by moths attacking birch forest in Fennoscandia. We do so by first establishing the relationship between ground measures of moth larval density and a defoliation score based on MODIS-NDVI. We then calibrate and validate a model with the MODIS-NDVI defoliation score as a classifier to discriminate between areas with and without visible defoliation as identified from orthophotos and provide two examples of application of the model. We found the MODIS defoliation score to be a valid proxy for larval density (R² = 0.88-0.93) above a certain, low threshold (a defoliation score of ~ 5%). Areas with and without visible defoliation could be discriminated based on defoliation score with a substantial strength of agreement (max kappa = 0.736), and the resulting model was able to predict the proportion of area with visible defoliation in independent test areas with good reliability across the range of proportions. We conclude that satellite-derived defoliation patterns can be an invaluable tool for generating indirect population dynamical data that permits the development of targeted monitoring on relevant regional scales.