Multimodal Biometric Score Fusion Using Gaussian Mixture Model and Monte Carlo Method

Multimodal biometric fusion is gaining more attention among researchers in recent days. As multimodal biometric system consolidates the information from multiple biometric sources, the effective fusion of information obtained at score level is a challenging task. In this paper, we propose a framework for optimal fusion of match scores based on Gaussian Mixture Model (GMM) and Monte Carlo sampling based hypothesis testing. The proposed fusion approach has the ability to handle: 1) small size of match scores as is more commonly encountered in biometric fusion, and 2) arbitrary distribution of match scores which is more pronounced when discrete scores and multimodal features are present. The proposed fusion scheme is compared with well established schemes such as Likelihood Ratio (LR) method and weighted SUM rule. Extensive experiments carried out on five different multimodal biometric databases indicate that the proposed fusion scheme achieves higher performance as compared with other contemporary state of art fusion techniques.     

Likelihood ratio based features for a trained biometric score fusion

In this work, we present a novel trained method for combining biometric matchers at the score level. The new method is based on a combination of machine learning classifiers trained using the match scores from different biometric approaches as features. The parameters of a finite Gaussian mixture model are used for modelling the genuine and impostor score densities during the fusion step.Several tests on different biometric verification systems (related to fingerprints, palms, fingers, hand geometry and faces) show that the new method outperforms other trained and non-trained approaches for combining biometric matchers.We have tested some different classifiers, support vector machines, AdaBoost of neural networks, and their random subspace versions, demonstrating that the choice for the proposed method is the Random Subspace of AdaBoost.     

Score normalization in multimodal biometric systems

Multimodal biometric systems consolidate the evidence presented by multiple biometric sources and typically provide better recognition performance compared to systems based on a single biometric modality. Although information fusion in a multimodal system can be performed at various levels, integration at the matching score level is the most common approach due to the ease in accessing and combining the scores generated by different matchers. Since the matching scores output by the various modalities are heterogeneous, score normalization is needed to transform these scores into a common domain, prior to combining them. In this paper, we have studied the performance of different normalization techniques and fusion rules in the context of a multimodal biometric system based on the face, fingerprint and hand-geometry traits of a user. Experiments conducted on a database of 100 users indicate that the application of min–max, z-score, and tanh normalization schemes followed by a simple sum of scores fusion method results in better recognition performance compared to other methods. However, experiments also reveal that the min–max and z-score normalization techniques are sensitive to outliers in the data, highlighting the need for a robust and efficient normalization procedure like the tanh normalization. It was also observed that multimodal systems utilizing user-specific weights perform better compared to systems that assign the same set of weights to the multiple biometric traits of all users.     

Overview of the combination of biometric matchers

Biometric identity verification refers to technologies used to measure human physical or behavioral characteristics, which offer a radical alternative to passports, ID cards, driving licenses or PIN numbers in authentication. Since biometric systems present several limitations in terms of accuracy, universality, distinctiveness, acceptability, methods for combining biometric matchers have attracted increasing attention of researchers with the aim of improving the ability of systems to handle poor quality and incomplete data, achieving scalability to manage huge databases of users, ensuring interoperability, and protecting user privacy against attacks. The combination of biometric systems, also known as “biometric fusion”, can be classified into unimodal biometric if it is based on a single biometric trait and multimodal biometric if it uses several biometric traits for person authentication.The main goal of this study is to analyze different techniques of information fusion applied in the biometric field. This paper overviews several systems and architectures related to the combination of biometric systems, both unimodal and multimodal, classifying them according to a given taxonomy. Moreover, we deal with the problem of biometric system evaluation, discussing both performance indicators and existing benchmarks.As a case study about the combination of biometric matchers, we present an experimental comparison of many different approaches of fusion of matchers at score level, carried out on three very different benchmark databases of scores. Our experiments show that the most valuable performance is obtained by mixed approaches, based on the fusion of scores. The source code of all the method implemented for this research is freely available for future comparisons¹.After a detailed analysis of pros and cons of several existing approaches for the combination of biometric matchers and after an experimental evaluation of some of them, we draw our conclusion and suggest some future directions of research, hoping that this work could be a useful start point for newer research.     

A comparison of imputation methods for handling missing scores in biometric fusion

Multibiometric systems, which consolidate or fuse multiple sources of biometric information, typically provide better recognition performance than unimodal systems. While fusion can be accomplished at various levels in a multibiometric system, score-level fusion is commonly used as it offers a good trade-off between data availability and ease of fusion. Most score-level fusion rules assume that the scores pertaining to all the matchers are available prior to fusion. Thus, they are not well equipped to deal with the problem of missing match scores. While there are several techniques for handling missing data in general, the imputation scheme, which replaces missing values with predicted values, is preferred since this scheme can be followed by a standard fusion scheme designed for complete data. In this work, the performance of the following imputation methods are compared in the context of multibiometric fusion: K-nearest neighbor (KNN) schemes, likelihood-based schemes, Bayesian-based schemes and multiple imputation (MI) schemes. Experiments on the MSU database assess the robustness of the schemes in handling missing scores at different missing rates. It is observed that the Gaussian mixture model (GMM)-based KNN imputation scheme results in the best recognition accuracy.     

基于证据理论的不确定模式匹配方法

由于数据源数据模式的自治性、异构性，不确定性是模式匹配过程固有的本质特性。提出了一种基于证据理论的不确定性匹配方法，首先根据属性类型把模式空间分成若干模式子空间；然后将不同的匹配器结果看作不同的证据源，利用不同的匹配器的结果生成了多个基本概率分配函数，采用改进的Dempster组合规则把多个匹配器结果自动组合，减少人工干预，并解决了不同的匹配器结果组合时证据间冲突的问题；最后利用Kuhn Munkres算法获取模式映射。实验结果表明了方法的可行性和有效性。     

Group sparse representation based classification for multi-feature multimodal biometrics

Multimodal biometrics technology consolidates information obtained from multiple sources at sensor level, feature level, match score level, and decision level. It is used to increase robustness and provide broader population coverage for inclusion. Due to the inherent challenges involved with feature-level fusion, combining multiple evidences is attempted at score, rank, or decision level where only a minimal amount of information is preserved. In this paper, we propose the Group Sparse Representation based Classifier (GSRC) which removes the requirement for a separate feature-level fusion mechanism and integrates multi-feature representation seamlessly into classification. The performance of the proposed algorithm is evaluated on two multimodal biometric datasets. Experimental results indicate that the proposed classifier succeeds in efficiently utilizing a multi-feature representation of input data to perform accurate biometric recognition.     

Score level fusion of multimodal biometrics using triangular norms

A multimodal biometric system that alleviates the limitations of the unimodal biometric systems by fusing the information from the respective biometric sources is developed. A general approach is proposed for the fusion at score level by combining the scores from multiple biometrics using triangular norms (t-norms) due to Hamacher, Yager, Frank, Schweizer and Sklar, and Einstein product. This study aims at tapping the potential of t-norms for multimodal biometrics. The proposed approach renders very good performance as it is quite computationally fast and outperforms the score level fusion using the combination approach (min, mean, and sum) and classification approaches like SVM, logistic linear regression, MLP, etc. The experimental evaluation on three databases confirms the effectiveness of score level fusion using t-norms.     

Dynamic registration selection for fingerprint verification

Information fusion is a powerful approach to increasing the accuracy of biometric authentication systems, and is currently an active area of research. The majority of studies focus on combining the results from multiple verification systems at the match score level using either a classification or combination scheme. However, there are advantages to performing the fusion at an earlier stage of processing. Fingerprint registration involves finding the translation and rotation parameters that align two fingerprints; a challenging problem that can be approached in a number of ways. The fusion of fingerprint alignment algorithms is introduced in the form of dynamic registration selection. A Bayesian statistical framework is used to select the most probable alignment produced by competing algorithms. The results of the proposed technique are tested on multiple FVC 2002 databases, and are shown to outperform methods based on match score combination.     

Multibiometrics for face recognition

Fusion is a popular practice to combine multiple sources of biometric information to achieve systems with greater performance and flexibility. In this paper various approaches to fusion within a multibiometrics context are considered and an application to the fusion of 2D and 3D face information is discussed. An optimal method for fusing the accept/reject decisions of individual biometric sources by means of simple logical rules is presented. Experimental results on the FRGC 2D and 3D face data show that the proposed technique performs effectively without the need for score normalization.     

Genetic programming for multibiometrics

Biometric systems suffer from some drawbacks: a biometric system can provide in general good performances except with some individuals as its performance depends highly on the quality of the capture… One solution to solve some of these problems is to use multibiometrics where different biometric systems are combined together (multiple captures of the same biometric modality, multiple feature extraction algorithms, multiple biometric modalities…). In this paper, we are interested in score level fusion functions application (i.e., we use a multibiometric authentication scheme which accept or deny the claimant for using an application). In the state of the art, the weighted sum of scores (which is a linear classifier) and the use of an SVM (which is a non linear classifier) provided by different biometric systems provide one of the best performances. We present a new method based on the use of genetic programming giving similar or better performances (depending on the complexity of the database). We derive a score fusion function by assembling some classical primitives functions (+, ∗, −, … ). We have validated the proposed method on three significant biometric benchmark datasets from the state of the art.     

Performance evaluation of score level fusion in multimodal biometric systems

In a multimodal biometric system, the effective fusion method is necessary for combining information from various single modality systems. In this paper the performance of sum rule-based score level fusion and support vector machines (SVM)-based score level fusion are examined. Three biometric characteristics are considered in this study: fingerprint, face, and finger vein. We also proposed a new robust normalization scheme (Reduction of High-scores Effect normalization) which is derived from min-max normalization scheme. Experiments on four different multimodal databases suggest that integrating the proposed scheme in sum rule-based fusion and SVM-based fusion leads to consistently high accuracy. The performance of simple sum rule-based fusion preceded by our normalization scheme is comparable to another approach, likelihood ratio-based fusion [8] (Nandakumar et al., 2008), which is based on the estimation of matching scores densities. Comparison between experimental results on sum rule-based fusion and SVM-based fusion reveals that the latter could attain better performance than the former, provided that the kernel and its parameters have been carefully selected.     

Fusing multimodal biometrics with quality estimates via a Bayesian belief network

Biometric systems for today's high security applications must meet stringent performance requirements; fusing multiple biometrics can help lower system error rates. Fusion methods include processing biometric modalities sequentially until an acceptable match is obtained, using logical (AND/OR) operations, or summing similarity scores. More sophisticated methods combine scores from separate classifiers for each modality. This paper develops a novel fusion architecture based on Bayesian belief networks. Although Bayesian update methods have been used before, our approach more fully exploits the graphical structure of Bayes nets to define and explicitly model statistical dependencies between relevant variables: per sample measurements, such as match scores and corresponding quality estimates, and global decision variables. These statistical dependencies are in the form of conditional distributions which we model as Gaussian, gamma, log-normal or beta, each of which is determined by its mean and variance, thus significantly reducing training data requirements. Moreover, by conditioning decision variables on quality as well as match score, we can extract information from lower quality measurements rather than rejecting them out of hand. Another feature of our method is a global quality measure designed to be used as a confidence estimate supporting decision making. Preliminary studies using the architecture to fuse fingerprints and voice are reported.     

Personal identity verification by serial fusion of fingerprint and face matchers

The use of personal identity verification systems with multi-modal biometrics has been proposed in order to increase the performance and robustness against environmental variations and fraudulent attacks. Usually multi-modal fusion of biometrics is performed in parallel at the score-level by combining the individual matching scores. This parallel strategy exhibits some drawbacks: (i) all available biometrics are necessary to perform fusion, thus the verification time depends on the slowest system; (ii) some users could be easily recognizable using a certain biometric instead of another one and (iii) the system invasiveness increases. A system characterized by the serial combination of multiple biometrics can be a good trade-off between verification time, performance and acceptability. However, these systems have been poorly investigated, and no support for designing the processing chain has been given so far. In this paper, we propose a novel serial scheme and a simple mathematical model able to predict the performance of two serially combined matchers as function of the selected processing chain. Our model helps the designer in finding the processing chain allowing a trade-off, in particular, between performance and matching time. Experiments carried out on well-known benchmark data sets made up of face and fingerprint images support the usefulness of the proposed methodology and compare it with standard parallel fusion.     

A supervised method to discriminate between impostors and genuine in biometry

In this paper, we describe a supervised technique that allows to develop a more robust biometric system with respect to those based directly on the similarities of the biometric matchers or on the similarities normalised by the unconstrained cohort normalisation.In order to discriminate between genuine and impostors a quadratic discriminant classifier is trained using four features: the similarities of the biometric matcher; the similarities of the biometric matcher after the unconstrained cohort normalisation (UCN); the average scores among the test pattern and the users that belong to the background model; the difference between the user-specific threshold and the user-independent threshold.The proposed technique is validated by extensive experiments carried out on several biometric datasets (palm, finger, 2D and 3D faces, and ear). The experimental results demonstrate that the capabilities provided by our supervised method can significantly improve the performance of a standard biometric matcher or the performance of the standard UCN.     

结合并行融合的序列化多模态生物特征识别系统框架

针对多模态生物特征识别系统并行融合模式中使用方便性和使用效率方面的问题,在现有序列化多模态生物特征识别系统的基础上,提出了一种结合并行融合和序列化融合的多生物特征识别系统框架。框架中首先采用步态、人脸与指纹三种生物特征的不同组合方式以加权相加的得分级融合算法进行的识别过程;其次,利用在线的半监督学习技术提高弱特征的识别性能,从而进一步增强系统的使用方便性和识别可靠性。理论分析和实验结果表明,在此框架下,随使用时间的推移,系统能够通过在线学习提高弱分类器的性能,用户的使用方便性和系统的识别精度都得到了进一步提升。     

Spatiotemporal analysis of human activities for biometric authentication

This paper presents a novel framework for unobtrusive biometric authentication based on the spatiotemporal analysis of human activities. Initially, the subject’s actions that are recorded by a stereoscopic camera, are detected utilizing motion history images. Then, two novel unobtrusive biometric traits are proposed, namely the static anthropometric profile that accurately encodes the inter-subject variability with respect to human body dimensions, while the activity related trait that is based on dynamic motion trajectories encodes the behavioral inter-subject variability for performing a specific action. Subsequently, score level fusion is performed via support vector machines. Finally, an ergonomics-based quality indicator is introduced for the evaluation of the authentication potential for a specific trial. Experimental validation on data from two different datasets, illustrates the significant biometric authentication potential of the proposed framework in realistic scenarios, whereby the user is unobtrusively observed, while the use of the static anthropometric profile is seen to significantly improve performance with respect to state-of-the-art approaches.     

Comparing human and automatic face recognition performance.

Face recognition technologies have seen dramatic improvements in performance over the past decade, and such systems are now widely used for security and commercial applications. Since recognizing faces is a task that humans are understood to be very good at, it is common to want to compare automatic face recognition (AFR) and human face recognition (HFR) in terms of biometric performance. This paper addresses this question by: 1) conducting verification tests on volunteers (HFR) and commercial AFR systems and 2) developing statistical methods to support comparison of the performance of different biometric systems. HFR was tested by presenting face-image pairs and asking subjects to classify them on a scale of "Same," "Probably Same," "Not sure," "Probably Different," and "Different"; the same image pairs were presented to AFR systems, and the biometric match score was measured. To evaluate these results, two new statistical evaluation techniques are developed. The first is a new way to normalize match-score distributions, where a normalized match score t is calculated as a function of the angle from a representation of [false match rate, false nonmatch rate] values in polar coordinates from some center. Using this normalization, we develop a second methodology to calculate an average detection error tradeoff (DET) curve and show that this method is equivalent to direct averaging of DET data along each angle from the center. This procedure is then applied to compare the performance of the best AFR algorithms available to us in the years 1999, 2001, 2003, 2005, and 2006, in comparison to human scores. Results show that algorithms have dramatically improved in performance over that time. In comparison to the performance of the best AFR system of 2006, 29.2% of human subjects performed better, while 37.5% performed worse.     

Multimodal biometrics: Weighted score level fusion based on non-ideal iris and face images

The iris and face are among the most promising biometric traits that can accurately identify a person because their unique textures can be swiftly extracted during the recognition process. However, unimodal biometrics have limited usage since no single biometric is sufficiently robust and accurate in real-world applications. Iris and face biometric authentication often deals with non-ideal scenarios such as off-angles, reflections, expression changes, variations in posing, or blurred images. These limitations imposed by unimodal biometrics can be overcome by incorporating multimodal biometrics. Therefore, this paper presents a method that combines face and iris biometric traits with the weighted score level fusion technique to flexibly fuse the matching scores from these two modalities based on their weight availability. The dataset use for the experiment is self established dataset named Universiti Teknologi Malaysia Iris and Face Multimodal Datasets (UTMIFM), UBIRIS version 2.0 (UBIRIS v.2) and ORL face databases. The proposed framework achieve high accuracy, and had a high decidability index which significantly separate the distance between intra and inter distance.     

Towards a multi-source fusion approach for eye movement-driven recognition

This paper presents a research for the use of multi-source information fusion in the field of eye movement biometrics. In the current state-of-the-art, there are different techniques developed to extract the physical and the behavioral biometric characteristics of the eye movements. In this work, we explore the effects from the multi-source fusion of the heterogeneous information extracted by different biometric algorithms under the presence of diverse visual stimuli. We propose a two-stage fusion approach with the employment of stimulus-specific and algorithm-specific weights for fusing the information from different matchers based on their identification efficacy. The experimental evaluation performed on a large database of 320 subjects reveals a considerable improvement in biometric recognition accuracy, with minimal equal error rate (EER) of 5.8%, and best case Rank-1 identification rate (Rank-1 IR) of 88.6%. It should be also emphasized that although the concept of multi-stimulus fusion is currently evaluated specifically for the eye movement biometrics, it can be adopted by other biometric modalities too, in cases when an exogenous stimulus affects the extraction of the biometric features.