多模态生物特征提取及相关性评价综述

生物特征识别是身份认证的重要手段,特征提取技术在其中扮演了关键角色,直接影响识别的结果。随着特征提取技术日趋成熟,学者们逐渐将目光投向了生物特征间的相关性问题。本文以单模态和多模态生物识别中的特征提取方法为研究对象,回顾了人脸与指纹的特征提取方法,分析了基于经验知识的特征分类提取方法以及基于深度学习的计算机逻辑采样提取方法,并从图像处理的角度对单模态与多模态方法进行对比。以当前多模态生物特征提取方法和DNA表达过程为引,提出了不同模态的生物特征之间存在相关性的猜想,以及对这一猜想进行建模的思路。在多模态生物特征提取的基础上,对今后可能有进展的各生物特征之间的相关性建模进行了展望。     

A fast feature selection technique in multi modal biometrics using cloud framework

Biometrics refers to the process that uses biological or physiological traits to identify individuals. The progress seen in technology and security has a vital role to play in Biometric recognition which is a reliable technique to validate individuals and their identity. The biometric identification is generally based on either their physical traits or their behavioural traits. The multimodal biometrics makes use of either two or more of the modalities to improve recognition. There are some popular modalities of biometrics that are palm print, finger vein, iris, face or fingerprint recognition. Another important challenge found with multimodal biometric features is the fusion, which could result in a large set of feature vectors. Most biometric systems currently use a single model for user authentication. In this existing work, a modified method of heuristics that is efficiently used to identify an optimal feature set that is based on a wrapper-based feature selection technique. The proposed method of feature selection uses the Ant Colony Optimization (ACO) and the Particle Swarm Optimization (PSO) are used to feature extraction and classification process utilizes the integration of face, and finger print texture patterns. The set of training images is converted to grayscale. The crossover operator is applied to generate multiple samples for each number of images. The wok proposed here is pre-planned for each weight of each biometric modality, which ensures that even if a biometric modality does not exist at the time of verification, a person can be certified to provide calculated weights the threshold value. The proposed method is demonstrated better result for fast feature selection in bio metric image authentication and also gives high effectiveness security.     

Feature-level fusion of fingerprint and finger-vein for personal identification

Multimodal biometrics based on feature-level fusion is a significant topic in personal identification research community. In this paper, a new fingerprint-vein based biometric method is proposed for making a finger more universal in biometrics. The fingerprint and finger-vein features are first exploited and extracted using a unified Gabor filter framework. Then, a novel supervised local-preserving canonical correlation analysis method (SLPCCAM) is proposed to generate fingerprint-vein feature vectors (FPVFVs) in feature-level fusion. Based on FPVFVs, the nearest neighborhood classifier is employed for personal identification finally. Experimental results show that the proposed approach has a high capability in fingerprint-vein based personal recognition as well as multimodal feature-level fusion.     

Adaptive management of multimodal biometrics fusion using ant colony optimization

This paper presents a new approach for the adaptive management of multimodal biometrics to meet a wide range of application dependent adaptive security requirements. In this work, ant colony optimization (ACO) is employed for the selection of key parameters like decision threshold and fusion rule, to ensure the optimal performance in meeting varying security requirements during the deployment of multimodal biometrics systems. Particle swarm optimization (PSO) has been widely utilized for the optimal selection of these parameters in the earlier attempts in the literature [Veeramachaneni et al., 2005] and [Kumar et al., 2010]. However, in PSO these parameters are computed in continuous domain while they are assumed to be better represented as discrete variables [Kumar et al., 2010]. This paper therefore proposes the use of ACO, in which discrete biometric verification parameters are computed to ensure the optimal performance from the multimodal biometrics system. The proposed ACO based framework is also extended to the pattern classification approach where fuzzy binary decision tree (FBDT) is utilized for two-class biometrics verification. The experimental results are presented on true multimodal systems from various publicly available databases; IITD databases of palmprint and iris, XM2VTS database of speech and faces, and the NIST BSSR1 databases of faces and fingerprint images. Our experimental results presented in this paper suggest that (i) ACO based approach is capable of operating on significantly small error rates in comparison to the widely employed PSO for automated selection of biometrics fusion rules/parameters, (ii) the score-level fusion yields better performance with lower error rate in comparison to the decision level fusion, and finally (iii) the FBDT based classification approach delivers considerably superior performance for the adaptive biometrics verification.     

Generation of new points for training set and feature-level fusion in multimodal biometric identification

Multimodal biometrics has gained interest in the recent past due to its improved recognition rate over unibiometric and unimodal systems. Fusion at feature level is considered here for the purpose of recognition. The biometrics considered for fusion are face and iris. Here, new face images along with iris images are generated, and they are included in the training set. Feature-level fusion is incorporated. The recognition rates of the classification algorithm thus obtained are statistically found to be significantly better than the existing feature-level fusion and classification techniques.     

Feature fusion of side face and gait for video-based human identification

Video-based human recognition at a distance remains a challenging problem for the fusion of multimodal biometrics. As compared to the approach based on match score level fusion, in this paper, we present a new approach that utilizes and integrates information from side face and gait at the feature level. The features of face and gait are obtained separately using principal component analysis (PCA) from enhanced side face image (ESFI) and gait energy image (GEI), respectively. Multiple discriminant analysis (MDA) is employed on the concatenated features of face and gait to obtain discriminating synthetic features. This process allows the generation of better features and reduces the curse of dimensionality. The proposed scheme is tested using two comparative data sets to show the effect of changing clothes and face changing over time. Moreover, the proposed feature level fusion is compared with the match score level fusion and another feature level fusion scheme. The experimental results demonstrate that the synthetic features, encoding both side face and gait information, carry more discriminating power than the individual biometrics features, and the proposed feature level fusion scheme outperforms the match score level and another feature level fusion scheme. The performance of different fusion schemes is also shown as cumulative match characteristic (CMC) curves. They further demonstrate the strength of the proposed fusion scheme.     

Minutiae and modified Biocode fusion for fingerprint-based key generation

Key generation from biometrics has been studied intensively in recent years, linking a key with certain biometric enhances the strength of identity authentication. But the state-of-the-art key generation systems are far away from practicality due to low accuracy. The special manner of biometric matching makes a single feature based key generation system difficult to obtain a high recognition accuracy. Integrating more features into key generation system may be a potential solution to improve the system performance. In this paper, we propose a fingerprint based key generation system under the framework of fuzzy extractor by fusing two kinds of features: minutia-based features and image-based features. Three types of sketch, including minutiae based sketch, modified Biocode based sketch, and combined feature based sketch, are constructed to deal with the feature differences. Our system is tested on FVC2002 DB1 and DB2, and the experimental results show that the fusion scheme effectively improves the system performance compared with the systems based only on minutiae or modified Biocode.     

Enhancement of multimodal biometric segregation using unconstrained cohort normalisation

This paper presents an investigation into the effects, on the accuracy of multimodal biometrics, of introducing unconstrained cohort normalisation (UCN) into the score-level fusion process. Whilst score normalisation has been widely used in voice biometrics, its effectiveness in other biometrics has not been previously investigated. This study aims to explore the potential usefulness of the said score normalisation technique in face biometrics and to investigate its effectiveness for enhancing the accuracy of multimodal biometrics. The experimental investigations involve the two recognition modes of verification and open-set identification, in clean mixed-quality and degraded data conditions. Based on the experimental results, it is demonstrated that the capabilities provided by UCN can significantly improve the accuracy of fused biometrics. The paper presents the motivation for, and the potential advantages of, the proposed approach and details the experimental study.     

融合多模态数据的人体动作识别方法研究

基于多模态融合的人体动作识别技术被广泛研究与应用,其中基于特征级或决策级的融合是在单一级别阶段下进行的,无法将真正的语义信息从数据映射到分类器。提出一种多级多模态融合的人体动作识别方法,使其更适应实际的应用场景。在输入端将深度数据转换为深度运动投影图,并将惯性数据转换成信号图像,通过局部三值模式分别对深度运动图和信号图像进行处理,使每个输入模态进一步转化为多模态。将所有的模态通过卷积神经网络训练进行提取特征,并把提取到的特征通过判别相关分析进行特征级融合。利用判别相关分析最大限度地提高两个特征集中对应特征的相关性,同时消除每个特征集中不同类之间的特征相关性,将融合后的特征作为多类支持向量机的输入进行人体动作识别。在UTD-MHAD和UTD Kinect V2 MHAD两个多模态数据集上的实验结果表明,多级多模态融合框架在两个数据集上的识别精度分别达到99.8%和99.9%,具有较高的识别准确率。     

Person recognition by fusing palmprint and palm vein images based on “Laplacianpalm” representation

Unimodal analysis of palmprint and palm vein has been investigated for person recognition. One of the problems with unimodality is that the unimodal biometric is less accurate and vulnerable to spoofing, as the data can be imitated or forged. In this paper, we present a multimodal personal identification system using palmprint and palm vein images with their fusion applied at the image level. The palmprint and palm vein images are fused by a new edge-preserving and contrast-enhancing wavelet fusion method in which the modified multiscale edges of the palmprint and palm vein images are combined. We developed a fusion rule that enhances the discriminatory information in the images. Here, a novel palm representation, called “Laplacianpalm” feature, is extracted from the fused images by the locality preserving projections (LPP). Unlike the Eigenpalm approach, the “Laplacianpalm” finds an embedding that preserves local information and yields a palm space that best detects the essential manifold structure. We compare the proposed “Laplacianpalm” approach with the Fisherpalm and Eigenpalm methods on a large data set. Experimental results show that the proposed “Laplacianpalm” approach provides a better representation and achieves lower error rates in palm recognition. Furthermore, the proposed multimodal method outperforms any of its individual modality.     

Single-sensor hand-vein multimodal biometric recognition using multiscale deep pyramidal approach

Biometrics has emerged as a powerful technology for person authentication in various scenarios including forensic and civilian applications. Deployment of biometric solutions that use cues from multiple modalities enhances the reliability and robustness of authentication necessary to meet the increasingly stringent security requirements. However, there are two drawbacks typically associated with multimodal biometrics. Firstly, the image acquisition process in such systems is not very user-friendly, primarily due to the time and effort required to capture biometric samples belonging to multiple modalities. Secondly, the overall cost is higher as they employ multiple biometric sensors. To overcome these drawbacks, we employ a single NIR sensor-based image acquisition in the proposed approach for hand-vein recognition. From the input hand image, a palm-vein and four finger-vein subimages are extracted. These images are then enhanced by CLAHE and transformed into illumination invariant representation using center-symmetric local binary pattern (CS-LBP). Further, a hierarchical non-rigid matching technique inspired by the architecture of deep convolutional networks is employed for matching the CS-LBP features. Finally, weighted sum rule-based matching score-level fusion is performed to combine the palm-vein and the four finger-vein modalities. A set of rigorous experiments has been performed on an in-house database collected from the left and right hands of 185 subjects and the publicly available CASIA dataset. The proposed approach achieves equal error rates of 0.13% and 1.21%, and rank-1 identification rates of 100% and 100% on the in-house and CASIA datasets, respectively. Additionally, we compare the proposed approach with the state-of-the-art techniques proposed for vascular biometric recognition in the literature. The important findings are (1) the proposed approach outperforms all the existing techniques considered in this study, (2) the fusion of palm-vein and finger-vein modalities consistently leads to better performance for all the feature extraction techniques considered in this work. (3) Furthermore, our experimental results also suggest that considering the constituent palm-vein and finger-vein images instead of the entire hand-vein images achieves better performance.     

Multimodal integration-a statistical view

We present a statistical approach to developing multimodal recognition systems and, in particular, to integrating the posterior probabilities of parallel input signals involved in the multimodal system. We first identify the primary factors that influence multimodal recognition performance by evaluating the multimodal recognition probabilities. We then develop two techniques, an estimate approach and a learning approach, which are designed to optimize accurate recognition during the multimodal integration process. We evaluate these methods using Quickset, a speech/gesture multimodal system, and report evaluation results based on an empirical corpus collected with Quickset. From an architectural perspective, the integration technique presented offers enhanced robustness. It also is premised on more realistic assumptions than previous multimodal systems using semantic fusion. From a methodological standpoint, the evaluation techniques that we describe provide a valuable tool for evaluating multimodal systems     

Fusion of Band Limited Phase Only Correlation and Width Centroid Contour Distance for finger based biometrics

In this paper, a new approach of multimodal finger biometrics based on the fusion of finger vein and finger geometry recognition is presented. In the proposed method, Band Limited Phase Only Correlation (BLPOC) is utilized to measure the similarity of finger vein images. Unlike previous methods, BLPOC is resilient to noise, occlusions and rescaling factors; thus can enhance the performance of finger vein recognition. As for finger geometry recognition, a new type of geometrical features called Width-Centroid Contour Distance (WCCD) is proposed. This WCCD combines the finger width with Centroid Contour Distance (CCD). As compared with the single type of feature, the fusion of W and CCD can improve the accuracy of finger geometry recognition. Finally, we integrate the finger vein and finger geometry recognitions by a score-level fusion method based on the weighted SUM rule. Experimental evaluation using our own database which was collected from 123 volunteers resulted in an efficient recognition performance where the equal error rate (EER) was 1.78% with a total processing time of 24.22 ms.     

基于Adaboost的人脸与虹膜融合识别

传统生物特征识别系统的识别率经常受到环境以及生物学特征的自身局限性影响。针对该不足,提出一种基于人脸与虹膜特征级融合的多模态生物识别系统,采用中心对称局部二值模式算子提取人脸和虹膜的纹理特征,将人脸特征与虹膜特征线性整合成混合特征向量,利用Adaboost算法从该混合特征向量中优选出一组最佳特征组合,从而构成强分类器。实验结果表明,该多模态系统相比单模态系统具有更好的鲁棒性。     

Selection of optimized features for fusion of palm print and finger knuckle‐based person authentication

The impact of digital technology in biometrics is much more efficient at interpreting data than humans, which results in completely replacement of manual identification procedures in forensic science. Because the single modality‐based biometric frameworks limit performance in terms of accuracy and anti‐spoofing capabilities due to the presence of low quality data, therefore, information fusion of more than one biometric characteristic in pursuit of high recognition results can be beneficial. In this article, we present a multimodal biometric system based on information fusion of palm print and finger knuckle traits, which are least associated to any criminal investigation as evidence yet. The proposed multimodal biometric system might be useful to identify the suspects in case of physical beating or kidnapping and establish supportive scientific evidences, when no fingerprint or face information is present in photographs. The first step in our work is data preprocessing, in which region of interest of palm and finger knuckle images have been extracted. To minimize nonuniform illumination effects, we first normalize the detected circular palm or finger knuckle and then apply line ordinal pattern (LOP)‐based encoding scheme for texture enrichment. The nondecimated quaternion wavelet provides denser feature representation at multiple scales and orientations when extracted over proposed LOP encoding and increases the discrimination power of line and ridge features. To best of our knowledge, this first attempt is a combination of backtracking search algorithm and 2D²LDA has been employed to select the dominant palm and knuckle features for classification. The classifiers output for two modalities are combined at unsupervised rank level fusion rule through Borda count method, which shows an increase in performance in terms of recognition and verification, that is, 100% (correct recognition rate), 0.26% (equal error rate), 3.52 (discriminative index), and 1,262 m (speed).     

基于视听觉感知系统的多模态情感识别

情绪识别作为人机交互的热门领域,其技术已经被应用于医学、教育、安全驾驶、电子商务等领域.情绪主要由面部表情、声音、话语等进行表达,不同情绪表达时的面部肌肉、语气、语调等特征也不相同,使用单一模态特征确定的情绪的不准确性偏高,考虑到情绪表达主要通过视觉和听觉进行感知,本文提出了一种基于视听觉感知系统的多模态表情识别算法,分别从语音和图像模态出发,提取两种模态的情感特征,并设计多个分类器为单特征进行情绪分类实验,得到多个基于单特征的表情识别模型.在语音和图像的多模态实验中,提出了晚期融合策略进行特征融合,考虑到不同模型间的弱依赖性,采用加权投票法进行模型融合,得到基于多个单特征模型的融合表情识别模型.本文使用AFEW数据集进行实验,通过对比融合表情识别模型与单特征的表情识别模型的识别结果,验证了基于视听觉感知系统的多模态情感识别效果要优于基于单模态的识别效果.     

指节纹识别综述*

相比其它生物特征,指节纹具有特征丰富,采集设备价格低,易于结合手形、手指静脉及掌纹组成性能鲁棒的多模态识别系统等优点.文中首先介绍指节纹的定义、数据采集、预处理方法等,之后详细介绍各种指节纹识别算法及多模态识别方案.根据特征提取及匹配方法的不同,将指节纹识别算法分为6类:基于结构的算法、基于子空间学习的算法、基于编码的算法、基于纹理特征的算法、基于相关滤波器的算法和基于局部特征描述子的算法.回顾和总结各种算法的特点,展望未来指节纹识别的发展方向.     

Situation awareness of cancelable biometric system

Recently, cancelable biometrics emerged as one of the highly effective methods of template protection. The concept behind the cancelable biometrics or cancelability is a transformation of a biometric data or extracted feature into an alternative form, which cannot be used by the imposter or intruder easily, and can be revoked if compromised. In this paper, we present a novel architecture for template generation in the context of situation awareness system in real and virtual applications. We develop a novel cancelable biometric template generation algorithm utilizing random biometric fusion, random projection and selection. Proposed random cross-folding method generate cancelable biometric template from multiple biometric traits. We further validate the performance of the proposed algorithm using a virtual multimodal face and ear database.