Feature fusion of side face and gait for video-based human identification

Video-based human recognition at a distance remains a challenging problem for the fusion of multimodal biometrics. As compared to the approach based on match score level fusion, in this paper, we present a new approach that utilizes and integrates information from side face and gait at the feature level. The features of face and gait are obtained separately using principal component analysis (PCA) from enhanced side face image (ESFI) and gait energy image (GEI), respectively. Multiple discriminant analysis (MDA) is employed on the concatenated features of face and gait to obtain discriminating synthetic features. This process allows the generation of better features and reduces the curse of dimensionality. The proposed scheme is tested using two comparative data sets to show the effect of changing clothes and face changing over time. Moreover, the proposed feature level fusion is compared with the match score level fusion and another feature level fusion scheme. The experimental results demonstrate that the synthetic features, encoding both side face and gait information, carry more discriminating power than the individual biometrics features, and the proposed feature level fusion scheme outperforms the match score level and another feature level fusion scheme. The performance of different fusion schemes is also shown as cumulative match characteristic (CMC) curves. They further demonstrate the strength of the proposed fusion scheme.     

基于单视图的多姿态人脸识别算法

针对基于多视图的多姿态人脸识别方法的缺陷，即需要对每个人脸拍摄多个视图为前提条件，提出了基于单视图的多姿态人脸识别技术，首先基于二元高次多项式函数最小二乘拟合方法由单视图通过变形生成多姿态人脸图像，然后基于该单视图和生成的多姿态图像进行多姿态人脸识别。实验结果表明该文算法识别的正确率远高于经典算法。     

View-invariant gait recognition based on kinect skeleton feature

Gait recognition is a popular remote biometric identification technology. Its robustness against view variation is one of the challenges in the field of gait recognition. In this paper, the second-generation Kinect (2G–Kinect) is used as a tool to build a 3D–skeleton-based gait dataset, which includes both 2D silhouette images captured by 2G–Kinect and their corresponding 3D coordinates of skeleton joints. Given this dataset, a human walking model is constructed. Referring to the walking model, the length of some specific skeletons is selected as the static features, and the angles of swing limbs as the dynamic features, which are verified to be view-invariant. In addition, the gait recognition abilities of the static and dynamic features are investigated respectively. Given the investigation, a view-invariant gait recognition scheme is proposed based on the matching-level-fusion of the static and dynamic features, and the nearest neighbor (NN) method is used for recognition. Comparison between the existing Kinect-based gait recognition method and the proposed one on different datasets show that the proposed one has better recognition performance.     

Ear recognition based on local information fusion

Ears have rich structural features that are almost invariant with increasing age and facial expression variations. Therefore ear recognition has become an effective and appealing approach to non-contact biometric recognition. This paper gives an up-to date review of research works on ear recognition. Current 2D ear recognition approaches achieve good performance in constrained environments. However the recognition performance degrades severely under pose, lighting and occlusion. This paper proposes a 2D ear recognition approach based on local information fusion to deal with ear recognition under partial occlusion. Firstly, the whole 2D image is separated to sub-windows. Then, Neighborhood Preserving Embedding is used for feature extraction on each sub-window, and we select the most discriminative sub-windows according to the recognition rate. Each sub-window corresponds to a sub-classifier. Thirdly, a sub-classifier fusion approach is used for recognition with partially occluded images. Experimental results on the USTB ear dataset and UND dataset have illustrated that using only few sub-windows we can represent the most meaningful region of the ear, and the multi-classifier model gets higher recognition rate than using the whole image for recognition.     

Novel image fusion scheme based on dependency measure for robust multispectral palmprint recognition

Multispectral palmprint is considered as an effective biometric modality to accurately recognize a subject with high confidence. This paper presents a novel multispectral palmprint recognition system consisting of three functional blocks namely: (1) novel technique to extract Region of Interest (ROI) from the hand images acquired using a contact less sensor (2) novel image fusion scheme based on dependency measure (3) new scheme for feature extraction and classification. The proposed ROI extraction scheme is based on locating the valley regions between fingers irrespective of the hand pose. We then propose a novel image fusion scheme that combines information from different spectral bands using a Wavelet transform from various sub-bands. We then perform the statistical dependency analysis between these sub-bands to perform fusion either by selection or by weighted fusion. To effectively process the information from the fused image, we perform feature extraction using Log-Gabor transform whose feature dimension is reduced using Kernel Discriminant Analysis (KDA) before performing the classification by employing a Sparse Representation Classifier (SRC). Extensive experiments are carried out on a CASIA multispectral palmprint database that shows the strong superiority of our proposed fusion scheme when benchmarked with contemporary state-of-the-art image fusion schemes.     

基于图的人-物交互识别

提出了一种基于图的人与物体的交互（Human-Object Interactions,HOIs）识别方法。为了对静态图像中人与物体间丰富的交互关系进行有效的表示,采用具有强大关系建模能力的图结构为图像生成对应的人-物交互关系图。为了对图像中上下文（context）信息加以利用,提出了引入注意力机制的特征处理网络（Feature Processing Network,FPNet）。通过图注意力（Graph Attention Network,GAT）网络完成对真实的HOIs的检测和识别。该方法在V-COCO数据集与HICO-DET数据集上进行了验证,并与其他方法进行了比较,结果表明该方法具有较好的效果。     

基于卷积神经网络的多尺度葡萄图像识别方法

葡萄品种质量检测需要识别多类别的葡萄，而葡萄图片中存在多种景深变化、多串等多种场景，单一预处理方法存在局限导致葡萄识别的效果不佳。实验的研究对象是大棚中采集的15个类别的自然场景葡萄图像，并建立相应图像数据集Vitis-15。针对葡萄图像中同一类别的差异较大而不同类别的差异较小的问题，提出一种基于卷积神经网络（CNN）的多尺度葡萄图像识别方法。首先，对Vitis-15数据集中的数据通过三种方法进行预处理：旋转图像的数据扩增方法、中心裁剪的多尺度图像方法以及前两种方法的数据融合方法；然后，采用迁移学习方法和卷积神经网络方法来进行分类识别，迁移学习选取ImageNet上预训练的Inception V3网络模型，卷积神经网络采用AlexNet、ResNet、Inception V3这三类模型；最后，提出适合Vitis-15的多尺度图像数据融合的分类模型MS-EAlexNet。实验结果表明，在同样的学习率和同样的测试集上，数据融合方法在MS-EAlexNet上的测试准确率达到了99.92%，相较扩增和多尺度图像方法提升了近1个百分点，并且所提方法在分类小样本数据集上具有较高的效率。     

一种2D+3D人耳融合识别方法研究

提出一种2D和3D模式相融合的人耳识别方法.利用基于Adaboost算法的人耳检测器在2D图像上进行人耳检测,在对应的深度图像中定位出人耳区域.对于2D人耳图像,利用核Fisher鉴别分析法进行特征提取,再利用最近邻分类器进行识别;对于3D人耳深度图,利用3D局部二值模式进行特征提取,结合几何约束和位置约束在测试耳和注册原型耳上进行特征点匹配,并利用匹配点数进行识别.最后将两者进行决策层融合.在UND人耳图像库上的实验结果表明,与单独2D或3D人耳识别相比,文中提出的2D+3D人耳融合识别方法在光照变化情况下能取得更好的识别性能.     

多姿态图像生成的行人重识别算法研究

不同行人的高度相似性以及相同行人外观姿态的差异性,使得不同摄像头下的行人重识别面临严峻的挑战。生成对抗网络可以合成新的图像,被认为是解决行人姿态变化的主要技术手段。提出一种基于多姿态图像生成的行人重识别算法,利用生成对抗网络生成不同姿态的行人图像,通过归一化消除姿态的影响,从而大幅度提升行人重识别的整体性能。该行人重识别算法包括多姿态行人图像生成、不同姿态的行人特征提取与融合、距离度量和重排序三部分内容。在Market-1501数据集和DukeMTMC-ReID数据集上的实验证实了所提出算法的有效性,通过与state-of-the-art行人重识别方法比较,展示了多姿态图像生成方法在行人重识别任务中的优越性,同时表明生成行人图像的特征与原始图像的特征是相互补充的。     

Air‐ground Matching: Appearance‐based GPS‐denied Urban Localization of Micro Aerial Vehicles

In this paper, we address the problem of globally localizing and tracking the pose of a camera‐equipped micro aerial vehicle (MAV) flying in urban streets at low altitudes without GPS. An image‐based global positioning system is introduced to localize the MAV with respect to the surrounding buildings. We propose a novel air‐ground image‐matching algorithm to search the airborne image of the MAV within a ground‐level, geotagged image database. Based on the detected matching image features, we infer the global position of the MAV by back‐projecting the corresponding image points onto a cadastral three‐dimensional city model. Furthermore, we describe an algorithm to track the position of the flying vehicle over several frames and to correct the accumulated drift of the visual odometry whenever a good match is detected between the airborne and the ground‐level images. The proposed approach is tested on a 2 km trajectory with a small quadrocopter flying in the streets of Zurich. Our vision‐based global localization can robustly handle extreme changes in viewpoint, illumination, perceptual aliasing, and over‐season variations, thus outperforming conventional visual place‐recognition approaches. The dataset is made publicly available to the research community. To the best of our knowledge, this is the first work that studies and demonstrates global localization and position tracking of a drone in urban streets with a single onboard camera.     

基于双分支时空步态特征融合的深度学习步态识别

针对现有步态识别方法易受拍摄视角、着装变化影响的问题,提出一种融合二维无肩姿态拓扑能量图(shoulderless pose topological energy maps, SPTEM)和三维局部骨骼步态特征(local skeleton gait features, LSGF)的深度学习步态识别方法.首先,利用轻量级BlazePose姿态估计算法提取步态视频序列中的人体姿态拓扑图以生成SPTEM,在提高检测速度的同时减弱衣物变化带来的影响;然后,引入LSGF以弥补单一能量图特征在多变视角情况下识别准确率较低的不足;最后,提出结合注意力机制的时空特征提取网络模型,并在全连接层将双流特征进行一致融合.在CASIA-B数据集上对所提出方法进行验证,并与当前主流的步态识别方法进行比较,结果表明,所提出方法在跨视角和穿大衣/棉衣条件下的步态识别率都有明显提升.     

双分支特征融合网络的步态识别算法

目的 在步态识别算法中,基于外观的方法准确率高且易于实施,但对外观变化敏感;基于模型的方法对外观变化更加鲁棒,但建模困难且准确率较低。为了使步态识别算法在获得高准确率的同时对外观变化具有更好的鲁棒性,提出了一种双分支网络融合外观特征和姿态特征,以结合两种方法的优点。方法 双分支网络模型包含外观和姿态两条分支,外观分支采用Gait Set网络从轮廓图像中提取外观特征;姿态分支采用5层卷积网络从姿态骨架中提取姿态特征。在此基础上构建特征融合模块,融合外观特征和姿态特征,并引入通道注意力机制实现任意尺寸的特征融合,设计的模块结构使其能够在融合过程中抑制特征中的噪声。最后将融合后的步态特征应用于识别行人身份。结果 实验在CASIA-B(Institute of Automation, Chinese Academy of Sciences, Gait Dataset B)数据集上通过跨视角和不同行走状态两种实验设置与目前主流的步态识别算法进行对比,并以Rank-1准确率作为评价指标。在跨视角实验设置的MT(medium-sample training)划分中,该算法在3种行走状态下的准确率分别...     

Fusing continuous spectral images for face recognition under indoor and outdoor illuminants

Novel image fusion approaches, including physics-based weighted fusion, illumination adjustment and rank-based decision level fusion, for spectral face images are proposed for improving face recognition performance compared to conventional images. A new multispectral imaging system is briefly presented which can acquire continuous spectral face images for our concept proof with fine spectral resolution in the visible spectrum. Several experiments are designed and validated by calculating the cumulative match characteristics of probe sets via the well-known recognition engine-FaceIt^®. Experimental results demonstrate that proposed fusion methods outperform conventional images when gallery and probes are acquired under different illuminations and with different time lapses. In the case where probe images are acquired outdoors under different daylight situations, the fused images outperform conventional images by up to 78%.     

Fusion-based encryption scheme for cancelable fingerprint recognition

This paper presents a fingerprint image encryption scheme based on fingerprint image fusion with another visible image that is rich in details. The encryption process is performed with chaotic Baker map, which has large immunity to noise. The image fusion process is performed with the Haar wavelet transform, and it can be implemented with the average or maximum fusion rule. The fusion process is performed, because fingerprint images are not rich in details, and hence the direct application of chaotic Baker map encryption will not be efficient for encrypting this type of images. To obtain an image that is rich in details, it is possible to use another encrypted image with a strong ciphering algorithm such as the RC6. Several perspectives are considered for performance evaluation of the proposed encryption scheme including visual inspection, histogram analysis, correlation coefficient, entropy analysis, processing time, and the effect of noise after decryption. The proposed fingerprint encryption scheme is appropriate for cancelable biometric applications to preserve the privacy of users by keeping their original fingerprints away from usage in the recognition system. The simulation results demonstrate that the proposed image encryption scheme gives a proficient and secure path for unique encrypted fingerprints. Both Equal Error Rate (EER) and Area under Receiver Operating Characteristic (AROC) curve are used for performance evaluation of the proposed cancelable fingerprint recognition scheme revealing high performance.

     

基于多姿态人脸图像合成的识别方法研究

为了解决多姿态人脸识别问题,提出基于独立成分分析（ICA）进行正面人脸合成的新方法。首先利用ICA和PCA提取不同姿态人脸的特征子空间,然后利用通过训练得到的姿态转换矩阵合成其相对应的正面人脸图像,实验表明ICA人脸识别算法要优于PCA人脸识别算法,并在此基础上用小波对人脸图像进行预处理,据姿态转换矩阵得到的正面人脸特征系数直接进行分类比较,识别率得到了很大的提高。     

Integrating face and gait for human recognition at a distance in video.

This paper introduces a new video-based recognition method to recognize noncooperating individuals at a distance in video who expose side views to the camera. Information from two biometrics sources, side face and gait, is utilized and integrated for recognition. For side face, an enhanced side-face image (ESFI), a higher resolution image compared with the image directly obtained from a single video frame, is constructed, which integrates face information from multiple video frames. For gait, the gait energy image (GEI), a spatio-temporal compact representation of gait in video, is used to characterize human-walking properties. The features of face and gait are obtained separately using the principal component analysis and multiple discriminant analysis combined method from ESFI and GEI, respectively. They are then integrated at the match score level by using different fusion strategies. The approach is tested on a database of video sequences, corresponding to 45 people, which are collected over seven months. The different fusion methods are compared and analyzed. The experimental results show that: 1) the idea of constructing ESFI from multiple frames is promising for human recognition in video, and better face features are extracted from ESFI compared to those from the original side-face images (OSFIs); 2) the synchronization of face and gait is not necessary for face template ESFI and gait template GEI; the synthetic match scores combine information from them; and 3) an integrated information from side face and gait is effective for human recognition in video.     

基于足底压力分布时空HOG特征的步态识别方法

提出一种基于足底压力分布时空HOG的步态识别算法,在特征层对足底压力的时间域和空间域信息进行融合。首先寻找足底总压力时间曲线上的极大值和极小值等几个特征点,利用这几个特征点所对应时刻的足底压力分布来构建时空HOG特征向量,最后采用SVM进行步态识别。采集不同行走速度下30人的单步足底压力分布数据进行实验,在不区分样本速度的情况下,该方法的识别率为93。5%。实验结果表明足底压力分布时空HOG特征能较好地刻画步态动力学特征,且具有良好的速度适应性。     

基于多尺度特征迁移学习的步态识别研究

为了解决行人步态数据集样本量较少、单特征或多特征融合的步态识别算法特征描述不足的问题,提出了一种基于多尺度特征深度迁移学习的行人步态识别方法。该算法步骤包括：改进VGG-16网络,去除网络中最后一个最大池化层（Maxpool Layer）,融合空间金字塔池化网络结构（SPP）获取行人步态能量图（GEI）的多尺度信息,利用Imagenet数据集预训练此网络模型,将提取特征能力迁移至行人步态识别网络模型中,采用行人步态样本集微调网络,修改网络中的全连接层参数,应用于行人步态识别研究。该方法在中科院自动化研究所的CASIA-B步态数据集上的识别精度达到了95.7%,与单一步态特征的步态识别方法以及融合多种步态特征的识别方法相比,步态识别率有了明显提升,表明该方法有更好的识别性能。     

A novel ensemble scheme for the multi-focus image fusion using support vector machine

In the multi-focus image fusion problem, the source images are obtained from the same scene. They are fused to get an image that contains all well-focussed objects. Previously, individual machine-learning models are proposed for image fusion. The performance of individual models is limited to fuse the useful information extracted from the blurred images. To address this problem, we developed a novel ensemble scheme for multi-focus image fusion using support vector machines (SVMs). In the proposed scheme, first, SVM models are constructed using different kernel functions of linear, polynomial, radial basis, and sigmoid. The predictions of individual SVM models are then combined using majority voting. In this way, the combined decision space becomes more informative and discriminant. A comparative analysis of the proposed scheme is carried out with previous techniques. It is found that our scheme is more accurate for synthesized-blurred and real defocussed images.