Towards automated visual surveillance using gait for identity recognition and tracking across multiple non-intersecting cameras

Despite the fact that personal privacy has become a major concern, surveillance technology is now becoming ubiquitous in modern society. This is mainly due to the increasing number of crimes as well as the essential necessity to provide secure and safer environment. Recent research studies have confirmed now the possibility of recognizing people by the way they walk i.e. gait. The aim of this research study is to investigate the use of gait for people detection as well as identification across different cameras. We present a new approach for people tracking and identification between different non-intersecting un-calibrated stationary cameras based on gait analysis. A vision-based markerless extraction method is being deployed for the derivation of gait kinematics as well as anthropometric measurements in order to produce a gait signature. The novelty of our approach is motivated by the recent research in biometrics and forensic analysis using gait. The experimental results affirmed the robustness of our approach to successfully detect walking people as well as its potency to extract gait features for different camera viewpoints achieving an identity recognition rate of 73.6 % processed for 2270 video sequences. Furthermore, experimental results confirmed the potential of the proposed method for identity tracking in real surveillance systems to recognize walking individuals across different views with an average recognition rate of 92.5 % for cross-camera matching for two different non-overlapping views.     

视频群体行为识别综述

群体行为识别是指给定一个包含多人场景的视频,模型需要识别出视频中多个人物正在共同完成的群体行为.群体行为识别是视频理解中的一个重要问题,可以被应用在运动比赛视频分析、监控视频识别、社交行为理解等现实场景中.多人场景视频较为复杂,时间和空间上的信息十分丰富,对模型提取关键信息的能力要求更高.模型只有高效地建模场景中的层次化关系,并为人物群体提取有区分性的时空特征,才能准确地识别出群体行为.由于其广泛的应用需求,群体行为识别问题受到了研究人员的广泛关注.对近几年来群体行为识别问题上的大量研究工作进行了深入分析,总结出了群体行为识别研究所面临的主要挑战,系统地归纳出了6种类型的群体行为识别方法,包含传统非深度学习识别方法以及基于深度学习技术的识别方法,并对未来研究的可能方向进行了展望.     

Performing content-based retrieval of humans using gait biometrics

In order to analyse surveillance video, we need to efficiently explore large datasets containing videos of walking humans. Effective analysis of such data relies on retrieval of video data which has been enriched using semantic annotations. A manual annotation process is time-consuming and prone to error due to subject bias however, at surveillance-image resolution, the human walk (their gait) can be analysed automatically. We explore the content-based retrieval of videos containing walking subjects, using semantic queries. We evaluate current research in gait biometrics, unique in its effectiveness at recognising people at a distance. We introduce a set of semantic traits discernible by humans at a distance, outlining their psychological validity. Working under the premise that similarity of the chosen gait signature implies similarity of certain semantic traits we perform a set of semantic retrieval experiments using popular Latent Semantic Analysis techniques. We perform experiments on a dataset of 2000 videos of people walking in laboratory conditions and achieve promising retrieval results for features such as Sex (mAP  =  14% above random), Age (mAP  =  10% above random) and Ethnicity (mAP  =  9% above random).     

视频监控场景下基于单视角步态的人体身份及属性识别系统

基于步态的特征识别是一种新兴的生物特征识别技术, 旨在通过人们走路的姿态进行身份和相关属性的分析. 与其他的生物识别技术相比, 基于步态的识别方法具有难隐藏性、非接触性和可远距离使用的优点. 本文设计出一个视频监控场景下基于单视角步态的人体身份及属性识别系统, 该系统通过图像处理方法从复杂的监控视频中实时检测出人体的步态, 经过利用深度学习训练过的算法进行分析后, 获取人体的身份、性别和年龄信息. 实验表明, 系统的身份识别准确率达98.1%, 性别预测准确率达97.1%, 年龄预测平均绝对误差为6.21岁, 实验结果均优于传统基准算法, 且系统开发成本低, 支持实时检测, 能充分满足中小规模步态研究与分析的需要.     

基于肢体长度参数的多视角步态识别算法

提出了一种基于肢体长度参数的用于多视角情况下的步态识别算法. 利用脚间距计算方法和动态身体分割方法，拟合出场景的转换参数，并以此估计出人运动情况下的5 个肢体长度参数，使用标准的分类器进行分类. 在中科院自动化所提供的NLPR 数据库上做了大量仿真实验，结果表明本方法在人的行走方向相对于摄像机光轴的角度改变时仍然保持了较高的识别率.     

基于SVM的步态识别方法综述

步态识别是非接触式生物特征识别领域的前沿课题,通过对人体行走方式的识别以确定个体的身份,在智能视频监控领域有较高的研究价值.步态分类是步态识别过程中的重要任务和关键步骤.首先概述了步态识别过程及分类方法,然后重点对基于支持向量机的步态分类方法进行了综述,分析了基于该方法的最新研究进展,对每个具体研究方法的优缺点进行了对比.最后,指出目前步态识别在实际应用中存在的局限性,并对该领域发展方向进行了展望.     

步态识别综述

步态识别作为一种新的生物认证技术,通过人的行走方式识别个体的身份。本文首先分析了步态识别的特点及其研究背景;其次,对目前步态识别中涉及到的几项关键技术所采用的方法进行了简单的描述及分类,并分析各种技术的主要优缺点,最后,提出了目前步态识别问题研究中的几个薄弱点,指出进一步研究发展空间。     

A comprehensive review of past and present vision-based techniques for gait recognition

Global security concerns have raised a proliferation of video surveillance devices. Intelligent surveillance systems seek to discover possible threats automatically and raise alerts. Being able to identify the surveyed object can help determine its threat level. The current generation of devices provide digital video data to be analysed for time varying features to assist in the identification process. Commonly, people queue up to access a facility and approach a video camera in full frontal view. In this environment, a variety of biometrics are available—for example, gait which includes temporal features like stride period. Gait can be measured unobtrusively at a distance. The video data will also include face features, which are short-range biometrics. In this way, one can combine biometrics naturally using one set of data. In this paper we survey current techniques of gait recognition and modelling with the environment in which the research was conducted. We also discuss in detail the issues arising from deriving gait data, such as perspective and occlusion effects, together with the associated computer vision challenges of reliable tracking of human movement. Then, after highlighting these issues and challenges related to gait processing, we proceed to discuss the frameworks combining gait with other biometrics. We then provide motivations for a novel paradigm in biometrics-based human recognition, i.e. the use of the fronto-normal view of gait as a far-range biometrics combined with biometrics operating at a near distance.     

Neural computing for walking gait pattern identification based on multi-sensor data fusion of lower limb muscles

The use of neural computing for gait analysis widely known as computational intelligent gait analysis is addressed recently. This research work reports multilayer feed-forward neural networks for walking gait pattern identification using multi-sensor data fusion; electromyography (EMG) signals and soft tissue deformation analysis using successive frames of video sequence extracted from lower limb muscles according to each gait phase within the considered gait cycle. Neural computing framework for walking gait pattern identification consists of system hardware and intelligent system software. System hardware comprises a wireless surface EMG sensor unit and two video cameras for measuring the neuromuscular activity of lower limb muscles, and a custom-developed artificial neural network for classifying the gait patterns of subjects during walking. The system uses root mean square and soft tissue deformation parameter as the input features. Multilayer feed-forward back propagation neural networks (FFBPNNs) with different network training functions were designed, and their classification results were compared. The intelligent gait analysis system validation has been carried out for a group of healthy and injured subjects. The results demonstrated that the overall accuracy of 98 % prediction is achieved for gait patterns classification established by multi-sensor data fusion of lower limb muscles using FFBPNN with Levenberg–Marquardt training function resulting better performance over FFBPNN with other training functions.

     

Integrating face and gait for human recognition at a distance in video.

This paper introduces a new video-based recognition method to recognize noncooperating individuals at a distance in video who expose side views to the camera. Information from two biometrics sources, side face and gait, is utilized and integrated for recognition. For side face, an enhanced side-face image (ESFI), a higher resolution image compared with the image directly obtained from a single video frame, is constructed, which integrates face information from multiple video frames. For gait, the gait energy image (GEI), a spatio-temporal compact representation of gait in video, is used to characterize human-walking properties. The features of face and gait are obtained separately using the principal component analysis and multiple discriminant analysis combined method from ESFI and GEI, respectively. They are then integrated at the match score level by using different fusion strategies. The approach is tested on a database of video sequences, corresponding to 45 people, which are collected over seven months. The different fusion methods are compared and analyzed. The experimental results show that: 1) the idea of constructing ESFI from multiple frames is promising for human recognition in video, and better face features are extracted from ESFI compared to those from the original side-face images (OSFIs); 2) the synchronization of face and gait is not necessary for face template ESFI and gait template GEI; the synthetic match scores combine information from them; and 3) an integrated information from side face and gait is effective for human recognition in video.     

Robust Gait Recognition by Learning and Exploiting Sub-gait Characteristics

Gait recognition algorithms often perform poorly because of low resolution video sequences, subjective human motion and challenging outdoor scenarios. Despite these challenges, gait recognition research is gaining momentum due to increasing demand and more possibilities for deployment by the surveillance industry. Therefore every research contribution which significantly improves this new biometric is a milestone. We propose a probabilistic sub-gait interpretation model to recognize gaits. A sub-gait is defined by us as part of the silhouette of a moving body. Binary silhouettes of gait video sequences form the basic input of our approach. A novel modular training scheme has been introduced in this research to efficiently learn subtle sub-gait characteristics from the gait domain. For a given gait sequence, we get useful information from the sub-gaits by identifying and exploiting intrinsic relationships using Bayesian networks. Finally, by incorporating efficient inference strategies, robust decisions are made for recognizing gaits. Our results show that the proposed model tackles well the uncertainties imposed by typical covariate factors and shows significant recognition performance.     

Accurate tracking of legged robots on natural terrain

Statically stable walking locomotion research has focused mainly on robot design and gait generation. However, there is a need to expand robots’ capabilities so that walking machines can accomplish the kinds of real tasks for which they are eminently suited. Many such tasks demand trajectory tracking, but researchers have traditionally ignored this subject. This article focuses on the tracking of predefined trajectories with hexapod robots walking on natural terrain with forbidden zones. The method presented herein, which relies on gait algorithms defined elsewhere, describes certain localization strategies and control techniques that have been employed to follow trajectories accurately and have been implemented in a real walking hexapod. Several experimental examples are included to assess the proposed algorithms.     

行人步态的特征表达及识别综述

从人体测量学数据、空间时间数据、运动学数据、动力学数据和视频流数据的特有方法总结步态识别的各种方法。对视频流的方法做了更为详尽的分析,特别是对类能量图法中各种方法的表达式和含义、蕴含的信息是静态还是动态、时变特性进行对比研究。综述步态自身的多种特征融合的识别方法、步态与其它的生物特征融合的识别方法以及“扩展的”步态识别等方面的研究现状,给出现有步态识别研究的评估方法,指出未来步态识别的研究方向。     

一种步态识别方法

步态识别作为一种新的生物特征识别技术,通过人走路的姿势实现对个人身份的识别和认证.算法利用步态轮廓图像边界到重心的距离矢量对步态轮廓图像进行描述,采用步态图像的高宽比进行步态的准周期性分析.利用隐马尔可夫模型进行步态时变数据匹配识别.算法在CMU数据库上面进行实验取得了较高的正确识别率.     

Spatio-Temporal Tube data representation and Kernel design for SVM-based video object retrieval system

In this article, we propose a new video object retrieval system. Our approach is based on a Spatio-Temporal data representation, a dedicated kernel design and a statistical learning toolbox for video object recognition and retrieval. Using state-of-the-art video object detection algorithms (for faces or cars, for example) we segment video object tracks from real movies video shots. We then extract, from these tracks, sets of spatio-temporally coherent features that we call Spatio-Temporal Tubes. To compare these complex tube objects, we design a Spatio-Temporal Tube Kernel (STTK) function. Based on this kernel similarity we present both supervised and active learning strategies embedded in Support Vector Machine framework. Additionally, we propose a multi-class classification framework dealing with unbalanced data. Our approach is successfully evaluated on two real movies databases, the french movie “L’esquive” and episodes from “Buffy, the Vampire Slayer” TV series. Our method is also tested on a car database (from real movies) and shows promising results for car identification task.     

基于GVF Snake模型的步态识别轮廓提取

步态识别作为一种新的生物识别技术,通过人走路的姿势实现对个人身份的识别和认证。步态特征提取是步态识别的关键步骤。采用背景消减法与对称差分法相结合对运动人体分割,采用改进的GVF Snake模型对人体运动步态轮廓进行边缘提取。实验结果表明该方法能准确高效地提取边缘特征作为步态识别的特征。     

监狱视频识别报警系统的研究与实现

监狱视频监控系统中视频监控、识别、智能报警等形成一个整体并实现联动,真正形成报警系统的整体解决方案。该文拟在具体监狱实际情况和视频图像识别技术的基础上,对基于神经网络分类器和智能终端在监狱报警系统中的应用研究,重点研究了基于BP神经网络的视频图像识别方法。通过实现监狱监控、识别和智能报警一体化,降低了监管人员工作负荷,提高监狱科学管理水平,从而保证监狱安全,减少越狱等恶性事件发生,提高对犯人的监督和改造效果。     

Gait analysis for human identification through manifold learning and HMM

With the increasing demands of visual surveillance systems, human identification at a distance has gained more attention from the researchers recently. Gait analysis can be used as an unobtrusive biometric measure to identify people at a distance without any attention of the human subjects. We propose a novel effective method for both automatic viewpoint and person identification by using only the silhouette sequence of the gait. The gait silhouettes are nonlinearly transformed into low-dimensional embedding by Gaussian process latent variable model (GP-LVM), and the temporal dynamics of the gait sequences are modeled by hidden Markov models (HMMs). The experimental results show that our method has higher recognition rate than the other methods.