A framework for online gait recognition based on multilinear tensor analysis

The gait recognition is to recognize an individual based on the characteristics extracted from the gait image sequence. There are many researches for the gait recognition which use diverse kinds of information such as shape of gait silhouette, motion variation caused by walking, and so on. In general, shape information is more useful for recognition. However, shape information is influenced by a variety of factors, which degrade the recognition performance. Moreover, the information used in most of those studies might be able to be extracted after all of one or more sequences of the gait cycle are known. And it is also hard to discriminate the gait cycle from given gait sequences exactly by the online approach. In regard to these difficulties, we propose a novel gait recognition method based on the multilinear tensor analysis. To recognize the cyclic characteristic of gait without an exact division for the gait cycle, this paper’s propose is the method to form the accumulated silhouette and then describes those as the tensor. For the accumulated silhouette proposed by this paper, the image sequence of one gait cycle is divided into four sections in the training phase. However, discrimination for the gait cycle in the training phase is not directly related to the recognition phase, thus the online approach is possible. We first form the accumulated silhouettes for every individual using gait silhouettes within each section. And then, we represent these accumulated silhouettes as the tensor. Using a multilinear tensor analysis, we compute the core tensor which governs the interaction between factors organizing the original tensor, and then compose the basis to recognize the individual in the online recognition framework. Finally, we recognize the individual using the computation of similarity based on the Euclidean distance, which is more suitable to our method. We verify the superiority of the proposed approach via experiments with real gait sequences.     

Silhouette-based gait recognition via deterministic learning

In this paper, we present a new silhouette-based gait recognition method via deterministic learning theory, which combines spatio-temporal motion characteristics and physical parameters of a human subject by analyzing shape parameters of the subject?s silhouette contour. It has been validated only in sequences with lateral view, recorded in laboratory conditions. The ratio of the silhouette?s height and width (H–W ratio), the width of the outer contour of the binarized silhouette, the silhouette area and the vertical coordinate of centroid of the outer contour are combined as gait features for recognition. They represent the dynamics of gait motion and can more effectively reflect the tiny variance between different gait patterns. The gait recognition approach consists of two phases: a training phase and a test phase. In the training phase, the gait dynamics underlying different individuals? gaits are locally accurately approximated by radial basis function (RBF) networks via deterministic learning theory. The obtained knowledge of approximated gait dynamics is stored in constant RBF networks. In the test phase, a bank of dynamical estimators is constructed for all the training gait patterns. The constant RBF networks obtained from the training phase are embedded in the estimators. By comparing the set of estimators with a test gait pattern, a set of recognition errors are generated, and the average L₁ norms of the errors are taken as the similarity measure between the dynamics of the training gait patterns and the dynamics of the test gait pattern. The test gait pattern similar to one of the training gait patterns can be recognized according to the smallest error principle. Finally, the recognition performance of the proposed algorithm is comparatively illustrated to take into consideration the published gait recognition approaches on the most well-known public gait databases: CASIA, CMU MoBo and TUM GAID.     

A Bayesian framework for extracting human gait using strong prior knowledge

Extracting full-body motion of walking people from monocular video sequences in complex, real-world environments is an important and difficult problem, going beyond simple tracking, whose satisfactory solution demands an appropriate balance between use of prior knowledge and learning from data. We propose a consistent Bayesian framework for introducing strong prior knowledge into a system for extracting human gait. In this work, the strong prior is built from a simple articulated model having both time-invariant (static) and time-variant (dynamic) parameters. The model is easily modified to cater to situations such as walkers wearing clothing that obscures the limbs. The statistics of the parameters are learned from high-quality (indoor laboratory) data and the Bayesian framework then allows us to "bootstrap" to accurate gait extraction on the noisy images typical of cluttered, outdoor scenes. To achieve automatic fitting, we use a hidden Markov model to detect the phases of images in a walking cycle. We demonstrate our approach on silhouettes extracted from fronto-parallel ("sideways on") sequences of walkers under both high-quality indoor and noisy outdoor conditions. As well as high-quality data with synthetic noise and occlusions added, we also test walkers with rucksacks, skirts, and trench coats. Results are quantified in terms of chamfer distance and average pixel error between automatically extracted body points and corresponding hand-labeled points. No one part of the system is novel in itself, but the overall framework makes it feasible to extract gait from very much poorer quality image sequences than hitherto. This is confirmed by comparing person identification by gait using our method and a well-established baseline recognition algorithm     

Performing content-based retrieval of humans using gait biometrics

In order to analyse surveillance video, we need to efficiently explore large datasets containing videos of walking humans. Effective analysis of such data relies on retrieval of video data which has been enriched using semantic annotations. A manual annotation process is time-consuming and prone to error due to subject bias however, at surveillance-image resolution, the human walk (their gait) can be analysed automatically. We explore the content-based retrieval of videos containing walking subjects, using semantic queries. We evaluate current research in gait biometrics, unique in its effectiveness at recognising people at a distance. We introduce a set of semantic traits discernible by humans at a distance, outlining their psychological validity. Working under the premise that similarity of the chosen gait signature implies similarity of certain semantic traits we perform a set of semantic retrieval experiments using popular Latent Semantic Analysis techniques. We perform experiments on a dataset of 2000 videos of people walking in laboratory conditions and achieve promising retrieval results for features such as Sex (mAP  =  14% above random), Age (mAP  =  10% above random) and Ethnicity (mAP  =  9% above random).     

不变矩系数拟合的步态识别

根据综合利用步态的静态和动态信息的思想，结合不变矩描述图像几何特性的功能，从步态序列提取不变矩作为步态特征进行识别。采用傅立叶级数描述步态图像序列人体轮廓不变矩的变化，利用遗传算法搜索傅立叶级数的系数，最后再用k近邻分类器对不变矩变化的幅度信息分类。在CMU步态数据库上进行的实验，达到了90%以上的识别率。结果表明，该方法具备很高识别性能，能较好地利用步态的静态和动态信息。     

Deep mutual learning network for gait recognition

Human identification plays a significant role in ensuring social security. However, face-based and appearance-based retrieval methods are not effective in monitoring due to the long distance and low camera resolution. Compared with other biological characteristics, the gait of humans has a strong discriminating ability even at long distance and low resolution. In this paper, the deep mutual learning strategy is applied to gait recognition, and by training collaboratively with other networks, the generalization ability of the network is improved simply and effectively. We use a set of independent frames of gait as input to two convolutional neural networks. This method is unaffected by frame alignment and can naturally integrate video frames of different walking conditions (e.g. different viewing angles, different clothing/carrying conditions). At the same time, the set can extract gait features from incomplete gait cycles due to occlusion. A mutual learning strategy can improve the running speed appropriately and realize the compactness and accuracy of the model. Two convolutional networks learn simultaneously and solve problems together. To evaluate the method’s performance, we compare it to several methods on the CASIA and OU-ISIR gait databases, and construct different sets of gaits with incomplete periods to compare the accuracy of identification with them and the complete gait set. Experimental results show that the method is effective.

     

基于步态识别的远距离身份认证

通过人走路的姿势实现对个人身份的远距离识别和认证是当前生物特征识别研究领域的一个研究热点。算法利用步态轮廓图像边界到重心的距离矢量对步态轮廓图像进行人体运动的静态形状描述,采用连续隐马尔可夫模型对人体运动时从一个动作到另一个动作的过渡进行动态描述。算法在CMU数据库上面进行实验取得了较高的正确识别率。     

The humanID gait challenge problem: data sets, performance, and analysis

Identification of people by analysis of gait patterns extracted from video has recently become a popular research problem. However, the conditions under which the problem is "solvable" are not understood or characterized. To provide a means for measuring progress and characterizing the properties of gait recognition, we introduce the humanlD gait challenge problem. The challenge problem consists of a baseline algorithm, a set of 12 experiments, and a large data set. The baseline algorithm estimates silhouettes by background subtraction and performs recognition by temporal correlation of silhouettes. The 12 experiments are of increasing difficulty, as measured by the baseline algorithm, and examine the effects of five covariates on performance. The covariates are: change in viewing angle, change in shoe type, change in walking surface, carrying or not carrying a briefcase, and elapsed time between sequences being compared. Identification rates for the 12 experiments range from 78 percent on the easiest experiment to 3 percent on the hardest. All five covariates had statistically significant effects on performance, with walking surface and time difference having the greatest impact. The data set consists of 1,870 sequences from 122 subjects spanning five covariates (1.2 gigabytes of data). This infrastructure supports further development of gait recognition algorithms and additional experiments to understand the strengths and weaknesses of new algorithms. The experimental results are presented, the more detailed is the possible meta-analysis and greater is the understanding. It is this potential from the adoption of this challenge problem that represents a radical departure from traditional computer vision research methodology.     

Human Gait Recognition Based on Kernel PCA Using Projections

This paper presents a novel approach for human identification at a distance using gait recognition. Recognition of a person from their gait is a biometric of increasing interest. The proposed work introduces a nonlinear machine learning method, kernel Principal Component Analysis （PCA）, to extract gait features from silhouettes for individual recognition. Binarized silhouette of a motion object is first represented by four 1-D signals which are the basic image features called the distance vectors. Fourier transform is performed to achieve translation invariant for the gait patterns accumulated from silhouette sequences which are extracted from different circumstances. Kernel PCA is then used to extract higher order relations among the gait patterns for future recognition. A fusion strategy is finally executed to produce a final decision. The experiments are carried out on the CMU and the USF gait databases and presented based on the different training gait cycles.     

Clothing-invariant gait identification using part-based clothing categorization and adaptive weight control

Variations in clothing alter an individual's appearance, making the problem of gait identification much more difficult. If the type of clothing differs between the gallery and a probe, certain parts of the silhouettes are likely to change and the ability to discriminate subjects decreases with respect to these parts. A part-based approach, therefore, has the potential of selecting the appropriate parts. This paper proposes a method for part-based gait identification in the light of substantial clothing variations. We divide the human body into eight sections, including four overlapping ones, since the larger parts have a higher discrimination capability, while the smaller parts are more likely to be unaffected by clothing variations. Furthermore, as there are certain clothes that are common to different parts, we present a categorization for items of clothing that groups similar clothes. Next, we exploit the discrimination capability as a matching weight for each part and control the weights adaptively based on the distribution of distances between the probe and all the galleries. The results of the experiments using our large-scale gait dataset with clothing variations show that the proposed method achieves far better performance than other approaches.     

Silhouette analysis-based gait recognition for human identification

Human identification at a distance has recently gained growing interest from computer vision researchers. Gait recognition aims essentially to address this problem by identifying people based on the way they walk. In this paper, a simple but efficient gait recognition algorithm using spatial-temporal silhouette analysis is proposed. For each image sequence, a background subtraction algorithm and a simple correspondence procedure are first used to segment and track the moving silhouettes of a walking figure. Then, eigenspace transformation based on principal component analysis (PCA) is applied to time-varying distance signals derived from a sequence of silhouette images to reduce the dimensionality of the input feature space. Supervised pattern classification techniques are finally performed in the lower-dimensional eigenspace for recognition. This method implicitly captures the structural and transitional characteristics of gait. Extensive experimental results on outdoor image sequences demonstrate that the proposed algorithm has an encouraging recognition performance with relatively low computational cost.     

Nonlinear Dynamical Shape Priors for Level Set Segmentation

The introduction of statistical shape knowledge into level set based segmentation methods was shown to improve the segmentation of familiar structures in the presence of noise, clutter or partial occlusions. While most work has been focused on shape priors which are constant in time, it is clear that when tracking deformable shapes certain silhouettes may become more or less likely over time. In fact, the deformations of familiar objects such as the silhouettes of a walking person are often characterized by pronounced temporal correlations. In this paper, we propose a nonlinear dynamical shape prior for level set based image segmentation. Specifically, we propose to approximate the temporal evolution of the eigenmodes of the level set function by means of a mixture of autoregressive models. We detail how such shape priors “with memory” can be integrated into a variational framework for level set segmentation. As an application, we experimentally validate that the nonlinear dynamical prior drastically improves the tracking of a person walking in different directions, despite large amounts of clutter and noise.     

Improved gait recognition by multiple-projections normalization

Recognizing people by gait promises to be useful for identifying individuals from a distance; in this regard, improved techniques are under development. In this paper, an improved method for gait recognition is proposed. Binarized silhouette of a motion object is first represented by four 1-D signals that are the basic image features called the distance vectors. The distance vectors are differences between the bounding box and silhouette, and extracted using four projections to silhouette. Fourier Transform is employed as a preprocessing step to achieve translation invariant for the gait patterns accumulated from silhouette sequences that are extracted from the subjects’ walk in different speed and/or different time. Then, eigenspace transformation is applied to reduce the dimensionality of the input feature space. Support vector machine (SVM)-based pattern classification technique is then performed in the lower-dimensional eigenspace for recognition. The input feature space is alternatively constructed by using two different approaches. The four projections (1-D signals) are independently classified in the first approach. A fusion task is then applied to produce the final decision. In the second approach, the four projections are concatenated to have one vector and then pattern classification with one vector is performed in the lower-dimensional eigenspace for recognition. The experiments are carried out on the most well-known public gait databases: the CMU, the USF, SOTON, and NLPR human gait databases. To effectively understand the performance of the algorithm, the experiments are executed and presented as increasing amounts of the gait cycles of each person available during the training procedure. Finally, the performance of the proposed algorithm is comparatively illustrated to take into consideration the published gait recognition approaches.     

Towards view-invariant gait modeling: Computing view-normalized body part trajectories

This paper proposes an approach to compute view-normalized body part trajectories of pedestrians walking on potentially non-linear paths. The proposed approach finds applications in gait modeling, gait biometrics, and in medical gait analysis. Our approach uses the 2D trajectories of both feet and the head extracted from the tracked silhouettes. On that basis, it computes the apparent walking (sagittal) planes for each detected gait half-cycle. A homography transformation is then computed for each walking plane to make it appear as if walking was observed from a fronto-parallel view. Finally, each homography is applied to head and feet trajectories over each corresponding gait half-cycle. View normalization makes head and feet trajectories appear as if seen from a fronto-parallel viewpoint, which is assumed to be optimal for gait modeling purposes. The proposed approach is fully automatic as it requires neither manual initialization nor camera calibration. An extensive experimental evaluation of the proposed approach confirms the validity of the normalization process.     

Automatic detection of abnormal gait

Analysing human gait has found considerable interest in recent computer vision research. So far, however, contributions to this topic exclusively dealt with the tasks of person identification or activity recognition. In this paper, we consider a different application for gait analysis and examine its use as a means of deducing the physical well-being of people. Understanding the detection of unusual movement patterns as a two-class problem suggests using support vector machines for classification. We present a homeomorphisms between 2D lattices and binary shapes that provides a robust vector space embedding of segmented body silhouettes. Experimental results demonstrate that feature vectors obtained from this scheme are well suited to detect abnormal gait. Wavering, faltering, and falling can be detected reliably across individuals without tracking or recognising limbs or body parts.     

Performance analysis for automated gait extraction and recognition in multi-camera surveillance

Many studies have confirmed that gait analysis can be used as a new biometrics. In this research, gait analysis is deployed for people identification in multi-camera surveillance scenarios. We present a new method for viewpoint independent markerless gait analysis that does not require camera calibration and works with a wide range of walking directions. These properties make the proposed method particularly suitable for gait identification in real surveillance scenarios where people and their behaviour need to be tracked across a set of cameras. Tests on 300 synthetic and real video sequences, with subjects walking freely along different walking directions, have been performed. Since the choice of the cameras’ characteristics is a key-point for the development of a smart surveillance system, the performance of the proposed approach is measured with respect to different video properties: spatial resolution, frame-rate, data compression and image quality. The obtained results show that markerless gait analysis can be achieved without any knowledge of camera’s position and subject’s pose. The extracted gait parameters allow recognition of people walking from different views with a mean recognition rate of 92.2% and confirm that gait can be effectively used for subjects’ identification in a multi-camera surveillance scenario.     

基于连续隐马尔可夫模型的步态识别

步态识别作为一种新的生物特征识别技术,通过人走路的姿势实现对个人身份的识别和认证.算法利用步态轮廓图像边界到重心的距离矢量对步态轮廓图像进行描述,采用步态图像的高宽比进行步态的准周期性分析.利用隐马尔可夫模型进行步态时变数据匹配识别.算法在CMU数据库上进行实验取得了较高的正确识别率.     

一种步态识别方法

步态识别作为一种新的生物特征识别技术,通过人走路的姿势实现对个人身份的识别和认证.算法利用步态轮廓图像边界到重心的距离矢量对步态轮廓图像进行描述,采用步态图像的高宽比进行步态的准周期性分析.利用隐马尔可夫模型进行步态时变数据匹配识别.算法在CMU数据库上面进行实验取得了较高的正确识别率.