Multi-feature hierarchical topic models for human behavior recognition

Human behavior recognition is one important task of image processing and surveillance system. One main challenge of human behavior recognition is how to effectively model behaviors on condition of unconstrained videos due to tremendous variations from camera motion,background clutter,object appearance and so on. In this paper,we propose two novel Multi-Feature Hierarchical Latent Dirichlet Allocation models for human behavior recognition by extending the bag-of-word topic models such as the Latent Dirichlet Allocation model and the Multi-Modal Latent Dirichlet Allocation model. The two proposed models with three hierarchies including low-level visual features,feature topics,and behavior topics can effectively fuse two different types of features including motion and static visual features,avoid detecting or tracking the motion objects,and improve the recognition performance even if the features are extracted with a great amount of noise. Finally,we adopt the variational EM algorithm to learn the parameters of these models. Experiments on the YouTube dataset demonstrate the effectiveness of our proposed models.     

Utilization of hierarchical, stochastic relationship modeling for Hangul character recognition

In structural character recognition, a character is usually viewed as a set of strokes and the spatial relationships between them. Therefore, strokes and their relationships should be properly modeled for effective character representation. For this purpose, we propose a modeling scheme by which strokes as well as relationships are stochastically represented by utilizing the hierarchical characteristics of target characters. A character is defined by a multivariate random variable over the components and its probability distribution is learned from a training data set. To overcome difficulties of the learning due to the high order of the probability distribution (a problem of curse of dimensionality), the probability distribution is factorized and approximated by a set of lower-order probability distributions by applying the idea of relationship decomposition recursively to components and subcomponents. Based on the proposed method, a handwritten Hangul (Korean) character recognition system is developed. Recognition experiments conducted on a public database show the effectiveness of the proposed relationship modeling. The recognition accuracy increased by 5.5 percent in comparison to the most successful system ever reported.     

Extracting hierarchical spatial and temporal features for human action recognition

Human action recognition is a challenging computer vision task and many efforts have been made to improve the performance. Most previous work has concentrated on the hand-crafted features or spatial-temporal features learned from multiple contiguous frames. In this paper, we present a dual-channel model to decouple the spatial and temporal feature extraction. More specifically, we propose to capture the complementary static form information from single frame and dynamic motion information from multi-frame differences in two separate channels. In both channels we use two stacked classical subspace networks to learn hierarchical representations, which are subsequently fused for action recognition. Our model is trained and evaluated on three typical benchmarks: KTH, UCF and Hollywood2 datasets. The experimental results illustrate that our approach achieves comparable performances to the state-of-the-art methods. In addition, both feature analysis and control experiments are also carried out to demonstrate the effectiveness of the proposed approach for feature extraction and thereby action recognition.     

Efficient simulation of hierarchical stochastic activity network models

Stochastic extensions to Petri nets have gained widespread acceptance as a method for describing the dynamic behavior of discrete-event systems. Both simulation and analytic methods have been proposed to solve such models. This paper describes a set of efficient procedures for simulating models that are represented as stochastic activity networks (SANs, a variant of stochastic Petri nets) and composed SAN-based reward models (SBRMs). Composed SBRMs are a hierarchical representation for SANs, in which individual SAN models can be replicated and joined together with other models, in an iterative fashion. The procedures exploit the hierarchical structure and symmetries introduced by the replicate operation in a composed SBRM to reduce the cost of future event list management. The procedures have been implemented as part of a larger performance-dependability modeling package known asUltraSAN, and have been applied to real, large-scale applications. This work was supported in part by the Digital Equipment Corporation Faculty Program: Incentives for Excellence.     

A hierarchical abnormal human activity recognition system based on R-transform and kernel discriminant analysis for elderly health care

A hierarchical human activity recognition (HAR) system is proposed to recognize abnormal activities from the daily life activities of elderly people living alone. The system is structured to have two-levels of feature extraction and activity recognition. The first level consists of R-transform, kernel discriminant analysis (KDA), $k$ -means algorithm and HMM to recognize the video activity. The second level consists of KDA, $k$ -means algorithm and HMM, and is selectively applied to the recognized activities from the first level when it belongs to the specified group. The proposed hierarchical approach is useful in increasing the recognition rate for the highly similar activities. System performance is analyzed by selecting the optimized number of features, number of HMM states and the number of frames per second to achieve maximum recognition rate. The system is validated by a novel set of six abnormal activities; falling backward, falling forward, chest pain, headache, vomiting, and fainting and a normal activity walking. Experimental results show an average recognition rate of 97.1 % for all the activities by using the proposed hierarchical HAR system.     

Efficient 2D viewpoint combination for human action recognition

The ability to recognize human actions using a single viewpoint is affected by phenomena such as self-occlusions or occlusions by other objects. Incorporating multiple cameras can help overcome these issues. However, the question remains how to efficiently use information from all viewpoints to increase performance. Researchers have reconstructed a 3D model from multiple views to reduce dependency on viewpoint, but this 3D approach is often computationally expensive. Moreover, the quality of each view influences the overall model and the reconstruction is limited to volumes where the views overlap. In this paper, we propose a novel method to efficiently combine 2D data from different viewpoints. Spatio-temporal features are extracted from each viewpoint and then used in a bag-of-words framework to form histograms. Two different sizes of codebook are exploited. The similarity between the obtained histograms is represented via the Histogram Intersection kernel as well as the RBF kernel with \(\chi ^2\) distance. Lastly, we combine all the basic kernels generated by selection of different viewpoints, feature types, codebook sizes and kernel types. The final kernel is a linear combination of basic kernels that are properly weighted based on an optimization process. For higher accuracy, the sets of kernel weights are computed separately for each binary SVM classifier. Our method not only combines the information from multiple viewpoints efficiently, but also improves the performance by mapping features into various kernel spaces. The efficiency of the proposed method is demonstrated by testing on two commonly used multi-view human action datasets. Moreover several experiments indicate the efficacy of each part of the method on the overall performance.     

Online continual learning for human activity recognition

Sensor-based human activity recognition (HAR), with the ability to recognise human activities from wearable or embedded sensors, has been playing an important role in many applications including personal health monitoring, smart home, and manufacturing. The real-world, long-term deployment of these HAR systems drives a critical research question: how to evolve the HAR model automatically over time to accommodate changes in an environment or activity patterns. This paper presents an online continual learning (OCL) scenario for HAR, where sensor data arrives in a streaming manner which contains unlabelled samples from already learnt activities or new activities. We propose a technique, OCL-HAR, making a real-time prediction on the streaming sensor data while at the same time discovering and learning new activities. We have empirically evaluated OCL-HAR on four third-party, publicly available HAR datasets. Our results have shown that this OCL scenario is challenging to state-of-the-art continual learning techniques that have significantly underperformed. Our technique OCL-HAR has consistently outperformed them in all experiment setups, leading up to 0.17 and 0.23 improvements in micro and macro F1 scores.     

Human activity recognition based on a sensor weighting hierarchical classifier

The analysis of daily living human behavior has proven to be of key importance to prevent unhealthy habits. The diversity of activities and the individuals’ particular execution style determine that several sources of information are normally required. One of the main issues is to optimally combine them to guarantee performance, scalability and robustness. In this paper we present a fusion classification methodology which takes into account the potential of the individual decisions yielded at both activity and sensor classification levels. Particularly tested on a wearable sensors based system, the method reinforces the idea that some parts of the body (i.e., sensors) may be specially informative for the recognition of each particular activity, thus supporting the ranking of the decisions provided by each associated sensor decision entity. Our method systematically outperforms the results obtained by traditional multiclass models which otherwise may require a high-dimensional feature space to acquire a similar performance. The comparison with other activity-recognition fusion approaches also demonstrates our model scales significantly better for small sensor networks.     

Discriminative human action recognition in the learned hierarchical manifold space

In this paper, we propose a hierarchical discriminative approach for human action recognition. It consists of feature extraction with mutual motion pattern analysis and discriminative action modeling in the hierarchical manifold space. Hierarchical Gaussian Process Latent Variable Model (HGPLVM) is employed to learn the hierarchical manifold space in which motion patterns are extracted. A cascade CRF is also presented to estimate the motion patterns in the corresponding manifold subspace, and the trained SVM classifier predicts the action label for the current observation. Using motion capture data, we test our method and evaluate how body parts make effect on human action recognition. The results on our test set of synthetic images are also presented to demonstrate the robustness.     

Experiments on human pattern recognition: a hierarchical sign-system approach

Successive stages can be distinguished in the development of the human visual system's ability to use and recognize signs. The stages involve perception of parts of objects, of whole objects, of several objects, and of their interrelations. The system of signs described in this paper was developed through experimental investigations of visual perception in adults, children, and mentally ill or brain-damaged persons.     

Efficient fall activity recognition by combining shape and motion features

Computational Visual Media - This paper presents a vision-based system for recognizing when elderly adults fall. A fall is characterized by shape deformation and high motion. We represent shape...     

A framework for heading-guided recognition of human activity

A combined 2D, 3D approach is presented that allows for robust tracking of moving people and recognition of actions. It is assumed that the system observes multiple moving objects via a single, uncalibrated video camera. Low-level features are often insufficient for detection, segmentation, and tracking of non-rigid moving objects. Therefore, an improved mechanism is proposed that integrates low-level (image processing), mid-level (recursive 3D trajectory estimation), and high-level (action recognition) processes. A novel extended Kalman filter formulation is used in estimating the relative 3D motion trajectories up to a scale factor. The recursive estimation process provides a prediction and error measure that is exploited in higher-level stages of action recognition. Conversely, higher-level mechanisms provide feedback that allows the system to reliably segment and maintain the tracking of moving objects before, during, and after occlusion. Heading-guided recognition (HGR) is proposed as an efficient method for adaptive classification of activity. The HGR approach is demonstrated using “motion history images” that are then recognized via a mixture-of-Gaussians classifier. The system is tested in recognizing various dynamic human outdoor activities: running, walking, roller blading, and cycling. In addition, experiments with real and synthetic data sets are used to evaluate stability of the trajectory estimator with respect to noise.     

Wireless image fuzzy recognition system for human activity

Multimedia Tools and Applications - This study developed a fuzzy image model system for transmitting data over a wireless network channel to efficiently realize human activity in virtual images...     

A hierarchical approach to real-time activity recognition in body sensor networks

Real-time activity recognition in body sensor networks is an important and challenging task. In this paper, we propose a real-time, hierarchical model to recognize both simple gestures and complex activities using a wireless body sensor network. In this model, we first use a fast and lightweight algorithm to detect gestures at the sensor node level, and then propose a pattern based real-time algorithm to recognize complex, high-level activities at the portable device level. We evaluate our algorithms over a real-world dataset. The results show that the proposed system not only achieves good performance (an average utility of 0.81, an average accuracy of 82.87%, and an average real-time delay of 5.7 seconds), but also significantly reduces the network’s communication cost by 60.2%.     

基于分层识别的快速说话人识别研究

随着说话人模型数量的增加,说话人识别系统的识别速度下降,不能满足实时性要求。针对这个问题,提出了基于分层识别模型的快速说话人识别方法。将变分法求解的KL散度的近似值作为模型间的相似性度量准则,并设计了说话人模型聚类的方法。结果表明,本文方法能够保证说话人模型聚类结果的有效性,在系统识别率损失很小的情况下,使系统的识别速度得到大幅度提升。     

A hierarchical representation for heterogeneous object modeling

A hierarchical representation for heterogeneous object modeling is presented in this paper. To model a heterogeneous object, Boundary representation is used for geometry representation, and a novel Heterogeneous Feature Tree (HFT) structure is proposed to represent the material distributions. HFT structure hierarchically organizes the material variation dependency relationships and is intuitive in modeling different types of material gradations. Based on the HFT structure, a recursive material evaluation algorithm is proposed to dynamically evaluate the material compositions at a specific location. Such a hierarchical representation guarantees complex material gradations and the user's design intent can be intuitively represented. Example heterogeneous objects modeled with this scheme are provided and potential applications are discussed.