An activity monitoring system for elderly care using generative and discriminative models

An activity monitoring system allows many applications to assist in care giving for elderly in their homes. In this paper we present a wireless sensor network for unintrusive observations in the home and show the potential of generative and discriminative models for recognizing activities from such observations. Through a large number of experiments using four real world datasets we show the effectiveness of the generative hidden Markov model and the discriminative conditional random fields in activity recognition.     

基于函数型数据时间序列建模的单传感器日常行为识别

在基于惯性传感器的人体行为识别中,传统算法常忽略行为的周期性与时序性,对提取特征的滑动窗口大小也有相应要求.文中基于单个腰部传感器分析人体日常行为,提出面向周期行为的函数型数据分析方法和隐马尔可夫模型结合的行为识别算法.首先,使用函数型数据分析方法,拟合周期性日常行为的动作捕捉数据,提取拟合后的单个周期数据.然后基于此周期时间序列数据建立描述各个日常行为过程的隐马尔可夫模型.最后,使用最大似然估计判别行为,得到识别结果.该算法通过单个腰部传感器即可快速有效地识别8种日常行为,在基于用户依赖策略和用户独立策略时识别率较高.     

Training data selection with user’s physical characteristics data for acceleration-based activity modeling

This paper proposes an activity recognition method that models an end user’s activities without using any labeled/unlabeled acceleration sensor data obtained from the user. Our method employs information about the end user’s physical characteristics such as height and gender to find and select appropriate training data obtained from other users in advance. Then, we model the end user’s activities by using the selected labeled sensor data. Therefore, our method does not require the end user to collect and label her training sensor data. In this paper, we propose and test two methods for finding appropriate training data by using information about the end user’s physical characteristics. Moreover, our recognition method improves the recognition performance without the need for any effort by the end user because the method automatically adapts the activity models to the end user when it recognizes her unlabeled sensor data. We confirmed the effectiveness of our method by using 100 h of sensor data obtained from 40 participants.     

Conditional models for contextual human motion recognition

We describe algorithms for recognizing human motion in monocular video sequences, based on discriminative conditional random fields (CRFs) and maximum entropy Markov models (MEMMs). Existing approaches to this problem typically use generative structures like the hidden Markov model (HMM). Therefore, they have to make simplifying, often unrealistic assumptions on the conditional independence of observations given the motion class labels and cannot accommodate rich overlapping features of the observation or long-term contextual dependencies among observations at multiple timesteps. This makes them prone to myopic failures in recognizing many human motions, because even the transition between simple human activities naturally has temporal segments of ambiguity and overlap. The correct interpretation of these sequences requires more holistic, contextual decisions, where the estimate of an activity at a particular timestep could be constrained by longer windows of observations, prior and even posterior to that timestep. This would not be computationally feasible with a HMM which requires the enumeration of a number of observation sequences exponential in the size of the context window. In this work we follow a different philosophy: instead of restrictively modeling the complex image generation process – the observation, we work with models that can unrestrictedly take it as an input, hence condition on it. Conditional models like the proposed CRFs seamlessly represent contextual dependencies and have computationally attractive properties: they support efficient, exact recognition using dynamic programming, and their parameters can be learned using convex optimization. We introduce conditional graphical models as complementary tools for human motion recognition and present an extensive set of experiments that show not only how these can successfully classify diverse human activities like walking, jumping, running, picking or dancing, but also how they can discriminate among subtle motion styles like normal walks and wander walks.     

Learning Variable-Length Markov Models of Behavior

In recent years there has been an increased interest in the modeling and recognition of human activities involving highly structured and semantically rich behavior such as dance, aerobics, and sign language. A novel approach for automatically acquiring stochastic models of the high-level structure of an activity without the assumption of any prior knowledge is presented. The process involves temporal segmentation into plausible atomic behavior components and the use of variable-length Markov models for the efficient representation of behaviors. Experimental results that demonstrate the synthesis of realistic sample behaviors and the performance of models for long-term temporal prediction are presented.     

Recognizing composite daily activities from crowd-labelled social media data

Human activity recognition is a core component of context-aware, ubiquitous computing systems. Traditionally, this task is accomplished by analysing signals of wearable motion sensors. While successful for low-level activities (e.g. walking or standing), high-level activities (e.g. watching movies or attending lectures) are difficult to distinguish from motion data alone. Furthermore, instrumentation of complex body sensor network at population scale is impractical. In this work, we take an alternative approach of leveraging rich, dynamic, and crowd-generated self-report data from social media platforms as the basis for in-situ activity recognition. By treating the user as the “sensor”, we make use of implicit signals emitted from natural use of mobile smartphones, in the form of textual content, semantic location, and time. Tackling both the task of recognizing a main activity (multi-class classification) and recognizing all applicable activity categories (multi-label tagging) from one instance, we are able to obtain mean accuracies of more than 75%. We conduct a thorough analysis and interpret of our model to illustrate a promising first step towards comprehensive, high-level activity recognition using instrumentation-free, crowdsourced, social media data.     

Linguistic summarization of video for fall detection using voxel person and fuzzy logic

In this paper, we present a method for recognizing human activity from linguistic summarizations of temporal fuzzy inference curves representing the states of a three-dimensional object called voxel person. A hierarchy of fuzzy logic is used, where the output from each level is summarized and fed into the next level. We present a two level model for fall detection. The first level infers the states of the person at each image. The second level operates on linguistic summarizations of voxel person’s states and inference regarding activity is performed. The rules used for fall detection were designed under the supervision of nurses to ensure that they reflect the manner in which elders perform these activities. The proposed framework is extremely flexible. Rules can be modified, added, or removed, allowing for per-resident customization based on knowledge about their cognitive and physical ability.     

Enhancing activity recognition using CPD-based activity segmentation

Segmenting behavior-based sensor data and recognizing the activity that the data represents are vital steps in all applications of human activity learning such as health monitoring, security, and intervention. In this paper, we enhance activity recognition by identifying activity transitions. To accomplish this goal, we introduce a change point detection-based activity segmentation model which partitions behavior-driven sensor data into non-overlapping activities in real time. In addition to providing valuable activity information, activity segmentation also can be used to improve the performance of activity recognition. We evaluate our proposed segmentation-enhanced activity recognition method on data collected from 29 smart homes. Results of this analysis indicate that the method not only provides useful information about activity boundaries and transitions between activities but also increases recognition accuracy by 7.59% and f measure by 6.69% in comparison with the traditional window-based methods.     

Activity Recognition Based on RFID Object Usage for Smart Mobile Devices

Activity recognition is a core aspect of ubiquitous computing applications.In order to deploy activity recognition systems in the real world,we need simple sensing systems with lightweight computational modules to accurately analyze sensed data.In this paper,we propose a simple method to recognize human activities using simple object information involved in activities.We apply activity theory for representing complex human activities and propose a penalized naive Bayes classifier for performing activity recognition.Our results show that our method reduces computation up to an order of magnitude in both learning and inference without penalizing accuracy,when compared to hidden Markov models and conditional random fields.     

Temporal and spatial data mining with second-order hidden markov models

In the frame of designing a knowledge discovery system, we have developed stochastic models based on high-order hidden Markov models. These models are capable to map sequences of data into a Markov chain in which the transitions between the states depend on the n previous states according to the order of the model. We study the process of achieving information extraction from spatial and temporal data by means of an unsupervised classification. We use therefore a French national database related to the land use of a region, named Ter Uti, which describes the land use both in the spatial and temporal domain. Land-use categories (wheat, corn, forest, ...) are logged every year on each site regularly spaced in the region. They constitute a temporal sequence of images in which we look for spatial and temporal dependencies. The temporal segmentation of the data is done by means of a second-order Hidden Markov Model (HMM2) that appears to have very good capabilities to locate stationary segments, as shown in our previous work in speech recognition. The spatial classification is performed by defining a fractal scanning of the images with the help of a Hilbert–Peano curve that introduces a total order on the sites, preserving the relation of neighborhood between the sites. We show that the HMM2 performs a classification that is meaningful for the agronomists. Spatial and temporal classification may be achieved simultaneously by means of a two levels HMM2 that measures the a posteriori probability to map a temporal sequence of images onto a set of hidden classes.     

Dance performance evaluation using hidden Markov models

We present in this paper a hidden Markov model‐based system for real‐time gesture recognition and performance evaluation. The system decodes performed gestures and outputs at the end of a recognized gesture, a likelihood value that is transformed into a score. This score is used to evaluate a performance comparing to a reference one. For the learning procedure, a set of relational features has been extracted from high‐precision motion capture system and used to train hidden Markov models. At runtime, a low‐cost sensor (Microsoft Kinect) is used to capture a learner's movements. An intermediate step of model adaptation was hence requested to allow recognizing gestures captured by this low‐cost sensor. We present one application of this gesture evaluation system in the context of traditional dance basics learning. The estimation of the log‐likelihood allows giving a feedback to the learner as a score related to his performance. Copyright © 2016 John Wiley & Sons, Ltd.     

Efficient duration and hierarchical modeling for human activity recognition

A challenge in building pervasive and smart spaces is to learn and recognize human activities of daily living (ADLs). In this paper, we address this problem and argue that in dealing with ADLs, it is beneficial to exploit both their typical duration patterns and inherent hierarchical structures. We exploit efficient duration modeling using the novel Coxian distribution to form the Coxian hidden semi-Markov model (CxHSMM) and apply it to the problem of learning and recognizing ADLs with complex temporal dependencies. The Coxian duration model has several advantages over existing duration parameterization using multinomial or exponential family distributions, including its denseness in the space of nonnegative distributions, low number of parameters, computational efficiency and the existence of closed-form estimation solutions. Further we combine both hierarchical and duration extensions of the hidden Markov model (HMM) to form the novel switching hidden semi-Markov model (SHSMM), and empirically compare its performance with existing models. The model can learn what an occupant normally does during the day from unsegmented training data and then perform online activity classification, segmentation and abnormality detection. Experimental results show that Coxian modeling outperforms a range of baseline models for the task of activity segmentation. We also achieve a recognition accuracy competitive to the current state-of-the-art multinomial duration model, while gaining a significant reduction in computation. Furthermore, cross-validation model selection on the number of phases K in the Coxian indicates that only a small K is required to achieve the optimal performance. Finally, our models are further tested in a more challenging setting in which the tracking is often lost and the activities considerably overlap. With a small amount of labels supplied during training in a partially supervised learning mode, our models are again able to deliver reliable performance, again with a small number of phases, making our proposed framework an attractive choice for activity modeling.     

基于时空关注度LSTM的行为识别

针对现有基于视频整体序列结构建模的行为识别方法中,存在着大量时空背景混杂信息,而引起的行为表达的判决能力低和行为类别错误判定的问题,提出一种基于双流特征的时空关注度长短时记忆网络模型.首先,本文定义了一种基于双流的时空关注度模块,其中,空间关注度用于抑制空间背景混杂,时间关注度用于抑制低信息量的视频帧.其次,本文为双流...     

Postprocessing of recognized strings using nonstationary Markovianmodels

This paper presents nonstationary Markovian models and their application to recognition of strings of tokens. Domain specific knowledge is brought to bear on the application of recognizing zip codes in the US mailstream by the use of postal directory files. These files provide a wealth of information on the delivery points (mailstops) corresponding to each zip code. This data feeds into the models as n-grams, statistics that are integrated with recognition scores of digit images. An especially interesting facet of the model is its ability to excite and inhibit certain positions in the n-grams leading to the familiar area of Markov random fields. We empirically illustrate the success of Markovian modeling in postprocessing applications of string recognition. We present the recognition accuracy of the different models on a set of 20000 zip codes. The performance is superior to the present system which ignores all contextual information and simply relies on the recognition scores of the digit recognizers     

一种基于BP—HMM的字符识别方法

传统隐马尔柯夫模型广泛地应用在字符识别中,并具有较强的识别能力,但不能兼顾每个模型对其对应目标有很强的识别能力和模型之间差异性的最大化。该文提出的BP—隐马尔柯夫模型通过训练样本的不断训练,调整自身参数,解决了传统隐马尔柯夫模型不能解决的问题。计算机仿真结果表明:BP—隐马尔柯夫模型较传统的隐马尔柯夫模型有更强的抗干扰能力和更高的字符识别率。     

Hidden Markov models with spectral features for 2D shaperecognition

We present a technique using Markov models with spectral features for recognizing 2D shapes. We analyze the properties of Fourier spectral features derived from closed contours of 2D shapes and use these features for 2D pattern recognition. We develop algorithms for reestimating parameters of hidden Markov models. To demonstrate the effectiveness of our models, we have tested our methods on two image databases: hand-tools and unconstrained handwritten numerals. We are able to achieve high recognition rates of 99.4 percent and 96.7 percent without rejection on these two sets of image data, respectively     

On hierarchical modelling of motion for workflow analysis from overhead view

Understanding human behaviour is a high level perceptual problem, one which is often dominated by the contextual knowledge of the environment, and where concerns such as occlusion, scene clutter and high within-class variations are commonplace. Nonetheless, such understanding is highly desirable for automated visual surveillance. We consider this problem in a context of a workflow analysis within an industrial environment. The hierarchical nature of the workflow is exploited to split the problem into ‘activity’ and ‘task’ recognition. In this, sequences of low level activities are examined for instances of a task while the remainder are labelled as background. An initial prediction of activity is obtained using shape and motion based features of the moving blob of interest. A sequence of these activities is further adjusted by a probabilistic analysis of transitions between activities using hidden Markov models (HMMs). In task detection, HMMs are arranged to handle the activities within each task. Two separate HMMs for task and background compete for an incoming sequence of activities. Imagery derived from a camera mounted overhead the target scene has been chosen over the more conventional oblique views (from the side) as this view does not suffer from as much occlusion, and it poses a manageable detection and tracking problem while still retaining powerful cues as to the workflow patterns. We evaluate our approach both in activity and task detection on a challenging dataset of surveillance of human operators in a car manufacturing plant. The experimental results show that our hierarchical approach can automatically segment the timeline and spatially localize a series of predefined tasks that are performed to complete a workflow.     

Towards a dynamical model for wireless sensor networks

In this paper we introduce two dynamical models for a broadcast process in wireless sensor networks. We obtain a convergent martingale sequence for the two models. To our knowledge, such martingales were unknown previously. We look at the formal models using the formalisms of martingales, dynamical systems and Markov chains, each formalism providing complementary and coherent information with each other. The dynamics of both models are comparable and are validated in their domain of application with numerical simulation of wireless sensor networks. We make explicit the situations where the models are realistic. We also provide a formal analysis of the quasi-stationary distribution associated to the Markov chain corresponding to the second model proposed.     

Stochastic models for recognition of occluded targets

Recognition of occluded objects in synthetic aperture radar (SAR) images is a significant problem for automatic target recognition. Stochastic models provide some attractive features for pattern matching and recognition under partial occlusion and noise. In this paper, we present a hidden Markov modeling based approach for recognizing objects in SAR images. We identify the peculiar characteristics of SAR sensors and using these characteristics we develop feature based multiple models for a given SAR image of an object. The models exploiting the relative geometry of feature locations or the amplitude of SAR radar return are based on sequentialization of scattering centers extracted from SAR images. In order to improve performance we integrate these models synergistically using their probabilistic estimates for recognition of a particular target at a specific azimuth. Experimental results are presented using both synthetic and real SAR images.