Identifying multiuser activity with overlapping acoustic data for mobile decision making in smart home environments

Smart services, one of the most intriguing areas of current Internet of Things(IoT) research, require improvement in terms of recognizing user activities. Sound is a useful medium for making decisions based on activity recognition in the smart home environment, which includes mobile devices such as sensors and actuators. Instead of visual sensors to recognize human activity, acoustic sensor data is acquired in an unobtrusive manner for greater privacy. However, multiuser activity provides a formidable challenge for acoustic data-based activity recognition systems because of the difficulty of identifying multiple sources of activity from among a variety of sounds. In our study, we propose a statistical method to detect the interval of interference, which is also known as the unexpected mesa, distinguishing activities based on the pre- and post-mesa intervals. The results suggest that the proposed method outperforms previously presented classification algorithms in terms of the accuracy of multiuser activity recognition. Future studies may utilize this method for improvement of existing smart home systems.     

GPU-based parallel Shadow Features generation at neural system for improving gait human activity recognition

Multimedia Tools and Applications - In this paper, we propose a new method for improving human activity recognition (HAR) datasets in order to increase their classification accuracy when trained...     

基于多层动态贝叶斯网络的人的行为多尺度分析及识别方法

人的行为识别是视频内容分析和计算机视觉领域中的一个重要问题. 在分析了人的行为包含多个尺度运动细节的基础上, 提出了一种分层且带驻留时间状态的动态贝叶斯网络(Hierarchical durational-state dynamic Bayesian network, HDS-DBN). HDS-DBN含有多层状态, 能够较好地表示人的行为包含的多尺度运动细节. 我们针对单人行为和两人交互行为进行了识别, 实验结果表明该方法具有较高的识别率, 并且在有噪声存在或信息缺失等不确定情况下均具有较好的鲁棒性. 实验结果表明 HDS-DBN 模型确实能够较好地表达行为中的多尺度运动细节.     

COSAR: hybrid reasoning for context-aware activity recognition

Human activity recognition is a challenging problem for context-aware systems and applications. Research in this field has mainly adopted techniques based on supervised learning algorithms, but these systems suffer from scalability issues with respect to the number of considered activities and contextual data. In this paper, we propose a solution based on the use of ontologies and ontological reasoning combined with statistical inferencing. Structured symbolic knowledge about the environment surrounding the user allows the recognition system to infer which activities among the candidates identified by statistical methods are more likely to be the actual activity that the user is performing. Ontological reasoning is also integrated with statistical methods to recognize complex activities that cannot be derived by statistical methods alone. The effectiveness of the proposed technique is supported by experiments with a complete implementation of the system using commercially available sensors and an Android-based handheld device as the host for the main activity recognition module.     

基于单目视觉的在线人体康复动作识别

为了进一步提高居家监护场景下人体动作识别的可靠性与实时性,更好地辅助出院后的卒中患者进行康复训练,提出一种基于单目视觉的在线人体动作识别算法。融合姿态估计OpenPose与最近邻匹配算法对监控视频流中的目标人体生成动作序列。通过滑动窗口选取原始姿态特征并对其预处理转化为鲁棒性特征,输入到多层LSTM长短时记忆网络中进行康复动作识别。实验结果表明,该方法对活动背景、人体穿着、无关人员的干扰等具有较强的适应能力,能够在线识别连续的康复动作且准确率达90.66%,在居家康复训练场景中有一定的应用价值。     

A hierarchical human detection system in (un)compressed domains

We propose a hierarchical retrieval system where shape, color and motion characteristics of the human body are captured in compressed and uncompressed domains. The proposed retrieval method provides human detection and activity recognition at different resolution levels from low complexity to low false rates and connects low level features to high level semantics by developing relational object and activity presentations. The available information of standard video compression algorithms are used in order to reduce the amount of time and storage needed for the information retrieval. The principal component analysis is used for activity recognition using MPEG motion vectors and results are presented for walking, kicking, and running to demonstrate that the classification among activities is clearly visible. For low resolution and monochrome images it is demonstrated that the structural information of human silhouettes can be captured from AC-DCT coefficients.     

Suspicious human activity recognition: a review

Suspicious human activity recognition from surveillance video is an active research area of image processing and computer vision. Through the visual surveillance, human activities can be monitored in sensitive and public areas such as bus stations, railway stations, airports, banks, shopping malls, school and colleges, parking lots, roads, etc. to prevent terrorism, theft, accidents and illegal parking, vandalism, fighting, chain snatching, crime and other suspicious activities. It is very difficult to watch public places continuously, therefore an intelligent video surveillance is required that can monitor the human activities in real-time and categorize them as usual and unusual activities; and can generate an alert. Recent decade witnessed a good number of publications in the field of visual surveillance to recognize the abnormal activities. Furthermore, a few surveys can be seen in the literature for the different abnormal activities recognition; but none of them have addressed different abnormal activities in a review. In this paper, we present the state-of-the-art which demonstrates the overall progress of suspicious activity recognition from the surveillance videos in the last decade. We include a brief introduction of the suspicious human activity recognition with its issues and challenges. This paper consists of six abnormal activities such as abandoned object detection, theft detection, fall detection, accidents and illegal parking detection on road, violence activity detection, and fire detection. In general, we have discussed all the steps those have been followed to recognize the human activity from the surveillance videos in the literature; such as foreground object extraction, object detection based on tracking or non-tracking methods, feature extraction, classification; activity analysis and recognition. The objective of this paper is to provide the literature review of six different suspicious activity recognition systems with its general framework to the researchers of this field.     

Recognizing interleaved and concurrent activities using qualitative and quantitative temporal relationships

The majority of approaches to activity recognition in sensor environments are either based on manually constructed rules for recognizing activities or lack the ability to incorporate complex temporal dependencies. Furthermore, in many cases, the rather unrealistic assumption is made that the subject carries out only one activity at a time. In this paper, we describe the use of Markov logic as a declarative framework for recognizing interleaved and concurrent activities incorporating both input from pervasive lightweight sensor technology and common-sense background knowledge. In particular, we assess its ability to learn statistical-temporal models from training data and to combine these models with background knowledge to improve the overall recognition accuracy. We also show the viability and the benefit of exploiting both qualitative and quantitative temporal relationships like the duration of the activities and their temporal order. To this end, we propose two Markov logic formulations for inferring the foreground activity as well as each activities’ start and end times. We evaluate the approach on an established dataset where it outperforms state-of-the-art algorithms for activity recognition.     

Weighted joint-based human behavior recognition algorithm using only depth information for low-cost intelligent video-surveillance system

Recent advances in 3D depth sensors have created many opportunities for security, surveillance, and entertainment. The 3D depth sensors provide more powerful monitoring systems for dangerous situations irrespective of lighting conditions in buildings or production facilities. To robustly recognize emergency actions or hazardous situations of workers at a production facility, we present human joint estimation and behavior recognition algorithms that solely use depth information in this paper. To estimate human joints on a low cost computing platform, we propose a human joint estimation algorithm that integrates a geodesic graph and a support vector machine (SVM). The human feature points are extracted within a range of geodesic distance from a geodesic graph. The geodesic graph is used for optimizing the estimation result. The SVM-based human joint estimator uses randomly selected human features to reduce computation. Body parts that typically involve many motions are then estimated by the geodesic distance value. The proposed algorithm can work for any human without calibration, and thus the system can be used with any subject immediately even with a low cost computing platform. In the case of the behavior recognition algorithm, the algorithm should have a simple behavior registration process, and it also should be robust to environmental changes. To meet these goals, we propose a template matching-based behavior recognition algorithm. Our method creates a behavior template set that consists of weighted human joint data with scale and rotation invariant properties. A single behavior template consists of the joint information that is estimated per frame. Additionally, we propose adaptive template rejection and a sliding window filter to prevent misrecognition between similar behaviors. The human joint estimation and behavior recognition algorithms are evaluated individually through several experiments and the performance is proven through a comparison with other algorithms. The experimental results show that our method performs well and is applicable in real environments.     

Independent shape component-based human activity recognition via Hidden Markov Model

In proactive computing, human activity recognition from image sequences is an active research area. In this paper, a novel human activity recognition method is proposed, which utilizes Independent Component Analysis (ICA) for activity shape information extraction from image sequences and Hidden Markov Model (HMM) for recognition. Various human activities are represented by shape feature vectors from the sequence of activity shape images via ICA. Based on these features, each HMM is trained and activity recognition is achieved by the trained HMMs of different activities. Our recognition performance has been compared to the conventional method where Principal Component Analysis (PCA) is typically used to derive activity shape features. Our results show that superior recognition is achieved with the proposed method especially for activities (e.g., skipping) that cannot be easily recognized by the conventional method. Furthermore, by employing Linear Discriminant Analysis (LDA) on IC features, the recognition results further improved significantly in the recognition performance.     

Transfer learning for activity recognition: a survey

Many intelligent systems that focus on the needs of a human require information about the activities being performed by the human. At the core of this capability is activity recognition, which is a challenging and well-researched problem. Activity recognition algorithms require substantial amounts of labeled training data yet need to perform well under very diverse circumstances. As a result, researchers have been designing methods to identify and utilize subtle connections between activity recognition datasets, or to perform transfer-based activity recognition. In this paper, we survey the literature to highlight recent advances in transfer learning for activity recognition. We characterize existing approaches to transfer-based activity recognition by sensor modality, by differences between source and target environments, by data availability, and by type of information that is transferred. Finally, we present some grand challenges for the community to consider as this field is further developed.     

Human activity recognition using quasiperiodic time series collected from a single tri-axial accelerometer

The current generation of portable mobile devices incorporates various types of sensors that open up new areas for the analysis of human behavior. In this paper, we propose a method for human physical activity recognition using time series, collected from a single tri-axial accelerometer of a smartphone. Primarily, the method solves a problem of online time series segmentation, assuming that each meaningful segment corresponds to one fundamental period of motion. To extract the fundamental period we construct the phase trajectory matrix, applying the technique of principal component analysis. The obtained segments refer to various types of human physical activity. To recognize these activities we use the k-nearest neighbor algorithm and neural network as an alternative. We verify the accuracy of the proposed algorithms by testing them on the WISDM dataset of labeled accelerometer time series from thirteen users. The results show that our method achieves high precision, ensuring nearly 96 % recognition accuracy when using the bunch of segmentation and k-nearest neighbor algorithms.     

Mobile health monitoring system based on activity recognition using accelerometer

We propose a new method to recognize a user’s activities of daily living with accelerometers and RFID sensor. Two wireless accelerometers are used for classification of five human body states using decision tree, and detection of RFID-tagged objects with hand movements provides additional instrumental activity information. Besides, we apply our activity recognition module to the health monitoring system. We derive linear regressions for each activity by finding the correlations between the attached accelerometers and the expended calories calculated from gas exchange analyzer under different activities. Finally, we can predict the expended calories more efficiently with only accelerometer sensor depend on the recognized activity. We implement our proposed health monitoring module on smart phones for better practical use.     

EFM: evolutionary fuzzy model for dynamic activities recognition using a smartphone accelerometer

Activity recognition is an emerging field of research that enables a large number of human-centric applications in the u-healthcare domain. Currently, there are major challenges facing this field, including creating devices that are unobtrusive and handling uncertainties associated with dynamic activities. In this paper, we propose a novel Evolutionary Fuzzy Model (EFM) to measure the uncertainties associated with dynamic activities and relax the domain knowledge constraints which are imposed by domain experts during the development of fuzzy systems. Based on the time and frequency domain features, we define the fuzzy sets and estimate the natural grouping of data through expectation maximization of the likelihoods. A Genetic Algorithm (GA) is investigated and designed to determine the optimal fuzzy rules. To evaluate the EFM, we performed experiments on seven daily life activities of ten human subjects. Our experiments show significant improvement of 9 % in class-accuracy and 11 % in the F-measures of recognized activities compared to existing counterparts. The practical solution to dynamic activity recognition problems is expected to be an EFM, due to EFM’s utilization of smartphones and natural way of handling uncertainties.     

Human action learning via hidden Markov model

To successfully interact with and learn from humans in cooperative modes, robots need a mechanism for recognizing, characterizing, and emulating human skills. In particular, it is our interest to develop the mechanism for recognizing and emulating simple human actions, i.e., a simple activity in a manual operation where no sensory feedback is available. To this end, we have developed a method to model such actions using a hidden Markov model (HMM) representation. We proposed an approach to address two critical problems in action modeling: classifying human action-intent, and learning human skill, for which we elaborated on the method, procedure, and implementation issues in this paper. This work provides a framework for modeling and learning human actions from observations. The approach can be applied to intelligent recognition of manual actions and high-level programming of control input within a supervisory control paradigm, as well as automatic transfer of human skills to robotic systems     

基于可穿戴传感器的普适化人体活动识别

为了提高日常活动识别的准确性和自动化程度,减少人为干预,提出了利用可穿戴传感信号作为输入,通过深度神经网络进行人体活动识别的方法.首先,设计了普适环境下人体活动识别的系统架构,建立了一套加速度、生理信号等传感数据的采集系统;然后,对获取的传感数据进行降噪、加窗与归一化等预处理,并设计了长短时记忆递归神经网络来进行特征的自动提取和融合,从而实现活动识别.实验结果表明,该方法减少了对人工和专家知识的依赖,自动进行多模态传感器的融合,智能化程度高,分类效果好.     

GPARS: a general-purpose activity recognition system

The fundamental problem of the existing Activity Recognition (AR) systems is that these are not general-purpose. An AR system trained in an environment would only be applicable to that environment. Such a system would not be able to recognize the new activities of interest. In this paper we propose a General-Purpose Activity Recognition System (GPARS) using simple and ubiquitous sensors. It would be applicable to almost any environment and would have the ability to handle growing amounts of activities and sensors in a graceful manner (Scalable). Given a set of activities to monitor, object names (with embedded sensors) and their corresponding locations, the GPARS first mines activity knowledge from the web, and then uses them as the basis of AR. The novelty of our system, compared to the existing general-purpose systems, lies in: (1) it uses more robust activity models, (2) it significantly reduces the mining time. We have tested our system with three real world datasets. It is observed that the accuracy of activity recognition using our system is more than 80%. Our proposed mechanism yields significant improvement (more than 30%) in comparison with its counterpart.     

Enhancing activity recognition using CPD-based activity segmentation

Segmenting behavior-based sensor data and recognizing the activity that the data represents are vital steps in all applications of human activity learning such as health monitoring, security, and intervention. In this paper, we enhance activity recognition by identifying activity transitions. To accomplish this goal, we introduce a change point detection-based activity segmentation model which partitions behavior-driven sensor data into non-overlapping activities in real time. In addition to providing valuable activity information, activity segmentation also can be used to improve the performance of activity recognition. We evaluate our proposed segmentation-enhanced activity recognition method on data collected from 29 smart homes. Results of this analysis indicate that the method not only provides useful information about activity boundaries and transitions between activities but also increases recognition accuracy by 7.59% and f measure by 6.69% in comparison with the traditional window-based methods.