Sensor Selection to Support Practical Use of Health-Monitoring Smart Environments

The data mining and pervasive sensing technologies found in smart homes offer unprecedented opportunities for providing health monitoring and assistance to individuals experiencing difficulties living independently at home. In order to monitor the functional health of smart home residents, we need to design technologies that recognize and track activities that people normally perform as part of their daily routines. One question that frequently arises, however, is how many smart home sensors are needed and where should they be placed in order to accurately recognize activities? We employ data mining techniques to look at the problem of sensor selection for activity recognition in smart homes. We analyze the results based on six data sets collected in five distinct smart home environments.     

ADR-SPLDA: Activity discovery and recognition by combining sequential patterns and latent Dirichlet allocation

This paper presents ADR-SPLDA, an unsupervised model for human activity discovery and recognition in pervasive environments. The activities are encoded in sequences recorded by non-intrusive sensors placed at various locations in the environment. Our model studies the relationship between the activities and the sequential patterns extracted from the sequences. Activity discovery is formulated as an optimization problem in which sequences are modeled as probability distributions over activities, and activities are, in turn, modeled as probability distributions over sequential patterns. The optimization problem is solved by maximization of the likelihood of data. We present experimental results on real datasets gathered in smart homes where people perform various activities of daily living. The results obtained demonstrate the suitability of our model for activity discovery and characterization. Also, we empirically demonstrate the effectiveness of our model for activity recognition by comparing it with two of the widely used models reported in the literature, the Hidden Markov model and the Conditional Random Field model.     

Data fusion and multiple classifier systems for human activity detection and health monitoring: Review and open research directions

Activity detection and classification using different sensor modalities have emerged as revolutionary technology for real-time and autonomous monitoring in behaviour analysis, ambient assisted living, activity of daily living (ADL), elderly care, rehabilitations, entertainments and surveillance in smart home environments. Wearable devices, smart-phones and ambient environments devices are equipped with variety of sensors such as accelerometers, gyroscopes, magnetometer, heart rate, pressure and wearable camera for activity detection and monitoring. These sensors are pre-processed and different feature sets such as time domain, frequency domain, wavelet transform are extracted and transform using machine learning algorithm for human activity classification and monitoring. Recently, deep learning algorithms for automatic feature representation have also been proposed to lessen the burden of reliance on handcrafted features and to increase performance accuracy. Initially, one set of sensor data, features or classifiers were used for activity recognition applications. However, there are new trends on the implementation of fusion strategies to combine sensors data, features and classifiers to provide diversity, offer higher generalization, and tackle challenging issues. For instances, combination of inertial sensors provide mechanism to differentiate activity of similar patterns and accurate posture identification while other multimodal sensor data are used for energy expenditure estimations, object localizations in smart homes and health status monitoring. Hence, the focus of this review is to provide in-depth and comprehensive analysis of data fusion and multiple classifier systems techniques for human activity recognition with emphasis on mobile and wearable devices. First, data fusion methods and modalities were presented and also feature fusion, including deep learning fusion for human activity recognition were critically analysed, and their applications, strengths and issues were identified. Furthermore, the review presents different multiple classifier system design and fusion methods that were recently proposed in literature. Finally, open research problems that require further research and improvements are identified and discussed.     

Interaction‐Feature Enhanced Multiuser Model Learning for a Home Environment Using Ambient Sensors

Activity recognition (AR) is a key enabler for a context‐aware smart home since knowing what the residents’ current activities helps a smart home provide more desirable services. This is why AR is often used in assistive technologies for cognitively impaired people to evaluate their abilities to undertake activities of daily living. In a real‐life scenario, multiple‐resident AR has been considered as a very challenging problem, primarily due to the complexity of data association. In addition, most prior research has not considered the potential interpersonal interactions among residents to simplify complexity, especially in an environment monitored by ambient sensors. In this study, we propose two types of multiuser activity models, both of which are derived from an interaction‐feature enhanced multiuser model learning framework. These two models consider interpersonal interactions and data association for multiuser AR using ambient sensors. We then compare their performance with the other two baseline models with or without consideration of data association and interpersonal interactions. The experimental results show that the derived models outperform other baseline classifiers. Therefore, the proposed approach can increase the opportunities for providing context‐aware services for a multiresident smart home.     

Identifying multiuser activity with overlapping acoustic data for mobile decision making in smart home environments

Smart services, one of the most intriguing areas of current Internet of Things(IoT) research, require improvement in terms of recognizing user activities. Sound is a useful medium for making decisions based on activity recognition in the smart home environment, which includes mobile devices such as sensors and actuators. Instead of visual sensors to recognize human activity, acoustic sensor data is acquired in an unobtrusive manner for greater privacy. However, multiuser activity provides a formidable challenge for acoustic data-based activity recognition systems because of the difficulty of identifying multiple sources of activity from among a variety of sounds. In our study, we propose a statistical method to detect the interval of interference, which is also known as the unexpected mesa, distinguishing activities based on the pre- and post-mesa intervals. The results suggest that the proposed method outperforms previously presented classification algorithms in terms of the accuracy of multiuser activity recognition. Future studies may utilize this method for improvement of existing smart home systems.     

A selection framework of sensor combination feature subset for human motion phase segmentation

Motion phase plays an important role in the spatial–temporal parameters of human motion analysis. Multi-sensor fusion technology based on inertial sensors frees the monitoring of the human body phase from space constraints and improves the flexibility of the system. However, human phase segmentation methods usually rely on the determination of the positioning of the sensor and the number of sensors, it is difficult to artificially select the number and position of the sensors, especially when human motion phases are diverse. This paper proposes a selection framework for the sensor combination feature subset for motion phase segmentation, which combines feature selection algorithms with the subsequent classifiers, and determine the optimum combination of the sensor and the feature subset according to the performance of the trained model. Through the constraint and the sensor combination feature subset (SCFS), the filter method can select any number of sensors and control the size of the feature subset; the embedded method can select any number of sensors, but the size of the feature subset is determined by the classifier model. Experimental results show that the proposed framework can effectively select a specified number of sensors without human intervention, and the number of sensors has an impact on the recognition rate of the classifier within 1.5%. In addition, the filter method has good adaptability to a variety of classifiers, and the classifier prediction time can be controlled by setting the subset size of the feature; the embedded method can achieve a better phase segmentation effect than the filter method. For the application of motion phase segmentation, the proposed framework can reliably and quickly identify redundant sensors that provide effective support for reducing the complexity of the wearable sensor system and improving user comfort.     

Neighborhood rough set based heterogeneous feature subset selection

Feature subset selection is viewed as an important preprocessing step for pattern recognition, machine learning and data mining. Most of researches are focused on dealing with homogeneous feature selection, namely, numerical or categorical features. In this paper, we introduce a neighborhood rough set model to deal with the problem of heterogeneous feature subset selection. As the classical rough set model can just be used to evaluate categorical features, we generalize this model with neighborhood relations and introduce a neighborhood rough set model. The proposed model will degrade to the classical one if we specify the size of neighborhood zero. The neighborhood model is used to reduce numerical and categorical features by assigning different thresholds for different kinds of attributes. In this model the sizes of the neighborhood lower and upper approximations of decisions reflect the discriminating capability of feature subsets. The size of lower approximation is computed as the dependency between decision and condition attributes. We use the neighborhood dependency to evaluate the significance of a subset of heterogeneous features and construct forward feature subset selection algorithms. The proposed algorithms are compared with some classical techniques. Experimental results show that the neighborhood model based method is more flexible to deal with heterogeneous data.     

Analysis and effects of smart home dataset characteristics for daily life activity recognition

Over the last few years, activity recognition in the smart home has become an active research area due to the wide range of human centric-applications. With the development of machine learning algorithms for activity classification, dataset is significantly important for algorithms testing and validation. Collection of real data is a challenging process due to involved budget, human resources, and annotation cost that’s why mostly researchers prefer to utilize existing datasets for evaluation purposes. However, openly available smart home datasets indicate variation in terms of performed activities, deployed sensors, and environment settings. Unfortunately, the analysis of existing datasets characteristic is a bottleneck for researchers while selecting datasets of their intent. In this paper, we develop a Framework for Smart Homes Dataset Analysis (FSHDA) to reflect their diverse dimensions in predefined format. It analyzes a list of data dimensions that covers the variations in time, activities, sensors, and inhabitants. For validation, we examine the effects of proposed data dimension on state-of-the-art activity recognition techniques. The results show that dataset dimensions highly affect the classifiers’ individual activity label assignments and their overall performances. The outcome of our study is helpful for upcoming researchers to develop a better understanding about the smart home datasets characteristics with classifier’s performance.     

The Mobile Sensing Platform: An Embedded Activity Recognition System

Activity-aware systems have inspired novel user interfaces and new applications in smart environments, surveillance, emergency response, and military missions. Systems that recognize human activities from body-worn sensors can further open the door to a world of healthcare applications, such as fitness monitoring, eldercare support, long-term preventive and chronic care, and cognitive assistance. Wearable systems have the advantage of being with the user continuously. So, for example, a fitness application could use real-time activity information to encourage users to perform opportunistic activities. Furthermore, the general public is more likely to accept such activity recognition systems because they are usually easy to turn off or remove.     

基于三维加速度的连续手势识别

由于基于图像处理的手势识别方法对环境背景要求较高且存在不稳定性问题,文章使用三维加速度传感器的连续数据进行手势识别.三维加速度传感器内置于大部分智能手机中,具有应用方便的特点.实验通过传感器获取加速度信号,经过低通滤波、去重力和特征提取的信号预处理过程后,结合隐马尔可夫模型和混合高斯模型的理论方法,实现手机手势的连续识别,并驱动应用层预先定义的交互命令.     

An efficient feature selection method for mobile devices with application to activity recognition

This paper presents a feature selection method for data classification, which combines a model-based variable selection technique and a fast two-stage subset selection algorithm. The relationship between a specified (and complete) set of candidate features and the class label is modeled using a non-linear full regression model which is linear-in-the-parameters. The performance of a sub-model measured by the sum of the squared-errors (SSE) is used to score the informativeness of the subset of features involved in the sub-model. The two-stage subset selection algorithm approaches a solution sub-model with the SSE being locally minimized. The features involved in the solution sub-model are selected as inputs to support vector machines (SVMs) for classification. The memory requirement of this algorithm is independent of the number of training patterns. This property makes this method suitable for applications executed in mobile devices where physical RAM memory is very limited.An application was developed for activity recognition, which implements the proposed feature selection algorithm and an SVM training procedure. Experiments are carried out with the application running on a PDA for human activity recognition using accelerometer data. A comparison with an information gain-based feature selection method demonstrates the effectiveness and efficiency of the proposed algorithm.     

Application of ultrasonic sensors in a smart environment

A key application of sensor networks in smart environments is in monitoring activities of people. We develop several scenarios in which ultrasonic sensors are used for monitoring of patients and the elderly. In each scenario, we apply different algorithms for data fusion and sensor selection using quality-based or time division approaches. We have devised trajectory-matching algorithms to classify trajectories of movement of people in indoor environments. The trajectories are divided into several routine classes and the current trajectory is compared against the known routine trajectories. The initial results are quite promising, and show the potential usability of ultrasonic sensors in monitoring indoor movements of people, and in capturing and classifying trajectories.     

Monitoring user activities in smart home environments

Wireless sensor networks (WSNs) enable smart environments to create pervasive and ubiquitous applications, which give context-aware and scalable services to the end users. In this paper, we propose an architecture and design of a web application for a sensor network monitoring. Further, the variation in received signal strength indicator values is used for knowledge extraction. Experiments are conducted in an in-door room environment to determine the activities of a person. For instance, a WSN consisting of Moteiv’s Tmote Sky sensors is deployed in a bedroom to determine the sleeping behavior and other activities of a person.     

COSAR: hybrid reasoning for context-aware activity recognition

Human activity recognition is a challenging problem for context-aware systems and applications. Research in this field has mainly adopted techniques based on supervised learning algorithms, but these systems suffer from scalability issues with respect to the number of considered activities and contextual data. In this paper, we propose a solution based on the use of ontologies and ontological reasoning combined with statistical inferencing. Structured symbolic knowledge about the environment surrounding the user allows the recognition system to infer which activities among the candidates identified by statistical methods are more likely to be the actual activity that the user is performing. Ontological reasoning is also integrated with statistical methods to recognize complex activities that cannot be derived by statistical methods alone. The effectiveness of the proposed technique is supported by experiments with a complete implementation of the system using commercially available sensors and an Android-based handheld device as the host for the main activity recognition module.     

Ontology-based modeling of dynamic ubiquitous computing applications as evolving activity spheres

As the computer disappears in the environments surrounding our activities, the objects therein become augmented with sensors, actuators, processors, memories, wireless communication modules and they can receive, store, process and transmit information. In addition to objects, spaces also undergo a change towards becoming smart and eventually Ambient Intelligence (AmI) spaces. In order to model the way everyday activities are carried out within an AmI environment, we introduce the notion of “activity sphere”. In this paper, we are interested in the ontology-based representation of activity spheres from two different perspectives (as creators and as observers), as well as the modeling and control of the dynamic nature of activity spheres.     

Luminous tiles: A new building device for smart architectures and applications

Advanced building materials are nowadays an active research domain. The integration of traditional materials and technologies in the field of electronics, photonics and computer science are leading to a new class of smart components that provide advanced functionalities and enable original applications.The LUMENTILE H2020 EU funded Project aims at the integration of existing and state-of-the-art technologies in the domain of large area electronic circuits, LED based lighting, embedded systems and communication. These technologies are blended with advancements in the manufacturing of ceramic tiles to obtain a new building component that can be managed as a common tile, while providing the possibility to self-illuminate and to sense the neighbor environment by means of dedicated sensors. The applications of these new material and technologies include indoor and outdoor architectural design, smart environments (also targeting improved safety and security issues), smart and high-efficiency lighting and art installations. State-of-the-art advancements are expected in the field of large area circuits and successful integration of heterogeneous materials, mainly focusing on ceramics and electronics.     

A Survey of User-Centred Approaches for Smart Home Transfer Learning and New User Home Automation Adaptation

Recent smart home applications enhance the quality of people’s home experiences by detecting their daily activities and providing them services that make their daily life more comfortable and safe. Human activity recognition is one of the fundamental tasks that a smart home should accomplish. However, there are still several challenges for such recognition in smart homes, with the target home adaptation process being one of the most critical, since new home environments do not have sufficient data to initiate the necessary activity recognition process. The transfer learning approach is considered the solution to this challenge, due to its ability to improve the adaptation process. This paper endeavours to provide a concrete review of user-centred smart homes along with the recent advancements in transfer learning for activity recognition. Furthermore, the paper proposes an integrated, personalised system that is able to create a dataset for target homes using both survey and transfer learning approaches, providing a personalised dataset based on user preferences and feedback.     

An evidential fusion approach for activity recognition in ambient intelligence environments

With the growing emergence of ambient intelligence, ubiquitous computing, sensor networks and wireless networking technologies, “ubiquitous networked robotics” is becoming an active research domain of intelligent autonomous systems. It targets new innovative applications in which robotic systems will become part of these networks of artifacts to provide novel capabilities and various assistive services anywhere and anytime, such as healthcare and monitoring services for elderly in Ambient Assisted Living (AAL) environments. Situation recognition, in general, and activity recognition, in particular, provide an added value on the contextual information that can help the ubiquitous networked robot to autonomously provide the best service that meet the needs of the elderly. Dempster–Shafer theory of evidence and its derivatives are an efficient tool to handle uncertainty and incompleteness in smart homes and ubiquitous computing environments. However, their combination rules yield counter-intuitive results in high conflicting activities. In this paper, we propose a new approach to support conflict resolution in activity recognition in AAL environments. This approach is based on a new mapping for conflict evidential fusion to increase the efficiency and accuracy of activity recognition. It gives intuitive interpretation for combining multiple sources in all conflicting situations. The proposed approach, evaluated on a real world smart home dataset, achieves 78% of accuracy in activity recognition. The obtained results outperform those obtained with the existing combination rules.     

基于移动增强现实的智慧城市导览

提出一种采用移动增强现实技术实现智慧城市导览的方法,满足用户个性化、多尺度、按需推送的智能导览需求,呈现用户虚实融合的周边环境.移动终端计算性能以及资源存储能力有限,但集成多种传感器,方便携带,易于显示.利用服务器实现基于词汇树的海量场景识别定位系统.依据地理位置信息动态划分分区缩减了场景检索范围,基于二进制鲁棒尺度不变特征(binary robust invariant scalable keypoints, BRISK)进行层级式聚类提高了识别算法的实时性.移动终端利用服务器返回的识别结果进行BRISK特征与光流算法结合的混合特征跟踪注册方法,并通过点集映射消除特征点漂移,利用前后帧信息以及关键帧信息减少跟踪抖动.UKbench标准图像库以及真实环境下的实验结果表明,虚实融合的智能导览效果良好.该原型系统已成功应用于上海电信体验馆等展馆智能导览系统.     

Robust Location-Aware Activity Recognition Using Wireless Sensor Network in an Attentive Home

This paper presents a robust location-aware activity recognition approach for establishing ambient intelligence applications in a smart home. With observations from a variety of multimodal and unobtrusive wireless sensors seamlessly integrated into ambient-intelligence compliant objects (AICOs), the approach infers a single resident's interleaved activities by utilizing a generalized and enhanced Bayesian Network fusion engine with inputs from a set of the most informative features. These features are collected by ranking their usefulness in estimating activities of interest. Additionally, each feature reckons its corresponding reliability to control its contribution in cases of possible device failure, therefore making the system more tolerant to inevitable device failure or interference commonly encountered in a wireless sensor network, and thus improving overall robustness. This work is part of an interdisciplinary Attentive Home pilot project with the goal of fulfilling real human needs by utilizing context-aware attentive services. We have also created a novel application called ldquoActivity Maprdquo to graphically display ambient-intelligence-related contextual information gathered from both humans and the environment in a more convenient and user-accessible way. All experiments were conducted in an instrumented living lab and their results demonstrate the effectiveness of the system.