Interaction‐Feature Enhanced Multiuser Model Learning for a Home Environment Using Ambient Sensors

Activity recognition (AR) is a key enabler for a context‐aware smart home since knowing what the residents’ current activities helps a smart home provide more desirable services. This is why AR is often used in assistive technologies for cognitively impaired people to evaluate their abilities to undertake activities of daily living. In a real‐life scenario, multiple‐resident AR has been considered as a very challenging problem, primarily due to the complexity of data association. In addition, most prior research has not considered the potential interpersonal interactions among residents to simplify complexity, especially in an environment monitored by ambient sensors. In this study, we propose two types of multiuser activity models, both of which are derived from an interaction‐feature enhanced multiuser model learning framework. These two models consider interpersonal interactions and data association for multiuser AR using ambient sensors. We then compare their performance with the other two baseline models with or without consideration of data association and interpersonal interactions. The experimental results show that the derived models outperform other baseline classifiers. Therefore, the proposed approach can increase the opportunities for providing context‐aware services for a multiresident smart home.     

SmartTouch: electric skin to touch the untouchable

Augmented haptics lets users touch surface information of any modality. SmartTouch uses optical sensors to gather information and electrical stimulation to translate it into tactile display. Augmented reality is an engineer's approach to this dream. In AR, sensors capture artificial information from the world, and existing sensing channels display it. Hence, we virtually acquire the sensor's physical ability as our own. Augmented haptics, the result of applying AR to haptics, would allow a person to touch the untouchable. Our system, SmartTouch, uses a tactile display and a sensor. When the sensor contacts an object, an electrical stimulation translates the acquired information into a tactile sensation, such as a vibration or pressure, through the tactile display. Thus, an individual not only makes physical contact with an object, but also touches the surface information of any modality, even those that are typically untouchable.     

Enhancing host security using external environment sensors

We propose a framework that uses (external) environment information to enhance computer security. The benefit of our framework is that the environment information is collected by sensors that are outside the control of a host and communicate to an external monitor via an out-of-band channel (w.r.t. the host), thus it cannot be compromised by malware on a host system. The information gathered still remains intact even if malware uses rootkit techniques to hide its activities. Our framework can be applied for a number of security applications: (1) intrusion detection; (2) rate monitoring/control of external resources; and (3) access control. We show that that the framework is useful even with coarse-grained and simple information. We present some experimental prototypes that employ the framework to detect/control email spam, detect/control DDoS zombie attacks and detect misuse of compute resources. Experimental evaluation shows that the framework is effecting in detecting or limiting the activities of such malware. The growing popularity of multimodal sensors and physical security information management systems suggests that such environmental sensors will become common making our framework cost effective and feasible in the near future.     

Recognizing composite daily activities from crowd-labelled social media data

Human activity recognition is a core component of context-aware, ubiquitous computing systems. Traditionally, this task is accomplished by analysing signals of wearable motion sensors. While successful for low-level activities (e.g. walking or standing), high-level activities (e.g. watching movies or attending lectures) are difficult to distinguish from motion data alone. Furthermore, instrumentation of complex body sensor network at population scale is impractical. In this work, we take an alternative approach of leveraging rich, dynamic, and crowd-generated self-report data from social media platforms as the basis for in-situ activity recognition. By treating the user as the “sensor”, we make use of implicit signals emitted from natural use of mobile smartphones, in the form of textual content, semantic location, and time. Tackling both the task of recognizing a main activity (multi-class classification) and recognizing all applicable activity categories (multi-label tagging) from one instance, we are able to obtain mean accuracies of more than 75%. We conduct a thorough analysis and interpret of our model to illustrate a promising first step towards comprehensive, high-level activity recognition using instrumentation-free, crowdsourced, social media data.     

A Resource Selection System for Cycle Stealing in GPU Grids

This paper presents a resource selection system for exploiting graphics processing units (GPUs) as general-purpose computational resources in desktop Grid environments. Our system allows Grid users to share remote GPUs, which are traditionally dedicated to local users who directly see the display output. The key contribution of the paper is to develop this novel system for non-dedicated environments. We first show criteria for defining idle GPUs from the Grid users’ point of view. Based on these criteria, our system uses a screensaver approach with some sensors that detect idle resources at a low overhead. The idea for this lower overhead is to avoid GPU intervention during resource monitoring. Detected idle GPUs are then selected according to a matchmaking service, making the system adaptive to the rapid advance of GPU architecture. Though the system itself is not yet interoperable with current desktop Grid systems, our idea can be applied to screensaver-based systems such as BOINC. We evaluate the system using Windows PCs with three generations of nVIDIA GPUs. The experimental results show that our system achieves a low overhead of at most 267 ms, minimizing interference to local users while maximizing the performance delivered to Grid users. Some case studies are also performed in an office environment to demonstrate the effectiveness of the system in terms of the amount of detected idle time.     

Activity recognition system using inbuilt sensors of smart mobile phone and minimizing feature vectors

Mobile phone is becoming a very popular tool due to having various user friendly applications with all flexible options. It is highly popular for its light weight, wearable and comfortable uses. Many extrinsic habitat of human being can be monitored by the help of inbuilt sensors and its application software. This has appealing use for healthcare applications using exploitation of Ambient Intelligence for daily activity monitoring system. Here, a standard dataset of UCI HAR (University of California, Irvine, Human Activity Recognition, http://archive.ics.uci.edu) is used for analysis purpose. Naive Bayes Classifier is used for recognition of runtime activities minimizing dimension of large feature vectors. Threshold based condition box is designed by us and finally these two results are compared with that of another classifier HF-SVM (Hardware Friendly-Support Vector Machine) of previous related work.     

IOT Assisted Biomedical Monitoring Sensors for Healthcare in Human

The Internet of Things (IoT) is a concept that refers to the deployment of Internet Protocol (IP) address sensors in health care systems to monitor patients’ health. It has the ability to access the Internet and collect data from sensors. Automated decisions are made after evaluating the information of illness people records. Patients’ health and well-being can be monitored through IoT medical devices. It is possible to trace the origins of biological, medical equipment and processes. Human reliability is a major concern in user activity and fitness trackers in day-to-day activities. The fundamental challenge is to measure the efficiency of the human system accurately. Aim to maintain tabs on the well-being of humans; this paper recommends the use of wireless body area networks (WBANs) and artificial neural networks (ANN) to create an IoT-based healthcare framework for hospital information systems (IoT-HF-HIS). Our evaluation system uses a server to estimate how much computing power is needed for modeling, and simulations of the framework have been done using data rate and latency requirements are implementing the energy-aware technology presented in this paper. The proposed framework implements several hospital information system case studies by building a time-saving simulation environment. As the world’s population ages, more and more people suffer from physical and emotional ailments. Using the recommended strategy regularly has been proven user-friendly, reliable, and cost-effective, with an overall performance of 95.2%.     

Combining emerging patterns with random forest for complex activity recognition in smart homes

New healthcare technologies are emerging with the increasing age of the society, where the development of smart homes for monitoring the elders’ activities is in the center of them. Identifying the resident’s activities in an apartment is an important module in such systems. Dense sensing approach aims to embed sensors in the environment to report the detected events continuously. The events are segmented and analyzed via classifiers to identify the corresponding activity. Although several methods were introduced in recent years for detecting simple activities, the recognition of complex ones requires more effort. Due to the different time duration and event density of each activity, finding the best size of the segments is one of the challenges in detecting the activity. Also, using appropriate classifiers that are capable of detecting simple and interleaved activities is the other issue. In this paper, we devised a two-phase approach called CARER (Complex Activity Recognition using Emerging patterns and Random forest). In the first phase, the emerging patterns are mined, and various features of the activities are extracted to build a model using the Random Forest technique. In the second phase, the sequences of events are segmented dynamically by considering their recency and sensor correlation. Then, the segments are analyzed by the generated model from the previous phase to recognize both simple and complex activities. We examined the performance of the devised approach using the CASAS dataset. To do this, first we investigated several classifiers. The outcome showed that the combination of emerging patterns and the random forest provide a higher degree of accuracy. Then, we compared CARER with the static window approach, which used Hidden Markov Model. To have a fair comparison, we replaced the dynamic segmentation module of CARER with the static one. The results showed more than 12% improvement in f-measure. Finally, we compared our work with Dynamic sensor segmentation for real-time activity recognition, which used dynamic segmentation. The f-measure metric demonstrated up to 12.73% improvement.     

STAR-Lite: A light-weight scalable self-taught learning framework for older adults’ activity recognition

Recognizing activities for older adults is challenging as we observe a variety of activity patterns caused due to aging (e.g., limited dexterity, limb control, slower response time) or/and underlying health conditions (e.g., dementia). However, existing literature with deep learning methods has successfully recognized activities when the dataset contains high-quality annotations and is captured in a controlled environment. On the contrary, data captured in a real-world environment, especially with older adults exhibiting memory-related symptoms, varying psychological and mental health status, reliance on caregivers to perform daily activities, and unavailability of domain-specific annotators, makes obtaining quality data with annotations challenging; leaving us with limited labeled data and abundant unlabeled data. In this paper, we hypothesize that projecting the labeled data representations comprising a specific set of activities onto a new representation space characterized by the unlabeled data comprising activities beyond the limited activities in the labeled dataset would help us rely less on the annotated data to improve activity detection performance. Motivated by this, we propose STAR-Lite, a self-taught learning framework that involves a pre-training framework to prepare the new representation space considering activities beyond the initial labels in the labeled dataset. STAR-Lite projects the labeled data representations on the new representation space characterized by unlabeled data labels and learns higher-level representations of the labeled dataset while optimizing inter- and intra- class distances without explicitly using a computation hungry similarity-based approach. We demonstrate that our proposed approach, STAR-Lite (a) improves activity recognition performance in a supervised setting and (b) is feasible for real-world deployment. To enhance the feasibility of deploying STAR-Lite on devices with limited memory resources, we explore model compression techniques such as pruning and quantization and propose a novel layer-wise pruning-rate optimization technique that effectively compresses the network while preserving the model performance. The evaluation was performed using the Alzheimer’s Activity Recognition dataset (AAR) captured from 25 individuals living in a retirement community center with IRB approval (#Y18NR12035) using an in-house SenseBox infrastructure while concurrently assessing the clinical evaluation of the participants for dementia, and independent living. Our extensive evaluation reveals that STAR-Lite can detect activities with an F1-score of 85.12% despite 62% reduction in model size and 5% improvement of execution time on a resource constrained device.     

EEM: evolutionary ensembles model for activity recognition in Smart Homes

Activity recognition requires further research to enable a multitude of human-centric applications in the smart home environment. Currently, the major challenges in activity recognition include the domination of major activities over minor activities, their non-deterministic nature and the lack of availability of human-understandable output. In this paper, we introduce a novel Evolutionary Ensembles Model (EEM) that values both minor and major activities by processing each of them independently. It is based on a Genetic Algorithm (GA) to handle the non-deterministic nature of activities. Our evolutionary ensemble learner generates a human-understandable rule profile to ensure a certain level of confidence for performed activities. To evaluate the EEM, we performed experiments on three different real world datasets. Our experiments show significant improvement of 0.6 % to 0.28 % in the F-measures of recognized activities compared to existing counterparts. It is expected that EEM would be a practical solution for the activity recognition problem due to its understandable output and improved accuracy.     

Federated Clustering and Semi-Supervised learning: A new partnership for personalized Human Activity Recognition

Federated Learning (FL) is currently studied by several research groups as a promising paradigm for sensor-based Human Activity Recognition (HAR) to mitigate the privacy and scalability issues of classic centralized approaches. However, in the HAR domain, data is non-independently and identically distributed (non-IID), and personalization is one of the major challenges. Federated Clustering has been recently proposed to mitigate this issue by creating specialized global models for groups of similar users. While this approach significantly improves personalization, it assumes that labeled data are available on each client. In this work, we propose SS-FedCLAR, a novel HAR framework that combines Federated Clustering and Semi-Supervised learning. In SS-FedCLAR , each client uses a combination of active learning and label propagation to provide pseudo labels to a large amount of unlabeled data, which is then used to collaboratively train a Federated Clustering model. We evaluated SS-FedCLAR on two well-known public datasets, showing that it outperforms existing semi-supervised FL solutions while reaching recognition rates similar to fully-supervised FL approaches.     

基于改进粒子群优化的支持向量机与情景感知的人体活动识别

针对目前人体活动类别识别准确率偏低的问题,提出一种支持向量机（SVM）与情景分析（人体运动状态转换的实际逻辑或统计模型）相结合的识别方法,对人体日常的六种活动（步行、上楼、下楼、坐下、站立、躺下）进行识别。该方法利用了人体活动样本之间存在逻辑关系的特点,首先使用经改进的粒子群优化（IPSO）算法对SVM模型进行优化,然后利用优化后的SVM对人体活动进行分类,最后通过情景分析的方法对错误的识别结果进行修正。实验结果表明,所提方法在加州大学欧文分校（UCI）的人体活动识别数据集（HARUS）上的分类准确率达到了94.2%,高于传统的仅使用模式识别进行分类的方法。     

A context-aware ubiquitous learning environment for language listening and speaking

This paper reported the results of a study that aimed to construct a sensor and handheld augmented reality (AR)-supported ubiquitous learning (u-learning) environment called the Handheld English Language Learning Organization (HELLO), which is geared towards enhancing students' language learning. The HELLO integrates sensors, AR, ubiquitous computing and information technologies. It is composed of two subsystems: an English learning management system and a u-learning tool. In order to evaluate the effects of the proposed learning environment on the learning performance of students, a case study on English learning was conducted on a school campus. The participants included high school teachers and students. A learning course entitled 'My Campus' was conducted in the class; it included three activities, namely 'Campus Environment', 'Campus Life' and 'Campus Story'. The evaluation results showed that the proposed HELLO and the learning activities could improve the students' English listening and speaking skills.     

Evaluating human factors in augmented reality systems

Augmented reality (AR) has been part of computer graphics methodology for decades. A number of prototype AR systems have shown the possibilities this paradigm creates. Mixing graphical annotations and objects in a user's view of the surrounding environment offers a powerful metaphor for conveying information about that environment. AR systems' potential still exceeds the practice. In fact, most AR systems remain laboratory prototypes. There are several reasons for this; two of the most prominent are that researchers need more advanced hardware than currently available to implement the systems, and (the subject of this article) the AR research community needs to resolve human factors issues. AR systems are usually interactive; thus, we must verify usability to determine if the system is effective.     

Intelligent Deep Learning Enabled Human Activity Recognition for Improved Medical Services

Human Activity Recognition (HAR) has been made simple in recent years, thanks to recent advancements made in Artificial Intelligence (AI) techniques. These techniques are applied in several areas like security, surveillance, healthcare, human-robot interaction, and entertainment. Since wearable sensor-based HAR system includes in-built sensors, human activities can be categorized based on sensor values. Further, it can also be employed in other applications such as gait diagnosis, observation of children/adult’s cognitive nature, stroke-patient hospital direction, Epilepsy and Parkinson’s disease examination, etc. Recently-developed Artificial Intelligence (AI) techniques, especially Deep Learning (DL) models can be deployed to accomplish effective outcomes on HAR process. With this motivation, the current research paper focuses on designing Intelligent Hyperparameter Tuned Deep Learning-based HAR (IHPTDL-HAR) technique in healthcare environment. The proposed IHPTDL-HAR technique aims at recognizing the human actions in healthcare environment and helps the patients in managing their healthcare service. In addition, the presented model makes use of Hierarchical Clustering (HC)-based outlier detection technique to remove the outliers. IHPTDL-HAR technique incorporates DL-based Deep Belief Network (DBN) model to recognize the activities of users. Moreover, Harris Hawks Optimization (HHO) algorithm is used for hyperparameter tuning of DBN model. Finally, a comprehensive experimental analysis was conducted upon benchmark dataset and the results were examined under different aspects. The experimental results demonstrate that the proposed IHPTDL-HAR technique is a superior performer compared to other recent techniques under different measures.     

Highly-accurate binary tiny neural network for low-power human activity recognition

An ultra low power hardware implementation of Human Activity Recognition systems imposes very tight constraints. Therefore it requires a very thoughtful balancing between highly accurate results and the reduction of the allocated physical resources. Deep Learning models provide the highest accuracies, however their typical computational complexity and memory requirements are not easily deployable in handheld or wearable devices which embody very constrained memory and computation capabilities. In this work, we introduce a new HAR system built with a new Hybrid Binarized Neural Network suitable for a very compact and ultra low-power hardware implementation. The system receives data from MEMS based inertial sensors and makes the acquisitions independent from the sensor spatial orientation and the gravity acceleration signal. The system has been trained and validated on the PAMAP2 and SHL dataset. In both cases, it achieves accuracies higher than 99% in the best case, with different input sensor configurations. A custom circuit has been implemented, which extensively shares circuitry between the different functional sub-modules of the system to minimize the amount of mapped physical resources. The FPGA implementation on a Xilinx Artix 7 achieves a total power dissipation of 72 mW and occupies 6788 LUTs.     

ADR-SPLDA: Activity discovery and recognition by combining sequential patterns and latent Dirichlet allocation

This paper presents ADR-SPLDA, an unsupervised model for human activity discovery and recognition in pervasive environments. The activities are encoded in sequences recorded by non-intrusive sensors placed at various locations in the environment. Our model studies the relationship between the activities and the sequential patterns extracted from the sequences. Activity discovery is formulated as an optimization problem in which sequences are modeled as probability distributions over activities, and activities are, in turn, modeled as probability distributions over sequential patterns. The optimization problem is solved by maximization of the likelihood of data. We present experimental results on real datasets gathered in smart homes where people perform various activities of daily living. The results obtained demonstrate the suitability of our model for activity discovery and characterization. Also, we empirically demonstrate the effectiveness of our model for activity recognition by comparing it with two of the widely used models reported in the literature, the Hidden Markov model and the Conditional Random Field model.     

Non-intrusive sleep pattern recognition with ubiquitous sensing in elderly assistive environment

The quality of sleep may be a reflection of an elderly individual’s health state, and sleep pattern is an important measurement. Recognition of sleep pattern by itself is a challenge issue, especially for elderly-care community, due to both privacy concerns and technical limitations. We propose a novel multi-parametric sensing system called sleep pattern recognition system (SPRS). This system, equipped with a combination of various non-invasive sensors, can monitor an elderly user’s sleep behavior. It accumulates the detecting data from a pressure sensor matrix and ultra wide band (UWB) tags. Based on these two types of complementary sensing data, SPRS can assess the user’s sleep pattern automatically via machine learning algorithms. Compared to existing systems, SPRS operates without disrupting the users’ sleep. It can be used in normal households with minimal deployment. Results of tests in our real assistive apartment at the Smart Elder-care Lab are also presented in this paper.     

A RISC approach to robotics

This article describes a framework that combines simple hardware traditionally used in manufacturing with sensor-based planning and design algorithms from robotics. For repetitive assembly, the authors argue that this combination can reduce start-up and maintenance costs, increase throughput, and greatly reduce the set-up and changeover times for new products. The proposed hardware bears a close resemblance to existing "hard" automation; what is new is the application of computational methods for robust design and control of these systems, and more extensive use of (simple) sensors. Clearly this enhances the capabilities of the hardware. A less-obvious benefit is that software capability is also enhanced--algorithms for fine-motion, grasp planning and some sensing algorithms which would be intractable on a general-purpose robot work in real-time when applied to simple hardware. To describe this approach the authors chose the acronym RISC--Reduced Intricacy in Sensing and Control-by analogy with computer architecture. Analogously, the authors propose to use simple hardware elements that are coordinated by software to perform complex tasks