Unsupervised learning for human activity recognition using smartphone sensors

To provide more sophisticated healthcare services, it is necessary to collect the precise information on a patient. One impressive area of study to obtain meaningful information is human activity recognition, which has proceeded through the use of supervised learning techniques in recent decades. Previous studies, however, have suffered from generating a training dataset and extending the number of activities to be recognized. In this paper, to find out a new approach that avoids these problems, we propose unsupervised learning methods for human activity recognition, with sensor data collected from smartphone sensors even when the number of activities is unknown. Experiment results show that the mixture of Gaussian exactly distinguishes those activities when the number of activities k is known, while hierarchical clustering or DBSCAN achieve above 90% accuracy by obtaining k based on Caliński–Harabasz index, or by choosing appropriate values for ɛ and MinPts when k is unknown. We believe that the results of our approach provide a way of automatically selecting an appropriate value of k at which the accuracy is maximized for activity recognition, without the generation of training datasets by hand.     

基于智能手机传感器数据的人类行为识别研究

运用智能手机传感器数据进行人类行为识别研究在医疗服务、智能环境和网络空间安全等领域有许多重要应用。目前,大多数的分类方法识别率都不高,尤其是在医疗服务领域。为了提高行为活动的识别准确率,先利用稀疏局部保持投影降维,将实验的数据集进行特征约简得到最优的实验特征子集,再用随机森林集成分类器完成了人类行为识别。实验结果证明,该方法不仅明显地降低了实验的特征数量,而且提高了整体精确度。     

Cell tracking using deep neural networks with multi-task learning

Cell tracking plays crucial role in biomedical and computer vision areas. As cells generally have frequent deformation activities and small sizes in microscope image, tracking the non-rigid and non-significant cells is quite difficult in practice. Traditional visual tracking methods have good performances on tracking rigid and significant visual objects, however, they are not suitable for cell tracking problem. In this paper, a novel cell tracking method is proposed by using Convolutional Neural Networks (CNNs) as well as multi-task learning (MTL) techniques. The CNNs learn robust cell features and MTL improves the generalization performance of the tracking. The proposed cell tracking method consists of a particle filter motion model, a multi-task learning observation model, and an optimized model update strategy. In the training procedure, the cell tracking is divided into an online tracking task and an accompanying classification task using the MTL technique. The observation model is trained by building a CNN to learn robust cell features. The tracking procedure is started by assigning the cell position in the first frame of a microscope image sequence. Then, the particle filter model is applied to produce a set of candidate bounding boxes in the subsequent frames. The trained observation model provides the confidence probabilities corresponding to all of the candidates and selects the candidate with the highest probability as the final prediction. Finally, an optimized model update strategy is proposed to enable the multi-task observation model for the variation of the tracked cell over the entire tracking procedure. The performance and robustness of the proposed method are analyzed by comparing with other commonly-used methods. Experimental results demonstrate that the proposed method has good performance to the cell tracking problem.     

Unrestricted deep metric learning using neural networks interaction

Pattern Analysis and Applications - In many machine learning applications and algorithms, the algorithm performance and accuracy are highly dependent on the metric used to measure the distance...     

Pedestrian detection with unsupervised multispectral feature learning using deep neural networks

Multispectral pedestrian detection is an important functionality in various computer vision applications such as robot sensing, security surveillance, and autonomous driving. In this paper, our motivation is to automatically adapt a generic pedestrian detector trained in a visible source domain to a new multispectral target domain without any manual annotation efforts. For this purpose, we present an auto-annotation framework to iteratively label pedestrian instances in visible and thermal channels by leveraging the complementary information of multispectral data. A distinct target is temporally tracked through image sequences to generate more confident labels. The predicted pedestrians in two individual channels are merged through a label fusion scheme to generate multispectral pedestrian annotations. The obtained annotations are then fed to a two-stream region proposal network (TS-RPN) to learn the multispectral features on both visible and thermal images for robust pedestrian detection. Experimental results on KAIST multispectral dataset show that our proposed unsupervised approach using auto-annotated training data can achieve performance comparable to state-of-the-art deep neural networks (DNNs) based pedestrian detectors trained using manual labels.     

Image interpolation using convolutional neural networks with deep recursive residual learning

Multimedia Tools and Applications - Recent developments of image super-resolution often utilize the deep convolutional neural network (CNN) and residual learning to relate the observed...     

Indoor corner recognition from crowdsourced trajectories using smartphone sensors

Recently, fingerprint crowdsourcing from pedestrian movement trajectories has been promoted to alleviate the site survey burden for radio map construction in fingerprinting-based indoor localization. Indoor corners, as one of the most common indoor landmarks, play an important role in movement trajectory analysis. This paper studies the problem of indoor corner recognition in crowdsourced movement trajectories. In a movement trajectory, smartphone internal sensor measurements experience some signal changes when passing by a corner. However, the state-of-the-art solutions based on signal change detection cannot well deal with the fake corner problem and pose diversity problem in most practical movement trajectories. In this paper, we study the corner recognition problem from an expert system viewpoint by applying machine learning techniques. In particular, we extract recognition features from both the time and frequency domain and propose a hierarchical corner recognition scheme consisting of three classifiers. The first pose classifier is to classify various poses into only two groups according to whether or not a smartphone is kept in a fixed position relative to a user upper body when collecting sensor measurements. Feature selection is then applied to train two corner classifiers each for one pose group. Field experiments are conducted to compare our proposed scheme with three state-of-the-art algorithms. In all cases, our scheme outperforms the best of these algorithms in terms of much higher F1-measure and precision for corner recognition. The results also provide insights on the potentials of using more advanced techniques from expert systems in indoor localization.     

A study of deep neural networks for human activity recognition

Human activity recognition and deep learning are two fields that have attracted attention in recent years. The former due to its relevance in many application domains, such as ambient assisted living or health monitoring, and the latter for its recent and excellent performance achievements in different domains of application such as image and speech recognition. In this article, an extensive analysis among the most suited deep learning architectures for activity recognition is conducted to compare its performance in terms of accuracy, speed, and memory requirements. In particular, convolutional neural networks (CNN), long short-term memory networks (LSTM), bidirectional LSTM (biLSTM), gated recurrent unit networks (GRU), and deep belief networks (DBN) have been tested on a total of 10 publicly available datasets, with different sensors, sets of activities, and sampling rates. All tests have been designed under a multimodal approach to take advantage of synchronized raw sensor' signals. Results show that CNNs are efficient at capturing local temporal dependencies of activity signals, as well as at identifying correlations among sensors. Their performance in activity classification is comparable with, and in most cases better than, the performance of recurrent models. Their faster response and lower memory footprint make them the architecture of choice for wearable and IoT devices.     

Feature matching and instance reweighting with transfer learning for human activity recognition using smartphone

Human activity recognition using smartphone has been attracting great interest. Since collecting large amount of labeled data is expensive and time-consuming for conventional machine learning techniques, transfer learning techniques have been proposed for activity recognition. However, existing transfer learning techniques typically rely on feature matching based on global domain shift and lack considering the intra-class knowledge transfer. In this paper, a novel transfer learning technique is proposed for cross-domain activity recognition, which can properly integrate feature matching and instance reweighting across the source and target domain in principled dimensionality reduction. The experiments using three real datasets demonstrate that the proposed scheme can achieve much higher precision (92%), recall (91%), and F1-score (92%), in comparison with the existing schemes.

     

Human activity recognition by combining external features with accelerometer sensor data using deep learning network model

Multimedia Tools and Applications - Various Human Activities are classified through time-series data generated by the sensors of wearable devices. Many real-time scenarios such as Healthcare...     

基于随机Dropout深度信念网络的移动用户行为识别方法

针对移动用户行为识别模型中存在过度拟合导致泛化性差的问题,提出一种基于随机Dropout深度信念网络DBN(Deep Belief Network)的移动用户行为识别方法,该方法通过随机更改Dropout算法中的概率参数,减少隐层单元的网络节点数,优化每次训练的网络权值,以提高行为识别的准确率和样本较少时的泛化能力。实验结果表明,加入随机Dropout的网络对静止、散步、跑步、上楼及下楼五种行为的平均识别准确率可达94.23%,相对于传统的DBN识别方法,准确率提高了4.57%。     

Human action recognition using three orthogonal planes with unsupervised deep convolutional neural network

Deep learning models have attained great success for an extensive range of computer vision applications including image and video classification. However, the complex architecture of the most recently developed networks imposes certain memory and computational resource limitations, especially for human action recognition applications. Unsupervised deep convolutional neural networks such as PCANet can alleviate these limitations and hence significantly reduce the computational complexity of the whole recognition system. In this work, instead of using 3D convolutional neural network architecture to learn temporal features of video actions, the unsupervised convolutional PCANet model is extended into (PCANet-TOP) which effectively learn spatiotemporal features from Three Orthogonal Planes (TOP). For each video sequence, spatial frames (XY) and temporal planes (XT and YT) are utilized to train three different PCANet models. Then, the learned features are fused after reducing their dimensionality using whitening PCA to obtain spatiotemporal feature representation of the action video. Finally, Support Vector Machine (SVM) classifier is applied for action classification process. The proposed method is evaluated on four benchmarks and well-known datasets, namely, Weizmann, KTH, UCF Sports, and YouTube action datasets. The recognition results show that the proposed PCANet-TOP provides discriminative and complementary features using three orthogonal planes and able to achieve promising and comparable results with state-of-the-art methods.

     

Towards unsupervised physical activity recognition using smartphone accelerometers

Multimedia Tools and Applications - The development of smartphones equipped with accelerometers gives a promising way for researchers to accurately recognize an individual’s physical activity...     

Human activity recognition using robust adaptive privileged probabilistic learning

Pattern Analysis and Applications - In this work, a supervised probabilistic approach is proposed that integrates the learning using privileged information (LUPI) paradigm into a hidden conditional...     

Deep learning based speed bumps detection and characterization using smartphone sensors

In recent years lot of research has been done on road surface anomaly detection due to the widespread availability of smartphones, pre-equipped with diverse sensors. The existing literature is mainly focused on speed bump, pot-hole and man-hole and does not provide any information about the type of bumps. Speed bumps have four major types i.e. sinusoidal-profile, flat-top, thermoplastic and round-top. The decision of deploying a particular type is made based upon the location and the speed limit in that area. In this paper we not only detect speed bumps but also identify the bump types. We first collected the speed bump dataset from smartphone sensors and performed series of dataset transformations. Using this dataset we proposed and experimented with deep learning-based speed bump detection and characterization system which is able to achieve the test accuracy of 98.92% and 95% respectively. This work can, not only help the government and policymakers to identify illegal speed bumps but also help decrease the number of road accidents, avoid damage to the vehicles, and reduce the environmental, health as well as financial losses due to non-standard or sub-standard bump types. Moreover, this work can be integrated into road navigation apps like Google Maps, Waze, etc. which can help determine the optimal routes for the drivers.     

Human activity recognition using multi-features and multiple kernel learning

This paper presents two sets of features, shape representation and kinematic structure, for human activity recognition using a sequence of RGB-D images. The shape features are extracted using the depth information in the frequency domain via spherical harmonics representation. The other features include the motion of the 3D joint positions (i.e. the end points of the distal limb segments) in the human body. Both sets of features are fused using the Multiple Kernel Learning (MKL) technique at the kernel level for human activity recognition. Our experiments on three publicly available datasets demonstrate that the proposed features are robust for human activity recognition and particularly when there are similarities among the actions.     

Latent feature learning for activity recognition using simple sensors in smart homes

Multimedia Tools and Applications - Activity recognition is an important step towards monitoring and evaluating the functional health of an individual, and it potentially promotes human-centric...     

基于深度学习的人体动作识别方法

针对人体动作深度视频的四维信息映射到二维空间后,动作分类容易发生混淆的问题,提出一种基于深度学习的人体动作识别方法。首先构建空间结构动态深度图,将深度视频的四维信息映射到二维空间,进行信息降维处理;然后提出基于联合代价函数的深度卷积神经网络,结合交叉熵损失函数与中心损失函数作为联合代价函数,指导卷积层学习到更具分辨力的深度特征,以进行更精确的分类。在MSRDailyActivity3D和SYSU 3D HOI两个数据集的实验结果表明,与现有方法相比,该方法识别率得到了较明显的提升,验证了其有效性和鲁棒性。该方法较好地解决了动作分类容易发生混淆的问题。