Federated Clustering and Semi-Supervised learning: A new partnership for personalized Human Activity Recognition

Federated Learning (FL) is currently studied by several research groups as a promising paradigm for sensor-based Human Activity Recognition (HAR) to mitigate the privacy and scalability issues of classic centralized approaches. However, in the HAR domain, data is non-independently and identically distributed (non-IID), and personalization is one of the major challenges. Federated Clustering has been recently proposed to mitigate this issue by creating specialized global models for groups of similar users. While this approach significantly improves personalization, it assumes that labeled data are available on each client. In this work, we propose SS-FedCLAR, a novel HAR framework that combines Federated Clustering and Semi-Supervised learning. In SS-FedCLAR , each client uses a combination of active learning and label propagation to provide pseudo labels to a large amount of unlabeled data, which is then used to collaboratively train a Federated Clustering model. We evaluated SS-FedCLAR on two well-known public datasets, showing that it outperforms existing semi-supervised FL solutions while reaching recognition rates similar to fully-supervised FL approaches.     

A survey on the use of machine learning methods in context-aware middlewares for human activity recognition

Human activity recognition (HAR) essentially uses (past) sensor data or complex context information for inferring the activities a user performs in his daily tasks. HAR has been extensively studied using different paradigms, such as different reasoning mechanisms, including probabilistic, rule-based, statistical, logical reasoning, or the machine learning (ML) paradigm, to construct inference models to recognize or predict user activities. ML for HAR allows that activities can be recognized and even anticipated through the analysis of collected data from different sensors, with greater accuracy than the other paradigms. On the other hand, context-aware middlewares (CAMs) can efficiently integrate a large number of different devices and sensors. Moreover, they provide a programmable and auto-configurable infrastructure for streamline the design and construction of software solutions in scenarios where lots of sensors and data are their bases, such as ambient intelligence, smart cities, and e-health domains. In this way, the full integration of ML capabilities as services in CAMs can advance the development of software solutions in these domains when ML is necessary, specially for HAR, which is the basis for many scenarios in these domains. In this work, we present a survey for identifying the state-of-the-art in using ML for HAR in CAMs through a systematic literature review (SLR). In our SLR, we worked to answer four research questions: (i) what are the different types of context reasoners available in CAMs; (ii) what are the ML algorithms and methods used for generating models for context reasoning; (iii) which CAMs support data processing in real time; and (iv) what are the HAR scenarios usually tackled by the research works. In our analysis, we observed that, although ML offers viable approaches to construct inference models for HAR using different ML approaches, including batch learning, adaptive learning and data stream learning, there are yet some gaps and research challenges to be tackled, specially on the use of data stream learning considering concept drift on data, mechanisms for adapting the inference models, and further considering all of this as services in CAMs, specially for HAR.

     

Human Stress Recognition by Correlating Vision and EEG Data

Because stress has such a powerful impact on human health, we must be able to identify it automatically in our everyday lives. The human activity recognition (HAR) system use data from several kinds of sensors to try to recognize and evaluate human actions automatically recognize and evaluate human actions. Using the multimodal dataset DEAP (Database for Emotion Analysis using Physiological Signals), this paper presents deep learning (DL) technique for effectively detecting human stress. The combination of vision-based and sensor-based approaches for recognizing human stress will help us achieve the increased efficiency of current stress recognition systems and predict probable actions in advance of when fatal. Based on visual and EEG (Electroencephalogram) data, this research aims to enhance the performance and extract the dominating characteristics of stress detection. For the stress identification test, we utilized the DEAP dataset, which included video and EEG data. We also demonstrate that combining video and EEG characteristics may increase overall performance, with the suggested stochastic features providing the most accurate results. In the first step, CNN (Convolutional Neural Network) extracts feature vectors from video frames and EEG data. Feature Level (FL) fusion that combines the features extracted from video and EEG data. We use XGBoost as our classifier model to predict stress, and we put it into action. The stress recognition accuracy of the proposed method is compared to existing methods of Decision Tree (DT), Random Forest (RF), AdaBoost, Linear Discriminant Analysis (LDA), and K-Nearest Neighborhood (KNN). When we compared our technique to existing state-of-the-art approaches, we found that the suggested DL methodology combining multimodal and heterogeneous inputs may improve stress identification.     

注意力迁移的联合平衡领域自适应

目的 现有的图像识别方法应用于从同一分布中提取的训练数据和测试数据时具有良好性能,但这些方法在实际场景中并不适用,从而导致识别精度降低。使用领域自适应方法是解决此类问题的有效途径,领域自适应方法旨在解决来自两个领域相关但分布不同的数据问题。方法 通过对数据分布的分析,提出一种基于注意力迁移的联合平衡自适应方法,将源域有标签数据中提取的图像特征迁移至无标签的目标域。首先,使用注意力迁移机制将有标签源域数据的空间类别信息迁移至无标签的目标域。通过定义卷积神经网络的注意力,使用关注信息来提高图像识别精度。其次,基于目标数据集引入网络参数的先验分布,并且赋予网络自动调整每个领域对齐层特征对齐的能力。最后,通过跨域偏差来描述特定领域的特征对齐层的输入分布,定量地表示每层学习到的领域适应性程度。结果 该方法在数据集Office-31上平均识别准确率为77.6%,在数据集Office-Caltech上平均识别准确率为90.7%,不仅大幅领先于传统手工特征方法,而且取得了与目前最优的方法相当的识别性能。结论 注意力迁移的联合平衡领域自适应方法不仅可以获得较高的识别精度,而且能够自动学习领域间特征的对齐程度,同时也验证了进行域间特征迁移可以提高网络优化效果这一结论。     

基于混合学习算法IHMCAP的故障诊断模型

在故障集和差错属性集的基础上,通过结合了基于概率论的符号学习与神经网络学 习的增量式混合型多概念获取算法IHMCAP寻找属性值与故障类型之间的对应关系,由此 建立一个故障诊断模型.实验表明,该模型不仅精度高、速度快、学习能力强,而且在利用系统 的先验知识与新增数据上也取得了均衡.     

Meta-learning meets the Internet of Things: Graph prototypical models for sensor-based human activity recognition

With the rapid growth of the Internet of Things (IoT), smart systems and applications are equipped with an increasing number of wearable sensors and mobile devices. These sensors are used not only to collect data but, more importantly, to assist in tracking and analyzing the daily human activities. Sensor-based human activity recognition is a hotspot and starts to employ deep learning approaches to supersede traditional shallow learning that rely on hand-crafted features. Although many successful methods have been proposed, there are three challenges to overcome: (1) deep model’s performance overly depends on the data size; (2) deep model cannot explicitly capture abundant sample distribution characteristics; (3) deep model cannot jointly consider sample features, sample distribution characteristics, and the relationship between the two. To address these issues, we propose a meta-learning-based graph prototypical model with priority attention mechanism for sensor-based human activity recognition. This approach learns not only sample features and sample distribution characteristics via meta-learning-based graph prototypical model, but also the embeddings derived from priority attention mechanism that mines and utilizes relations between sample features and sample distribution characteristics. What is more, the knowledge learned through our approach can be seen as a priori applicable to improve the performance for other general reasoning tasks. Experimental results on fourteen datasets demonstrate that the proposed approach significantly outperforms other state-of-the-art methods. On the other hand, experiments of applying our model to two other tasks show that our model effectively supports other recognition tasks related to human activity and improves performance on the datasets of these tasks.     

基于可穿戴传感器的人体活动识别研究综述

人体活动识别（HAR）在医疗、安全、娱乐等方面有着广泛的应用。随着传感器器件的发展,各类能准确采集人体行为活动数据的传感器在手环、手表、手机等可穿戴设备上得到了广泛使用,相比基于视频图像的行为识别方法,基于传感器的行为识别具有成本低、灵活、可移植性好的特点,因此,基于可穿戴传感器的人体活动识别研究成为行为识别中的研究热点。介绍了人体活动识别研究中原始数据采集、特征提取、特征选择以及分类方法,对识别流程中每一部分常用的技术以及研究现状进行了综述总结,最后分析人体活动识别研究当前存在的主要问题并展望了今后可能的研究方向。     

基于BLSTM的命名实体识别方法

传统的命名实体识别方法直接依靠大量的人工特征和专门的领域知识,解决了监督学习语料不足的问题,但设计人工特征和获取领域知识的代价昂贵。针对该问题,提出一种基于BLSTM(Bidirectional Long Short-Term Memory)的神经网络结构的命名实体识别方法。该方法不再直接依赖于人工特征和领域知识,而是利用基于上下文的词向量和基于字的词向量,前者表达命名实体的上下文信息,后者表达构成命名实体的前缀、后缀和领域信息;同时,利用标注序列中标签之间的相关性对BLSTM的代价函数进行约束,并将领域知识嵌入模型的代价函数中,进一步增强模型的识别能力。实验表明,所提方法的识别效果优于传统方法。     

基于迁移学习的讽刺检测

准确的讽刺检测对于情感分析等任务至关重要.传统的方法严重依赖于离散的人工制定的特征.现有的研究大多将讽刺检测作为一种标准的监督学习文本分类任务,但是监督学习需要有大量数据,而这些数据的收集和标注都存在困难.由于目标任务有限的数据集可能导致讽刺检测的低性能,为此将讽刺检测作为一种迁移学习任务,将讽刺标记文本的监督学习与外部分析资源的知识转移相结合.通过转移的资源知识来改进神经网络模型,以此提升对目标任务的检测性能.在公开可用的数据集上的实验结果表明,提出的基于迁移学习的讽刺检测模型优于现有较先进的讽刺检测模型.     

Common-sense reasoning for human action recognition

This paper presents a novel method that leverages reasoning capabilities in a computer vision system dedicated to human action recognition. The proposed methodology is decomposed into two stages. First, a machine learning based algorithm – known as bag of words – gives a first estimate of action classification from video sequences, by performing an image feature analysis. Those results are afterward passed to a common-sense reasoning system, which analyses, selects and corrects the initial estimation yielded by the machine learning algorithm. This second stage resorts to the knowledge implicit in the rationality that motivates human behaviour. Experiments are performed in realistic conditions, where poor recognition rates by the machine learning techniques are significantly improved by the second stage in which common-sense knowledge and reasoning capabilities have been leveraged. This demonstrates the value of integrating common-sense capabilities into a computer vision pipeline.     

Cascade ARTMAP: integrating neural computation and symbolic knowledge processing

This paper introduces a hybrid system termed cascade adaptive resonance theory mapping (ARTMAP) that incorporates symbolic knowledge into neural-network learning and recognition. Cascade ARTMAP, a generalization of fuzzy ARTMAP, represents intermediate attributes and rule cascades of rule-based knowledge explicitly and performs multistep inferencing. A rule insertion algorithm translates if-then symbolic rules into cascade ARTMAP architecture. Besides that initializing networks with prior knowledge can improve predictive accuracy and learning efficiency, the inserted symbolic knowledge can be refined and enhanced by the cascade ARTMAP learning algorithm. By preserving symbolic rule form during learning, the rules extracted from cascade ARTMAP can be compared directly with the originally inserted rules. Simulations on an animal identification problem indicate that a priori symbolic knowledge always improves system performance, especially with a small training set. Benchmark study on a DNA promoter recognition problem shows that with the added advantage of fast learning, cascade ARTMAP rule insertion and refinement algorithms produce performance superior to those of other machine learning systems and an alternative hybrid system known as knowledge-based artificial neural network (KBANN). Also, the rules extracted from cascade ARTMAP are more accurate and much cleaner than the NofM rules extracted from KBANN.     

Knowledge discovery by inductive neural networks

A new neural network model for inducing symbolic knowledge from empirical data is presented. This model capitalizes on the fact that the certainty factor-based activation function can improve the network generalization performance from a limited amount of training data. The formal properties of the procedure for extracting symbolic knowledge from such a trained neural network are investigated. In the domain of molecular genetics, a case study demonstrated that the described learning system effectively discovered the prior domain knowledge with some degree of refinement. Also, in cross-validation experiments, the system outperformed C4.5, a commonly used rule learning system     

OntoILPER: an ontology- and inductive logic programming-based system to extract entities and relations from text

Named entity recognition (NER) and relation extraction (RE) are two important subtasks in information extraction (IE). Most of the current learning methods for NER and RE rely on supervised machine learning techniques with more accurate results for NER than RE. This paper presents OntoILPER a system for extracting entity and relation instances from unstructured texts using ontology and inductive logic programming, a symbolic machine learning technique. OntoILPER uses the domain ontology and takes advantage of a higher expressive relational hypothesis space for representing examples whose structure is relevant to IE. It induces extraction rules that subsume examples of entities and relation instances from a specific graph-based model of sentence representation. Furthermore, OntoILPER enables the exploitation of the domain ontology and further background knowledge in the form of relational features. To evaluate OntoILPER, several experiments over the TREC corpus for both NER and RE tasks were conducted and the yielded results demonstrate its effectiveness in both tasks. This paper also provides a comparative assessment among OntoILPER and other NER and RE systems, showing that OntoILPER is very competitive on NER and outperforms the selected systems on RE.     

一种改进的GRU-InFCN人体行为识别模型

为了克服传统机器学习方法在采用传感器数据进行人体行为识别领域上识别效果对人工特征选取依赖严重、识别准确率不高等问题,提出一种改进的全卷积神经网络和多层循环神经网络并联的深度学习模型(GRU-InFCN),并对传感器数据特征进行自动提取,实现人体动作的识别。该模型通过多尺度卷积神经网络和双层GRU网络(Gated Recurrent Unit,GRU)分别对传感器数据进行特征提取,将特征矩阵在矩阵维度上进行特征拼接再通过Softmax完成特征分类。实验结果表明,在开源人体行为识别(HAR)数据集上采用该方法进行人体行为识别,准确率达到了97.76%。该模型在取得高准确率的同时,避免了复杂的信号预处理和特征工程。     

Learning Kernels for Unsupervised Domain Adaptation with Applications to Visual Object Recognition

Domain adaptation aims to correct the mismatch in statistical properties between the source domain on which a classifier is trained and the target domain to which the classifier is to be applied. In this paper, we address the challenging scenario of unsupervised domain adaptation, where the target domain does not provide any annotated data to assist in adapting the classifier. Our strategy is to learn robust features which are resilient to the mismatch across domains and then use them to construct classifiers that will perform well on the target domain. To this end, we propose novel kernel learning approaches to infer such features for adaptation. Concretely, we explore two closely related directions. In the first direction, we propose unsupervised learning of a geodesic flow kernel (GFK). The GFK summarizes the inner products in an infinite sequence of feature subspaces that smoothly interpolates between the source and target domains. In the second direction, we propose supervised learning of a kernel that discriminatively combines multiple base GFKs. Those base kernels model the source and the target domains at fine-grained granularities. In particular, each base kernel pivots on a different set of landmarks—the most useful data instances that reveal the similarity between the source and the target domains, thus bridging them to achieve adaptation. Our approaches are computationally convenient, automatically infer important hyper-parameters, and are capable of learning features and classifiers discriminatively without demanding labeled data from the target domain. In extensive empirical studies on standard benchmark recognition datasets, our appraches yield state-of-the-art results compared to a variety of competing methods.     

多源域混淆的双流深度迁移学习

目的 针对深度学习严重依赖大样本的问题,提出多源域混淆的双流深度迁移学习方法,提升了传统深度迁移学习中迁移特征的适用性。方法 采用多源域的迁移策略,增大源域对目标域迁移特征的覆盖率。提出两阶段适配学习的方法,获得域不变的深层特征表示和域间分类器相似的识别结果,将自然光图像2维特征和深度图像3维特征进行融合,提高小样本数据特征维度的同时抑制了复杂背景对目标识别的干扰。此外,为改善小样本机器学习中分类器的识别性能,在传统的softmax损失中引入中心损失,增强分类损失函数的惩罚监督能力。结果 在公开的少量手势样本数据集上进行对比实验,结果表明,相对于传统的识别模型和迁移模型,基于本文模型进行识别准确率更高,在以DenseNet-169为预训练网络的模型中,识别率达到了97.17%。结论 利用多源域数据集、两阶段适配学习、双流卷积融合以及复合损失函数,构建了多源域混淆的双流深度迁移学习模型。所提模型可增大源域和目标域的数据分布匹配率、丰富目标样本特征维度、提升损失函数的监督性能,改进任意小样本场景迁移特征的适用性。     

基于多尺度特征多对抗网络的雾天图像识别

目的 当前的大型数据集,例如ImageNet,以及一些主流的网络模型,如ResNet等能直接高效地应用于正常场景的分类,但在雾天场景下则会出现较大的精度损失。雾天场景复杂多样,大量标注雾天数据成本过高,在现有条件下,高效地利用大量已有场景的标注数据和网络模型完成雾天场景下的分类识别任务至关重要。方法 本文使用了一种低成本的数据增强方法,有效减小图像在像素域上的差异。基于特征多样性和特征对抗的思想,提出多尺度特征多对抗网络,通过提取数据的多尺度特征,增强特征在特征域分布的代表性,利用对抗机制,在多个特征上减少特征域上的分布差异。通过缩小像素域和特征域分布差异,进一步减小领域偏移,提升雾天场景的分类识别精度。结果 在真实的多样性雾天场景数据上,通过消融实验,使用像素域数据增强方法后,带有标签的清晰图像数据在风格上更趋向于带雾图像,总的分类精度提升了8.2%,相比其他的数据增强方法,至少提升了6.3%,同时在特征域上使用多尺度特征多对抗网络,相比其他的网络,准确率至少提升了8.0%。结论 像素域数据增强以及多尺度特征多对抗网络结合的雾天图像识别方法,综合考虑了像素域和特征域的领域分布差异,结合了多尺度的丰富特征信息,同时使用多对抗来缩小雾天数据的领域偏移,在真实多样性雾天数据集上获得了更好的图像分类识别效果。     

对抗学习遥感图像场景识别

目的 在高分辨率遥感图像场景识别问题中,经典的监督机器学习算法大多需要充足的标记样本训练模型,而获取遥感图像的标注费时费力。为解决遥感图像场景识别中标记样本缺乏且不同数据集无法共享标记样本问题,提出一种结合对抗学习与变分自动编码机的迁移学习网络。方法 利用变分自动编码机（variational auto-encoders,VAE）在源域数据集上进行训练,分别获得编码器和分类器网络参数,并用源域编码器网络参数初始化目标域编码器。采用对抗学习的思想,引入判别网络,交替训练并更新目标域编码器与判别网络参数,使目标域与源域编码器提取的特征尽量相似,从而实现遥感图像源域到目标域的特征迁移。结果 利用两个遥感场景识别数据集进行实验,验证特征迁移算法的有效性,同时尝试利用SUN397自然场景数据集与遥感场景间的迁移识别,采用相关性对齐以及均衡分布适应两种迁移学习方法作为对比。两组遥感场景数据集间的实验中,相比于仅利用源域样本训练的网络,经过迁移学习后的网络场景识别精度提升约10%,利用少量目标域标记样本后提升更为明显;与对照实验结果相比,利用少量目标域标记样本时提出方法的识别精度提升均在3%之上,仅利用源域标记样本时提出方法场景识别精度提升了10%~40%;利用自然场景数据集时,方法仍能在一定程度上提升场景识别精度。结论 本文提出的对抗迁移学习网络可以在目标域样本缺乏的条件下,充分利用其他数据集中的样本信息,实现不同场景图像数据集间的特征迁移及场景识别,有效提升遥感图像的场景识别精度。     

民航突发事件实体识别方法研究

针对民航突发事件实体自动获取的问题,提出一种将双向长短期记忆网络(BiLSTM)与条件随机场(CRF)结合的民航突发事件实体识别方法。利用领域词典对民航突发事件实体进行自动标注,并将已标注的数据转化为分布式字符向量;采用BiLSTM模型获取序列文本的上下文特征;通过CRF模型获得序列标注结果。实验结果表明,与传统的CRF方法相比,该方法的F值有明显的提升,有效解决了传统方法在识别民航突发事件文本信息中包含复合实体或混合实体方面效率低且大量使用人工定义特征模板的问题,为民航突发事件知识图谱的自动构建提供了更好的方法支撑。