Meta-learning meets the Internet of Things: Graph prototypical models for sensor-based human activity recognition

With the rapid growth of the Internet of Things (IoT), smart systems and applications are equipped with an increasing number of wearable sensors and mobile devices. These sensors are used not only to collect data but, more importantly, to assist in tracking and analyzing the daily human activities. Sensor-based human activity recognition is a hotspot and starts to employ deep learning approaches to supersede traditional shallow learning that rely on hand-crafted features. Although many successful methods have been proposed, there are three challenges to overcome: (1) deep model’s performance overly depends on the data size; (2) deep model cannot explicitly capture abundant sample distribution characteristics; (3) deep model cannot jointly consider sample features, sample distribution characteristics, and the relationship between the two. To address these issues, we propose a meta-learning-based graph prototypical model with priority attention mechanism for sensor-based human activity recognition. This approach learns not only sample features and sample distribution characteristics via meta-learning-based graph prototypical model, but also the embeddings derived from priority attention mechanism that mines and utilizes relations between sample features and sample distribution characteristics. What is more, the knowledge learned through our approach can be seen as a priori applicable to improve the performance for other general reasoning tasks. Experimental results on fourteen datasets demonstrate that the proposed approach significantly outperforms other state-of-the-art methods. On the other hand, experiments of applying our model to two other tasks show that our model effectively supports other recognition tasks related to human activity and improves performance on the datasets of these tasks.     

融合知识表征的多模态Transformer场景文本视觉问答

目的 现有视觉问答方法通常只关注图像中的视觉物体,忽略了对图像中关键文本内容的理解,从而限制了图像内容理解的深度和精度。鉴于图像中隐含的文本信息对理解图像的重要性,学者提出了针对图像中场景文本理解的“场景文本视觉问答”任务以量化模型对场景文字的理解能力,并构建相应的基准评测数据集TextVQA(text visual question answering)和ST-VQA(scene text visual question answering)。本文聚焦场景文本视觉问答任务,针对现有基于自注意力模型的方法存在过拟合风险导致的性能瓶颈问题,提出一种融合知识表征的多模态Transformer的场景文本视觉问答方法,有效提升了模型的稳健性和准确性。方法 对现有基线模型M4C(multimodal multi-copy mesh)进行改进,针对视觉对象间的“空间关联”和文本单词间的“语义关联”这两种互补的先验知识进行建模,并在此基础上设计了一种通用的知识表征增强注意力模块以实现对两种关系的统一编码表达,得到知识表征增强的KR-M4C(knowledge-representation-enhanced M4C)方法。结果 在TextVQA和ST-VQA两个场景文本视觉问答基准评测集上,将本文KR-M4C方法与最新方法进行比较。本文方法在TextVQA数据集中,相比于对比方法中最好的结果,在不增加额外训练数据的情况下,测试集准确率提升2.4%,在增加ST-VQA数据集作为训练数据的情况下,测试集准确率提升1.1%;在ST-VQA数据集中,相比于对比方法中最好的结果,测试集的平均归一化Levenshtein相似度提升5%。同时,在TextVQA数据集中进行对比实验以验证两种先验知识的有效性,结果表明提出的KR-M4C模型提高了预测答案的准确率。结论 本文提出的KR-M4C方法的性能在TextVQA和ST-VQA两个场景文本视觉问答基准评测集上均有显著提升,获得了在该任务上的最好结果。     

Knowledge engineering issues in developing a case-based reasoning application

Case-Based Reasoning (CBR) has emerged from research in cognitive psychology as a model of human memory and remembering. It has been embraced by researchers of AI applications as a methodology that avoids some of the knowledge acquisition and reasoning problems that occur with other methods for developing knowledge-based systems. In this paper we propose that, in developing knowledge based systems, knowledge engineering addresses two tasks. There is a problem analysis task that produces the problem representation and there is the task of developing the inference mechanism. CBR has an impact on the second of these tasks but helps less with the first. We argue that in some domains this problem analysis process can be significant and propose an iterative methodology for addressing it. To evaluate this, we describe the application of case-based reasoning to the problem of aircraft conflict resolution in a system called ISAC. We describe the application of this iterative methodology and assess the knowledge engineering impact of CBR.     

Adaptive case-based reasoning using retention and forgetting strategies

Case-based reasoning systems need to maintain their case base in order to avoid performance degradation. Degradation mainly results from memory swamping or exposure to harmful experiences and so, it becomes vital to keep a compact, competent case base. This paper proposes an adaptive case-based reasoning model that develops the case base during the reasoning cycle by adding and removing cases. The rationale behind this approach is that a case base should develop over time in the same way that a human being evolves her overall knowledge: by incorporating new useful experiences and forgetting invaluable ones. Accordingly, our adaptive case-based reasoning model evolves the case base by using a measure of “case goodness” in different retention and forgetting strategies. This paper presents empirical studies of how the combination of this new goodness measure and our adaptive model improves three different performance measures: classification accuracy, efficiency and case base size.     

基于知识话题模型的文本蕴涵识别

该文分析了现有基于分类策略的文本蕴涵识别方法的问题,并提出了一种基于知识话题模型的文本蕴涵分类识别方法。 其假设是: 文本可看作是语义关系的组合,这些语义关系构成若干话题;若即若文本T蕴涵假设H,说明 T 和 H 具有相似的话题分布,反之说明T 和 H 不具有相似的话题分布。基于此,我们将 T 和 H 的蕴涵识别问题转化为相关话题的生成过程,同时将文本推理知识融入到抽样过程,由此建立一个面向文本蕴涵识别的话题模型。实验结果表明基于知识话题模型在一定程度上改进了文本蕴涵识别系统的性能。     

A hybrid reasoning mechanism for effective sensor selection for tasks

In this paper, we present Ontological Logic Programming (OLP), a novel approach that combines logic programming with ontological reasoning. OLP enables the use of ontological terms (i.e., individuals, classes and properties) directly within logic programmes. The interpretation of these terms is delegated to an ontology reasoner during the interpretation of the programme. Unlike similar approaches, OLP makes use of the full capacity of both ontological reasoning and logic programming. We evaluate the computational properties of OLP in different settings and show that its performance can be significantly improved using caching mechanisms. We then introduce a comprehensive sensor-task selection solution based on OLP and discuss the benefits one can obtain by using OLP. The solution is based on a set of interlinking ontologies that capture the crucial domain knowledge of sensor networks. We then make use of OLP to create and manage complex concepts in the domain as well as to implement effective resource-task assignment algorithms, which compute appropriate resources for tasks such that they sufficiently cover the tasks needs. We compare the advantages of OLP with a knowledge-based set-covering mechanism for resource-task selection.     

基于综合推理的构图知识生成模型

讨论了一种集成图案拓扑结构和元素表达内容的构图设计知识的表达模型,研究并提出一种应用综合推理的思想来生成新的构图设计知识的推理模型。这种方法可以直接从原有构图设计知识的形象信息出发,根据一定的初始要求,迅速综合生成新的构图设计知识,大大提高了知识的生成速度,使得智能图案设计系统具有更高的智能性与创新能力。     

多模态引导的局部特征选择小样本学习方法

深度学习模型取得了令人瞩目的成绩,但其训练依赖于大量的标注样本,在标注样本匮乏的场景下模型表现不尽人意.针对这一问题,近年来以研究如何从少量样本快速学习的小样本学习被提了出来,方法主要采用元学习方式对模型进行训练,取得了不错的学习效果.但现有方法:1)通常仅基于样本的视觉特征来识别新类别,信息源较为单一; 2)元学习的使用使得模型从大量相似的小样本任务中学习通用的、可迁移的知识,不可避免地导致模型特征空间趋于一般化,存在样本特征表达不充分、不准确的问题.为解决上述问题,将预训练技术和多模态学习技术引入小样本学习过程,提出基于多模态引导的局部特征选择小样本学习方法.所提方法首先在包含大量样本的已知类别上进行模型预训练,旨在提升模型的特征表达能力;而后在元学习阶段,方法利用元学习对模型进行进一步优化,旨在提升模型的迁移能力或对小样本环境的适应能力,所提方法同时基于样本的视觉特征和文本特征进行局部特征选择来提升样本特征的表达能力,以避免元学习过程中模型特征表达能力的大幅下降;最后所提方法利用选择后的样本特征进行小样本学习.在MiniImageNet、CIFAR-FS和FC-100这3个基准数...     

融合图像注意力的多模态机器翻译模型

已有工作表明,融入图像视觉语义信息可以提升文本机器翻译模型的效果。已有的工作多数将图片的整体视觉语义信息融入到翻译模型,而图片中可能包含不同的语义对象,并且这些不同的局部语义对象对解码端单词的预测具有不同程度的影响和作用。基于此,该文提出一种融合图像注意力的多模态机器翻译模型,将图片中的全局语义和不同部分的局部语义信息与源语言文本的交互信息作为图像注意力融合到文本注意力权重中,从而进一步增强解码端隐含状态与源语言文本的对齐信息。在多模态机器翻译数据集Multi30k上英语—德语翻译对以及人工标注的印尼语—汉语翻译对上的实验结果表明,该文提出的模型相比已有的基于循环神经网络的多模态机器翻译模型效果具有较好的提升,证明了该模型的有效性。     

Model-based remotely-sensed imagery interpretation

Abstract

We present a model-based remotely-sensed image interpretation expert system embeded in a knowledge-based geographic information system (K. BIS). The KBIS consists of four sub-systems: a pictorial data base system, an image interpretation expert system, a computer-aided planning system and a computer-aided cartographic system. The image interpretation expert system represents ecological knowledge and other expert knowledge by frames. Its reasoning process consists of a forward reasoning based on the Bayes classification of Landsat imagery, a backward reasoning using frame knowledge and reasoning using a spatial consistency model. A forest inventory study was conducted in Shaxian county, in the southern part of China, using this expert system. The results have shown a significant improvement. Building image interpretation expert systems within knowledge-based pictorial systems is very convenient and efficient because there are well-organized data, knowledge and procedures available.     

Blackboard推理方法的应用与实现

针对电子商务订单实时智能处理控制问题，采用人工智能领域复杂问题推理的黑板(blackboard)方法，组织电子商务订单实时智能处理控制与推理。考虑物流配送能力的电子商务订单实时智能处理是基于知识的多任务处理。为此，提出基于blackboard的推理处理方法，解决订单处理过程中涉及到的人工处理问题的经验和知识的运用和调度。该研究有助于提高电子商务订单处理的实时性、科学性和智能性。     

新的动态记忆网络的视觉问答

视觉问答任务旨在给机器输入一幅图像和一相关问题,计算机能够准确作答。针对这一任务,对记忆和注意力机制的神经网络结构进行了深入研究,这类网络显示出问题回答所需的某些推理能力。在分析动态记忆网络的基础上,提出了一种新的动态记忆网络,对原来的DMN的内存和输入模块进行改进。结合这些变化,一个新的图像输入模块引入到视觉问答系统中。在DAQUAR-ALL、COCO-QA和VQA数据集上验证了该方法的有效性。实验结果表明,所提出的新动态记忆模型取得了很好的结果,比一些经典深度方法都更出色。     

Knowledge-based treatment planning for adolescent early intervention of mental healthcare: a hybrid case-based reasoning approach

Abstract: Treatment planning is a crucial and complex task in the social services industry. There is an increasing need for knowledge-based systems for supporting caseworkers in the decision-making of treatment planning. This paper presents a hybrid case-based reasoning approach for building a knowledge-based treatment planning system for adolescent early intervention of mental healthcare. The hybrid case-based reasoning approach combines aspects of case-based reasoning, rule-based reasoning and fuzzy theory. The knowledge base of case-based reasoning is a case base of client records consisting of documented experience while that for rule-based reasoning is a set of IF–THEN rules based on the experience of social service professionals. Fuzzy theory is adopted to deal with the uncertain nature of treatment planning. A prototype system has been implemented in a social services company and its performance is evaluated by a group of caseworkers. The results indicate that hybrid case-based reasoning has an enhanced performance and the knowledge-based treatment planning system enables caseworkers to construct more efficient treatment planning in less cost and less time.     

基于去冗余特征和语义关系约束的零样本属性识别

基于生成式的零样本识别方法在生成特征时受冗余信息和域偏移的影响,识别精度不佳.针对此问题,文中提出基于去冗余特征和语义关系约束的零样本属性识别方法.首先,将视觉特征映射到一个新的特征空间,通过互相关信息对视觉特征进行去冗余处理,在去除冗余视觉特征的同时保留类别的相关性,由于在识别过程中减少冗余信息的干扰,从而提高零样本识别的精度.然后,利用可见类和不可见类之间的语义关系建立知识迁移模型,并引入语义关系约束损失,约束知识迁移的过程,使生成器生成的视觉特征更能反映可见类和不可见类之间语义关系,缓解两者之间的域偏移问题.最后,引入循环一致性结构,使生成的伪特征更接近真实特征.在数据集上的实验证实文中方法提高零样本识别任务的精度,并具有较优的泛化性能.     

Phonetic-enriched text representation for Chinese sentiment analysis with reinforcement learning

The Chinese pronunciation system offers two characteristics that distinguish it from other languages: deep phonemic orthography and intonation variations. In this paper, we hypothesize that these two important properties can play a major role in Chinese sentiment analysis. In particular, we propose two effective features to encode phonetic information and, hence, fuse it with textual information. With this hypothesis, we propose Disambiguate Intonation for Sentiment Analysis (DISA), a network that we develop based on the principles of reinforcement learning. DISA disambiguates intonations for each Chinese character (pinyin) and, hence, learns precise phonetic representations. We also fuse phonetic features with textual and visual features to further improve performance. Experimental results on five different Chinese sentiment analysis datasets show that the inclusion of phonetic features significantly and consistently improves the performance of textual and visual representations and surpasses the state-of-the-art Chinese character-level representations.     

增量学习算法对文本情感识别模型的改进

常规的文本情感识别模型不能适应语言的发展,使新生的词汇不能得到有效的情感划分,并且情感识别率低.使用增量学习算法来改进文本情感识别模型,通过收集用户反馈数据,提取其中有价值的情感信息来更新常识库,从而实现对情感识别模型的改进.通过情感聊天的两组对比实验证明了加入增量学习算法的文本情感识别模型准确率优于没有加入增量学习算...     

PERFEX: An expert system for interpreting 3D myocardial perfusion

Interpreting three-dimensional (3D) data is generally recognized as an ill-defined and information-intensive task. The task becomes increasingly difficult in the context of medical diagnostic imagery, wherein the visual information must be interpreted in conjunction with other, nonvisual information. A novel approach is presented to perform the interpretation of such multidimensional information, concentrating on a medically important application: the interpretation of 3D tomograms of myocardial perfusion distribution. The overall goal is to assist in the diagnosis of coronary artery disease. The approach employs knowledge-based methods to process and map the 3D visual information into symbolic representations, which are subsequently used to infer structure (anatomy) from function (physiology), as well as to interpret the temporal effects of perfusion redistribution, and assess the extent and severity of cardiovascular disease both quantitatively. The knowledge-based system presents the resulting diagnostic recommendations in both visual and textual forms in an interactive framework, thereby enhancing overall utility. This paper presents the methodology underlying this approach, including the implementation and testing of this system within an actual clinical environment.     

一个基于神经网络的专家系统

医疗专家系统主要使用基于知识的技术，其中的决策规则和策略来自于人类的专家。把这些知识和各种推理方法结合，可以建立一个模拟专家决策过程的系统。建立这样一个系统，需要经常与专家磋商，以获取专家的知识，因而需要大量的时间和精力。为此，本文提出直接从数据中提取有效的信息，即用神经网络提取隐含在大量数据中对医疗诊断有效的信息，继之与基于规则的知识，各种推理方法相结合，建立一个神经网络专家系统。