一种话题敏感的抽取式多文档摘要方法

抽取式摘要的核心问题在于合理地建模句子,正确地判断句子重要性。该文提出一种计算句子话题重要性的方法,通过分析句子与话题的语义关系,判断句子是否描述话题的重要信息。针对自动摘要任务缺乏参考摘要作为训练数据的问题,该文提出一种基于排序学习的半监督训练框架,利用大规模未标注新闻语料训练模型。在DUC2004多文档摘要任务上的实验结果表明,该文提出的话题重要性特征能够作为传统启发式特征的有效补充,改进摘要质量。     

中文分词和词性标注联合模型综述

中文分词和词性标注任务作为中文自然语言处理的初始步骤,已经得到广泛的研究。由于中文句子缺乏词边界,所以中文词性标注往往采用管道模式完成:首先对句子进行分词,然后使用分词阶段的结果进行词性标注。然而管道模式中,分词阶段的错误会传递到词性标注阶段,从而降低词性标注效果。近些年来,中文词性标注方面的研究集中在联合模型。联合模型同时完成句子的分词和词性标注任务,不但可以改善错误传递的问题,并且可以通过使用词性标注信息提高分词精度。联合模型分为基于字模型、基于词模型及混合模型。本文对联合模型的分类、训练算法及训练过程中的问题进行详细的阐述和讨论。     

基于预训练模型的中文电子病历实体识别

为提升传统中文电子病历实体识别预训练模型的语义特征提取能力并增强中文隐含特征表示,提出基于改进预训练语言模型的医疗命名实体识别方法。提出动态词长的逆向最大匹配算法对病历文本进行标注歧义处理,在此基础上构建用户自定义医疗实体字典辅助PKU分词,提高预训练模型掩码效果。输入向量层加入字向量,引入注意力机制学习字向量的全局语义特征。改进预训练模型mask策略和机制并去掉下一句预测,提升词向量语义表征能力。实验结果表明,该方法有效提高了医疗实体的识别效果,F1值达到90.57%。     

基于多掩码与提示句向量融合分类的立场检测

立场检测是指分析文本对于某一目标话题表达的立场,立场通常分为支持、反对和其他。近期的工作大多采用BERT等方法提取文本和话题的句语义特征,通常采用BERT首符号隐藏状态或者句子中每个词隐藏状态取平均作为句向量。该文对句向量的获取进行了改进,采用提示学习模板获取提示句向量,提高句向量的特征提取效果。设计了一种基于多掩码与提示句向量融合分类的立场检测模型(PBMSV),将提示句向量分类与多掩码的模板-答案器结构提示学习分类结合,向句向量引入文本、话题和立场词信息,融合句向量和答案器分类结果,对模型进行联合优化。在NLPCC中文立场检测数据集上的实验表明,在五个话题单独训练模型的实验中,该文方法与此前最优方法相比在三个目标上取得领先或持平,取得了79.3的总F1值,与最优方法接近,并在句向量对比实验中,验证了提示句向量的优势。     

基于深度学习和迁移学习的领域自适应中文分词

为了提高专业领域中文分词性能,以及弥补专业领域大规模标注语料难以获取的不足,该文提出基于深度学习以及迁移学习的领域自适应分词方法。首先,构建包含词典特征的基于深度学习的双向长短期记忆条件随机场(BI-LSTM-CRF)分词模型,在通用领域分词语料上训练得到模型参数;接着,以建设工程法律领域文本作为小规模分词训练语料,对通用领域语料的BI-LSTM-CRF分词模型进行参数微调,同时在模型的词典特征中加入领域词典。实验结果表明,迁移学习减少领域分词模型的迭代次数,同时,与通用领域的BI-LSTM-CRF模型相比,该文提出的分词方法在工程法律领域的分词结果F1值提高了7.02%,与预测时加入领域词典的BI-LSTM-CRF模型相比,分词结果的F1值提高了4.22%。该文提出的分词模型可以减少分词的领域训练语料的标注,同时实现分词模型跨领域的迁移。     

基于词向量技术与主题词特征的微博立场检测

微博话题随着移动互联网的发展变得火热起来,单个热门话题可能有数万条评论,微博话题的立场检测是针对某话题判断发言人对该话题的态度是支持的、反对的或中立的.本文一方面由Word2Vec训练语料库中每个词的词向量获取句子的语义信息,另一方面使用TextRank构建主题集作为话题的立场特征,同时结合情感词典获取句子的情感信息,最后将特征选择后的词向量使用支持向量机对其训练和预测完成最终的立场检测模型.实验表明基于主题词及情感词相结合的立场特征可以获得不错的立场检测效果.     

一种融合字词信息的中文情感分析模型

中文情感分析模型的文本表示通常只采用词粒度信息，这会导致模型在特征提取时丧失字粒度的特性，同时常用的分词方法的分词结果过于精简，也一定程度上限制了文本表示的丰富度。对此，提出了一种融合字粒度特征与词粒度特征的中文情感分析模型，采用全模式分词得到更丰富的词序列，经词嵌入后将词向量输入Bi-LSTM中提取全文的语义信息，并将隐层语义表示与对应字向量进行初步融合，增强词级信息的鲁棒性；另一方面将字向量输入多窗口卷积，捕捉更细粒度的字级特征信息。最后将字词粒度特征进一步融合后输入分类器得到情感分类结果，在2个公开数据集上的性能测试结果表明，该模型相比同类模型有更好的分类性能。     

融合多粒度信息的文本分类研究

目前对中文文本分类的研究主要集中于对字符粒度、词语粒度、句子粒度、篇章粒度等数据信息的单一模式划分，这往往缺少不同粒度下语义所包含的信息特征。为了更加有效提取文本所要表达的核心内容，提出一种基于注意力机制融合多粒度信息的文本分类模型。该模型对字、词和句子粒度方面构造嵌入向量，其中对字和词粒度采用Word2Vec训练模型将数据转换为字向量和词向量，通过双向长短期记忆网络（bi-directional long short-term memory,BiLSTM）获取字和词粒度向量的上下文语义特征，利用FastText模型提取句子向量中包含的特征，将不同种特征向量分别送入到注意力机制层进一步获取文本重要的语义信息。实验结果表明，该模型在三种公开的中文数据集上的分类准确率比单一粒度和两两粒度结合的分类准确率都有所提高。     

融合多粒度特征的低资源语言词性标注和依存分析联合模型

研究低资源语言的词性标注和依存分析对推动低资源自然语言处理任务有着重要的作用。针对低资源语言词嵌入表示,已有工作并没有充分利用字符、子词层面信息编码,导致模型无法利用不同粒度的特征。对此,该文提出融合多粒度特征的词嵌入表示,利用不同的语言模型分别获得字符、子词以及词语层面的语义信息,将三种粒度的词嵌入进行拼接,达到丰富语义信息的目的,缓解由于标注数据稀缺导致的依存分析模型性能不佳的问题。进一步将词性标注和依存分析模型进行联合训练,使模型之间能相互共享知识,降低词性标注错误在依存分析任务上的线性传递。以泰语、越南语为研究对象,在宾州树库数据集上的试验表明,该文方法相比于基线模型的UAS、LAS、POS均有明显提升。     

基于字词融合与对抗训练的行业人物实体识别

行业人物命名实体识别旨在从行业人物语料中抽取出有效的实体信息,是深度挖掘行业人物信息资源的基础性和关键性任务。由于主流的命名实体识别模型未充分利用词信息特征,导致在对具有特色的行业人物实体进行识别时语义、实体边界识别不准确。提出一种基于字词融合与对抗训练的行业人物实体识别模型。利用RoBERTa-wwm-ext预训练模型提取句子的字特征,并融合词典构造出句子的词特征。在字词融合的向量表示上添加扰动生成对抗样本,将融合向量表示与对抗样本作为训练数据输入双向长短期记忆神经网络(BiLSTM)学习上下文信息,并通过条件随机场(CRF)推理最优的序列标注结果。根据行业人物文本特点设计命名实体标注方案,构建数据集进行实验验证。实验结果表明,该模型在测试集上的精确率、召回率、F1值分别达到92.94%、94.35%、93.64%,相较于BERT-BiLSTM-CRF模型分别提升3.68、1.24、2.39个百分点。     

Learning from a Population of Hypotheses

We introduce a new formal model in which a learning algorithm must combine a collection of potentially poor but statistically independent hypothesis functions in order to approximate an unknown target function arbitrarily well. Our motivation includes the question of how to make optimal use of multiple independent runs of a mediocre learning algorithm, as well as settings in which the many hypotheses are obtained by a distributed population of identical learning agents.     

On the Complexity of Function Learning

The majority of results in computational learning theory are concerned with concept learning, i.e. with the special case of function learning for classes of functions with range {0, 1}. Much less is known about the theory of learning functions with a larger range such as or . In particular relatively few results exist about the general structure of common models for function learning, and there are only very few nontrivial function classes for which positive learning results have been exhibited in any of these models.We introduce in this paper the notion of a binary branching adversary tree for function learning, which allows us to give a somewhat surprising equivalent characterization of the optimal learning cost for learning a class of real-valued functions (in terms of a max-min definition which does not involve any learning model).Another general structural result of this paper relates the cost for learning a union of function classes to the learning costs for the individual function classes.Furthermore, we exhibit an efficient learning algorithm for learning convex piecewise linear functions from ^d into . Previously, the class of linear functions from ^d into was the only class of functions with multidimensional domain that was known to be learnable within the rigorous framework of a formal model for online learning.Finally we give a sufficient condition for an arbitrary class of functions from into that allows us to learn the class of all functions that can be written as the pointwise maximum ofk functions from . This allows us to exhibit a number of further nontrivial classes of functions from into for which there exist efficient learning algorithms.     

Transfer in variable-reward hierarchical reinforcement learning

Transfer learning seeks to leverage previously learned tasks to achieve faster learning in a new task. In this paper, we consider transfer learning in the context of related but distinct Reinforcement Learning (RL) problems. In particular, our RL problems are derived from Semi-Markov Decision Processes (SMDPs) that share the same transition dynamics but have different reward functions that are linear in a set of reward features. We formally define the transfer learning problem in the context of RL as learning an efficient algorithm to solve any SMDP drawn from a fixed distribution after experiencing a finite number of them. Furthermore, we introduce an online algorithm to solve this problem, Variable-Reward Reinforcement Learning (VRRL), that compactly stores the optimal value functions for several SMDPs, and uses them to optimally initialize the value function for a new SMDP. We generalize our method to a hierarchical RL setting where the different SMDPs share the same task hierarchy. Our experimental results in a simplified real-time strategy domain show that significant transfer learning occurs in both flat and hierarchical settings. Transfer is especially effective in the hierarchical setting where the overall value functions are decomposed into subtask value functions which are more widely amenable to transfer across different SMDPs.     

The Power of Self-Directed Learning

This article studies self-directed learning, a variant of the on-line (or incremental) learning model in which the learner selects the presentation order for the instances. Alternatively, one can view this model as a variation of learning with membership queries in which the learner is only charged for membership queries for which it could not predict the outcome. We give tight bounds on the complexity of self-directed learning for the concept classes of monomials, monotone DNF formulas, and axis-parallel rectangles in {0, 1, , n – 1} ^d . These results demonstrate that the number of mistakes under self-directed learning can be surprisingly small. We then show that learning complexity in the model of self-directed learning is less than that of all other commonly studied on-line and query learning models. Next we explore the relationship between the complexity of self-directed learning and the Vapnik-Chervonenkis (VC-)dimension. We show that, in general, the VC-dimension and the self-directed learning complexity are incomparable. However, for some special cases, we show that the VC-dimension gives a lower bound for the self-directed learning complexity. Finally, we explore a relationship between Mitchell's version space algorithm and the existence of self-directed learning algorithms that make few mistakes.     

Toward Efficient Agnostic Learning

In this paper we initiate an investigation of generalizations of the Probably Approximately Correct (PAC) learning model that attempt to significantly weaken the target function assumptions. The ultimate goal in this direction is informally termed agnostic learning, in which we make virtually no assumptions on the target function. The name derives from the fact that as designers of learning algorithms, we give up the belief that Nature (as represented by the target function) has a simple or succinct explanation. We give a number of positive and negative results that provide an initial outline of the possibilities for agnostic learning. Our results include hardness results for the most obvious generalization of the PAC model to an agnostic setting, an efficient and general agnostic learning method based on dynamic programming, relationships between loss functions for agnostic learning, and an algorithm for a learning problem that involves hidden variables.     

一种新的连续动作集学习自动机

学习自动机（Learning automation,LA）是一种自适应决策器。其通过与一个随机环境不断交互学习从一个允许的动作集里选择最优的动作。在大多数传统的LA模型中,动作集总是被取作有限的。因此,对于连续参数学习问题,需要将动作空间离散化,并且学习的精度取决于离散化的粒度。本文提出一种新的连续动作集学习自动机（Continuous action set learning automaton,CALA）,其动作集为一个可变区间,同时按照均匀分布方式选择输出动作。学习算法利用来自环境的二值反馈信号对动作区间的端点进行自适应更新。通过一个多模态学习问题的仿真实验,演示了新算法相对于3种现有CALA算法的优越性。     

Exploring learnability between exact and PAC

We study a model of probably exactly correct (PExact) learning that can be viewed either as the Exact model (learning from equivalence queries only) relaxed so that counterexamples to equivalence queries are distributionally drawn rather than adversarially chosen or as the probably approximately correct (PAC) model strengthened to require a perfect hypothesis. We also introduce a model of probably almost exactly correct (PAExact) learning that requires a hypothesis with negligible error and thus lies between the PExact and PAC models. Unlike the Exact and PExact models, PAExact learning is applicable to classes of functions defined over infinite instance spaces. We obtain a number of separation results between these models. Of particular note are some positive results for efficient parallel learning in the PAExact model, which stand in stark contrast to earlier negative results for efficient parallel Exact learning.     

不同程度的监督机制在自动文本分类中的应用

自动文本分类技术涉及信息检索、模式识别及机器学习等领域。本文以监督的程度为线索，综述了分属全监督，非监督以及半监督学习策略的若干方法-NBC(Naive Bayes Classifier)，FCM(Fuzzy C-Means)，SOM(Self-Organizing Map)，ssFCM(serni-supervised Fuzzy C-Means)gSOM(guided Self-Organizing Map)，并应用于文本分类中。其中，gSOM是我们在SOM基础上发展得到的半监督形式。并以Reuters-21578为语料，研究了监督程度对分类效果的影响，从而提出了对实际文本分类工作的建议。     

Context counts: How learners' contexts influence learning in a MOOC

Massive Open Online Courses (MOOCs) require individual learners to self-regulate their own learning, determining when, how and with what content and activities they engage. However, MOOCs attract a diverse range of learners, from a variety of learning and professional contexts. This study examines how a learner's current role and context influences their ability to self-regulate their learning in a MOOC: Introduction to Data Science offered by Coursera. The study compared the self-reported self-regulated learning behaviour between learners from different contexts and with different roles. Significant differences were identified between learners who were working as data professionals or studying towards a higher education degree and other learners in the MOOC. The study provides an insight into how an individual's context and role may impact their learning behaviour in MOOCs.     

Indexing,Elaboration and Refinement: Incremental Learning of Explanatory Cases

This article describes how a reasoner can improve its understanding of an incompletely understood domain through the application of what it already knows to novel problems in that domain. Case-based reasoning is the process of using past experiences stored in the reasoner's memory to understand novel situations or solve novel problems. However, this process assumes that past experiences are well understood and provide good lessons to be used for future situations. This assumption is usually false when one is learning about a novel domain, since situations encountered previously in this domain might not have been understood completely. Furthermore, the reasoner may not even have a case that adequately deals with the new situation, or may not be able to access the case using existing indices. We present a theory of incremental learning based on the revision of previously existing case knowledge in response to experiences in such situations. The theory has been implemented in a case-based story understanding program that can (a) learn a new case in situations where no case already exists, (b) learn how to index the case in memory, and (c) incrementally refine its understanding of the case by using it to reason about new situations, thus evolving a better understanding of its domain through experience. This research complements work in case-based reasoning by providing mechanisms by which a case library can be automatically built for use by a case-based reasoning program.