Automatic generation of multiple choice questions using dependency-based semantic relations

In this paper, we present an unsupervised dependency-based approach to extract semantic relations to be applied in the context of automatic generation of multiple choice questions (MCQs). MCQs also known as multiple choice tests provide a popular solution for large-scale assessments as they make it much easier for test-takers to take tests and for examiners to interpret their results. Manual generation of MCQs is a very expensive and time-consuming task and yet they often need to be produced on a large scale and within short iterative cycles. We approach the problem of automated MCQ generation with the help of unsupervised relation extraction, a technique used in a number of related natural language processing problems. The goal of Unsupervised relation extraction is to identify the most important named entities and terminology in a document and then recognise semantic relations between them, without any prior knowledge as to the semantic types of the relations or their specific linguistic realisation. We use these techniques to process instructional texts and identify those facts (terminology, entities, and semantic relations between them) that are likely to be important for assessing test-takers’ familiarity with the instructional material. We investigate an approach to learn semantic relations between named entities by employing a dependency tree model. Our findings show that an optimised configuration of our MCQ generation system is capable of attaining high precision rates, which are much more important than recall in the automatic generation of MCQs. We also carried out a user-centric evaluation of the system, where subject domain experts evaluated automatically generated MCQ items in terms of readability, usefulness of semantic relations, relevance, acceptability of questions and distractors and overall MCQ usability. The results of this evaluation make it possible for us to draw conclusions about the utility of the approach in practical e-Learning applications.     

基于跨场景推理的事件关系检测方法

事件关系检测是一项面向事件之间逻辑关系的自然语言处理技术。事件关系识别的核心任务是以事件为基本语义单元,通过分析事件的篇章结构信息及语义特征,实现事件逻辑关系的深层检测。该文首次建立一套事件关系检测的任务和研究体系,包括任务定义、关系体系划分、语料采集与标注、评价方法等。同时,该文提出了一种跨场景推理的事件关系检测方法,该方法认为,具有相同事件场景的“事件对”,往往具有相同的事件关系类型。该文提出的基于跨场景推理的事件关系检测方法在针对四大类事件关系类型的检测精确率为54.21%。     

基于最大熵模型的汉语标点句缺失话题自动识别初探

本文的任务是判别标点句缺失话题是上句的主语还是宾语,将该任务作为标点句缺失话题自动识别研究的切入点。首先归纳了判别这一任务的一系列字面特征和语义特征,然后结合规则和最大熵模型,进行自动判别实验。结果显示,对特定类别动词的实验F值达到82%。对实验结果的分析说明,动词特征和语义特征对判别该任务的作用最大,规则方法和统计方法在判别任务中不能偏废,精细化的知识对判别的性能有重要影响。     

中文篇章级句间语义关系识别

篇章句间关系识别(Discourse Relation Recognition)是篇章分析的重要内容,该文对中文篇章句间关系识别任务进行初步探索,包括显式篇章句间关系识别与隐式篇章句间关系识别两类任务。针对显式篇章句间关系,我们提出基于关联词规则的方法进行识别,取得了很好的效果;针对隐式篇章句间关系,我们抽取词汇、句法、语义等特征,采用有指导模型进行识别。该文的分析和实验结果为后续研究提供了参考和基本对照系统。     

基于多粒度语义交互理解网络的幽默等级识别

幽默在人们日常交流中发挥着重要作用.随着人工智能的快速发展,幽默等级识别成为自然语言处理领域的热点研究问题之一.已有的幽默等级识别研究往往将幽默文本看作一个整体,忽视了幽默文本内部的语义关系.该文将幽默等级识别视为自然语言推理任务,将幽默文本划分为"铺垫"和"笑点"两个部分,分别对其语义和语义关系进行建模,提出了一种多...     

基于注意力掩码语言模型的隐式篇章关系识别

隐式篇章关系识别是在缺少显式连接词的条件下,自动判定论元之间的语义关系。其挑战性在于现有训练数据的规模较小,其蕴含的语义多样性也相对有限。针对上述问题,该文利用掩码语言模型架构建立篇章关系分类模型。其动因包括: ①掩码语言模型在自监督学习过程中具备局部的语言生成能力,即在理解上下文语义的基础上“重构掩码区域语义表示”的能力; ②掩码重构形成了数据增强(潜在的自动数据扩展)的效果,有助于提高篇章关系分类模型的鲁棒性。特别地,该文提出一种基于交互注意力的掩码语言模型,该方法计算论元之间的交互注意力矩阵,并依赖交互注意力动态选择论元之间高关联性的关键词项进行遮蔽、掩码重构,从而形成更有针对性的数据增强(非关键信息的数据增强对关系分类影响不大)。该文利用宾州篇章树库语料进行实验。实验结果表明,相较于基准系统,我们提出的方法的F₁值在四大类关系(对比关系、偶然性关系、扩展关系和时序关系)上分别提高了3.21%、6.46%、2.74%和6.56%。     

基于柱搜索的高阶依存句法分析

该文提出使用所有的孙子节点构成祖孙特征的高阶依存模型,并且使用柱搜索策略限制搜索空间,最终找到近似最优依存树。另外,该文以较小的时间复杂度为代价,使用了丰富的依存关系特征,并且允许模型在解码的过程中进行依存关系选择。作者参加了CoNLL 2009年多语依存句法分析和语义角色标注国际评测,最终获得联合任务总成绩第一名,依存句法分析总成绩第三名。     

Rumor detection with self-supervised learning on texts and social graph

Rumor detection has become an emerging and active research field in recent years. At the core is to model the rumor characteristics inherent in rich information, such as propagation patterns in social network and semantic patterns in post content, and differentiate them from the truth. However, existing works on rumor detection fall short in modeling heterogeneous information, either using one single information source only (e.g., social network, or post content) or ignoring the relations among multiple sources (e.g., fusing social and content features via simple concatenation).Therefore, they possibly have drawbacks in comprehensively understanding the rumors, and detecting them accurately. In this work, we explore contrastive self-supervised learning on heterogeneous information sources, so as to reveal their relations and characterize rumors better. Technically, we supplement the main supervised task of detection with an auxiliary self-supervised task, which enriches post representations via post self-discrimination.Specifically, given two heterogeneous views of a post (i.e., representations encoding social patterns and semantic patterns), the discrimination is done by maximizing the mutual information between different views of the same post compared to that of other posts. We devise cluster-wise and instance-wise approaches to generate the views and conduct the discrimination, considering different relations of information sources. We term this framework as self-supervised rumor detection (SRD). Extensive experiments on three real-world datasets validate the effectiveness of SRD for automatic rumor detection on social media.     

A review on automatic image annotation techniques

Nowadays, more and more images are available. However, to find a required image for an ordinary user is a challenging task. Large amount of researches on image retrieval have been carried out in the past two decades. Traditionally, research in this area focuses on content based image retrieval. However, recent research shows that there is a semantic gap between content based image retrieval and image semantics understandable by humans. As a result, research in this area has shifted to bridge the semantic gap between low level image features and high level semantics. The typical method of bridging the semantic gap is through the automatic image annotation (AIA) which extracts semantic features using machine learning techniques. In this paper, we focus on this latest development in image retrieval and provide a comprehensive survey on automatic image annotation. We analyse key aspects of the various AIA methods, including both feature extraction and semantic learning methods. Major methods are discussed and illustrated in details. We report our findings and provide future research directions in the AIA area in the conclusions     

面向信息检索的概念关系自动构建

概念之间的依存分析是提高信息检索性能的关键。相比概念关系的强弱而言,识别关系的类型更有意义。本文在Bootstrapping框架下,以“(地理)领属”,“(实体)功能”和“(动作)对象”三种语义关系类型为例,获得了构建上下文中两个概念特定关系的语义模板,并开发了一个名为SPG的系统。本文的工作: (1)引入了生物信息计算中序列比对的方法两两生成相似上下文的模板;(2)定义了新的模板评价机制估计模板的置信度。就这三种概念关系的识别而言,SPG获得的模板集合相比DIPRE系统获得了更高的正确率和覆盖能力。     

稠密子图发现的视频语义挖掘方法

目前基于内容的视频语义挖掘方法并未考虑到视频的多模态特性,不能够实现对于目前海量涌现视频的自动分析处理任务。针对此问题,提出了基于稠密子图发现的视频语义挖掘方法。该方法对待处理的视频进行中文连续语音识别、视频目标识别和视频文字识别,对于识别结果进行中文分词和词性标注,保留名词和动词作为图模型的顶点,顶点之间的边权重设置为两个顶点所代表的词语的中文语义距离,根据稠密子图发现算法挖掘视频的语义信息。实验结果表明这种方法是有效的。     

一种面向移动终端的自然口语任务理解方法

随着移动互联时代的到来和语音识别技术的日益成熟,通过语音的交互方式来使用移动终端成为一种趋势.如何理解用户自然状态下的口语输入,传统的做法是手写上下文无关的文法规则,但是文法规则的书写需耗费大量的人力和物力,很难去维护和更新.提出一种采用支持向量机和条件随机场串行结合的方法,把口语任务理解分解为任务发现和信息抽取两个过程,并最终将任务表达成语义向量的形式.最终对“讯飞语点”语音助手用户返回的八个不同的任务种类的数据进行了测试,在一比一的噪声中识别任务语义表达的准确率为90.29%,召回率为88.87%.     

Incorporating contextual evidence to improve implicit discourse relation recognition in Chinese

The discourse analysis task, which focuses on understanding the semantics of long text spans, has received increasing attention in recent years. As a critical component of discourse analysis, discourse relation recognition aims to identify the rhetorical relations between adjacent discourse units (e.g., clauses, sentences, and sentence groups), called arguments, in a document. Previous works focused on capturing the semantic interactions between arguments to recognize their discourse relations, ignoring important textual information in the surrounding contexts. However, in many cases, more than capturing semantic interactions from the texts of the two arguments are needed to identify their rhetorical relations, requiring mining more contextual clues. In this paper, we propose a method to convert the RST-style discourse trees in the training set into dependency-based trees and train a contextual evidence selector on these transformed structures. In this way, the selector can learn the ability to automatically pick critical textual information from the context (i.e., as evidence) for arguments to assist in discriminating their relations. Then we encode the arguments concatenated with corresponding evidence to obtain the enhanced argument representations. Finally, we combine original and enhanced argument representations to recognize their relations. In addition, we introduce auxiliary tasks to guide the training of the evidence selector to strengthen its selection ability. The experimental results on the Chinese CDTB dataset show that our method outperforms several state-of-the-art baselines in both micro and macro F1 scores.     

《知网》语义关系图的自动构建

在真实语言环境中,词语间的联系普遍存在、错综复杂。为了更好融合和使用各种语义资源库中的语义关系,构建可计算的汉语词汇语义资源,该文提出了通过构建语义关系图整合各种语义资源的方法,并在《知网》上实现。《知网》作为一个知识库系统,对各个词语义项是以分条记录的形式存储的,各种词汇语义关系隐含在词典文件和义原描述文件中。为提取《知网》中语义间的关系,本文首先将《知网》中的概念以概念树的形式重新表示,并从概念树中提取适当的语义关系,构建语义关系图。经过处理,得到88种589 984条语义关系,图上各种节点具有广泛的联系,为基于语义关系图的进一步分析和计算打下了基础。     

基于语义依存图库的兼语句句模研究

句子语义分析是语言研究深入发展的客观要求,也是当前制约语言信息处理技术深度应用的主要因素。在探索深层语义分析方法的基础上,该文根据汉语的特点,提出了一整套语义依存图的构建方法,并建立了一个包含30 000个句子的语义依存图库。以兼语句为重点研究对象,该文研究了语料库中所有纯粹的兼语句所对应的句模情况,进而试图构建基于语义依存图的句模系统,总结句型和句模的映射规则,从而为更好的建立语义自动分析模型提供相应的知识库。
     

词汇间语义相关关系量化计算方法

词汇间语义关系的定量化研究是自然语言处理任务中一个重要的基础性工作。词汇间语义关系总体上分为等同关系、上下位关系、相关关系,现有的语义关系定量化工作主要集中于词汇间语义的等同关系(相似性)量化研究。该文研究和提出了量化词汇间语义相关关系的基本思路和新方法,即构造词汇相关关系二分图来求解和量化词汇间间接相关关系,该方法能够解决在统计语料中没有出现的词汇对的相关关系量化求解问题。实验结果表明,该文提出的方法比单纯用互信息来计算和量化词汇间语义相关关系更为可行。同时,对于一个特定词汇而言,该文的方法能够得到一个相关关系量化的相对合理的趋势性结果。     

基于词频同现与WordNet的图像自动标注改善算法研究

图像自动标注是模式识别与计算机视觉等领域中的重要问题。针对现有图像自动标注模型普遍受到语义鸿沟问题的影响,提出了基于关键词同现的图像自动标注改善方法,该方法利用数据集中标注词间的关联性来改善图像自动标注的结果。此外,针对上述方法不能反映更广义的人的知识以及易受数据库规模影响等问题,提出了基于语义相似的图像自动标注改善方法,通过引入具有大量词汇、包含了人知识的结构化电子词典WordNet来计算词汇间的关系并改善图像自动标注结果。实验结果表明,提出的两个图像自动标注改善方法在各项评价指标上相比以往模型均有所提高。     

基于增强BiLSTM-CRF模型的推文恶意软件名称识别

针对推文中恶意软件名称识别任务存在的文本简短、非正式、实体类别单一以及实体歧义等问题,提出了一种基于BERT-BiLSTM-Self-attention-CRF的实体识别方法,以实现推文中恶意软件名称的自动识别。在BiLSTM-CRF模型的基础上,利用BERT模型编码单词语境信息,提升词嵌入的上下文语义质量,增强原有模型的语义消歧能力;同时,借助Self-attention机制学习单词间关系和句子结构特征,利用加权表征帮助单一类别实体的解码,以提升恶意软件名称实体的识别效果。通过构建包含恶意软件名称实体的推文标记数据集进行实验测试,结果表明,提出的方法可以实现更好的性能,其精确率、召回率、F1值分别为86.38%,84.73%,85.55%,相较于基线模型BiLSTM-CRF,F1值提升了12.61%。     

Annotation of semantic roles for the Turkish Proposition Bank

In this work, we report large-scale semantic role annotation of arguments in the Turkish dependency treebank, and present the first comprehensive Turkish semantic role labeling (SRL) resource: Turkish Proposition Bank (PropBank). We present our annotation workflow that harnesses crowd intelligence, and discuss the procedures for ensuring annotation consistency and quality control. Our discussion focuses on syntactic variations in realization of predicate-argument structures, and the large lexicon problem caused by complex derivational morphology. We describe our approach that exploits framesets of root verbs to abstract away from syntax and increase self-consistency of the Turkish PropBank. The issues that arise in the annotation of verbs derived via valency changing morphemes, verbal nominals, and nominal verbs are explored, and evaluation results for inter-annotator agreement are provided. Furthermore, semantic layer described here is aligned with universal dependency (UD) compliant treebank and released to enable more researchers to work on the problem. Finally, we use PropBank to establish a baseline score of 79.10 F1 for Turkish SRL using the mate-tool (an open-source SRL tool based on supervised machine learning) enhanced with basic morphological features. Turkish PropBank and the extended SRL system are made publicly available.     

基于主题相似度的宏观篇章主次关系识别方法

篇章分析是自然语言处理领域的一个重要任务。分析篇章主次关系有助于理解篇章的结构和语义,并为自然语言处理的应用提供有力的支持。该文在微观篇章主次关系识别研究的基础上,重点研究宏观篇章主次关系,提出了一种基于word2vec和LDA的主题相似度的宏观篇章主次关系识别模型。基于word2vec的主题相似度和基于LDA的主题相似度在不同维度上计算语义相似度,两者在语义层面形成互补,因而增强了模型识别宏观篇章主次关系的能力。该模型在宏观汉语篇章树库(MCDTB)上实验的F1值达到79.9%,正确率达到81.82%,相较基准系统分别提升了1.7%和1.81%。