Domain structure-based transfer learning for cross-domain word representation

Cross-domain word representation aims to learn high-quality semantic representations in an under-resourced domain by leveraging information in a resourceful domain. However, most existing methods mainly transfer the semantics of common words across domains, ignoring the semantic relations among domain-specific words. In this paper, we propose a domain structure-based transfer learning method to learn cross-domain representations by leveraging the relations among domain-specific words. To accomplish this, we first construct a semantic graph to capture the latent domain structure using domain-specific co-occurrence information. Then, in the domain adaptation process, beyond domain alignment, we employ Laplacian Eigenmaps to ensure the domain structure is consistently distributed in the learned embedding space. As such, the learned cross-domain word representations not only capture shared semantics across domains, but also maintain the latent domain structure. We performed extensive experiments on two tasks, namely sentiment analysis and query expansion. The experiment results show the effectiveness of our method for tasks in under-resourced domains.     

Joint semantic similarity assessment with raw corpus and structured ontology for semantic-oriented service discovery

Semantic-oriented service matching is one of the challenges in automatic Web service discovery. Service users may search for Web services using keywords and receive the matching services in terms of their functional profiles. A number of approaches to computing the semantic similarity between words have been developed to enhance the precision of matchmaking, which can be classified into ontology-based and corpus-based approaches. The ontology-based approaches commonly use the differentiated concept information provided by a large ontology for measuring lexical similarity with word sense disambiguation. Nevertheless, most of the ontologies are domain-special and limited to lexical coverage, which have a limited applicability. On the other hand, corpus-based approaches rely on the distributional statistics of context to represent per word as a vector and measure the distance of word vectors. However, the polysemous problem may lead to a low computational accuracy. In this paper, in order to augment the semantic information content in word vectors, we propose a multiple semantic fusion (MSF) model to generate sense-specific vector per word. In this model, various semantic properties of the general-purpose ontology WordNet are integrated to fine-tune the distributed word representations learned from corpus, in terms of vector combination strategies. The retrofitted word vectors are modeled as semantic vectors for estimating semantic similarity. The MSF model-based similarity measure is validated against other similarity measures on multiple benchmark datasets. Experimental results of word similarity evaluation indicate that our computational method can obtain higher correlation coefficient with human judgment in most cases. Moreover, the proposed similarity measure is demonstrated to improve the performance of Web service matchmaking based on a single semantic resource. Accordingly, our findings provide a new method and perspective to understand and represent lexical semantics.     

基于语义向量表示的查询扩展方法

针对传统查询扩展方法在专业领域中扩展词与原始查询之间缺乏语义关联的问题,提出一种基于语义向量表示的查询扩展方法。首先,构建了一个语义向量表示模型,通过对语料库中词的上下文语义进行学习,得到词的语义向量表示;其次,根据词语义向量表示,计算词之间的语义相似度;然后,选取与查询中词汇的语义最相似的词作为查询的扩展词,扩展原始查询语句;最后,基于提出的查询扩展方法构建了生物医学文档检索系统,针对基于维基百科或WordNet的传统查询扩展方法和BioASQ 2014—2015参加竞赛的系统进行对比实验和显著性差异指标分析。实验结果表明,基于语义向量表示查询扩展的检索方法所得到结果优于传统查询扩展方法的结果,平均准确率至少提高了1个百分点,在与竞赛系统的对比中,系统的效果均有显著性提高。     

改进词向量模型的用户画像研究

用户画像技术可以给企业带来巨大的商业价值。针对用户的历史查询词,利用词向量可以得到查询词在语义层次上的表达,但词向量模型对于同一个单词生成的词向量是相同的,使得该模型无法很好的处理一词多义的情况。因此,使用LDA主题模型为每个查询词分配主题,使查询词和其主题共同放入神经网络模型中学习得到其主题词向量,最后采用随机森林分类算法对用户基本属性进行分类构建用户画像。实验结果表明,该模型的分类精度要高于词向量模型。     

基于主题词向量聚类的话题内新事件检测

目前关于话题内新事件检测的研究较少,传统的新事件检测方法多采用基于主题模型的方法,无法兼顾主题信息和语义信息,效果不够理想。针对该问题,该文提出一种基于主题词向量聚类的话题内新事件检测方法。该方法首先使用主题词嵌入(TWE)模型对经过预处理的语料进行训练,获取主题词向量;其次,通过对主题词向量进行K-means聚类来获取话题分布;再次,按照话题内新事件检测流程,将新事件检测问题转化为新子话题发现问题;最后,利用获取到的话题分布,对按时间顺序的文档进行检测。实验结果表明,该方法能够兼顾主题信息和语义信息,有效提高话题内新事件检测的性能。     

HiWalk: Learning node embeddings from heterogeneous networks

Heterogeneous networks, such as bibliographical networks and online business networks, are ubiquitous in everyday life. Nevertheless, analyzing them for high-level semantic understanding still poses a great challenge for modern information systems. In this paper, we propose HiWalk to learn distributed vector representations of the nodes in heterogeneous networks. HiWalk is inspired by the state-of-the-art representation learning algorithms employed in the context of both homogeneous networks and heterogeneous networks, based on word embedding learning models. Different from existing methods in the literature, the purpose of HiWalk is to learn vector representations of the targeted set of nodes by leveraging the other nodes as “background knowledge”, which maximizes the structural correlations of contiguous nodes. HiWalk decomposes the adjacent probabilities of the nodes and adopts a hierarchical random walk strategy, which makes it more effective, efficient and concentrated when applied to practical large-scale heterogeneous networks. HiWalk can be widely applied in heterogeneous networks environments to analyze targeted types of nodes. We further validate the effectiveness of the proposed HiWalk through multiple tasks conducted on two real-world datasets.     

基于深度语义信息的查询扩展

随着互联网时代的到来,搜索引擎开始被普遍使用。在针对冷门数据时,由于用户的搜索词范围过小,搜索引擎无法检索出需要的数据,此时查询扩展系统可以有效辅助搜索引擎来提供可靠服务。基于全局文档分析的查询扩展方法,提出结合神经网络模型与包含语义信息的语料的语义相关模型,来更深层地提取词语间的语义信息。这些深层语义信息可以为查询扩展系统提供更加全面有效的特征支持,从而分析词语间的可扩展关系。在近义词林、语言知识库“HowNet”义原标注信息等语义数据中抽取局部可扩展词分布,利用神经网络模型的深度挖掘能力将语料空间中每一个词语的局部可扩展词分布拟合成全局可扩展词分布。在与分别基于语言模型和近义词林的查询扩展方法对比实验中,使用基于语义相关模型的查询扩展方法拥有较高的查询扩展效率;尤其针对冷门搜索数据时,语义相关模型的查全率比对比方法分别提高了11.1个百分点与5.29个百分点。     

基于深度语义信息的查询扩展

随着互联网时代的到来，搜索引擎开始被普遍使用。在针对冷门数据时，由于用户的搜索词范围过小，搜索引擎无法检索出需要的数据，此时查询扩展系统可以有效辅助搜索引擎来提供可靠服务。基于全局文档分析的查询扩展方法，提出结合神经网络模型与包含语义信息的语料的语义相关模型，来更深层地提取词语间的语义信息。这些深层语义信息可以为查询扩展系统提供更加全面有效的特征支持，从而分析词语间的可扩展关系。在近义词林、语言知识库“HowNet”义原标注信息等语义数据中抽取局部可扩展词分布，利用神经网络模型的深度挖掘能力将语料空间中每一个词语的局部可扩展词分布拟合成全局可扩展词分布。在与分别基于语言模型和近义词林的查询扩展方法对比实验中，使用基于语义相关模型的查询扩展方法拥有较高的查询扩展效率；尤其针对冷门搜索数据时，语义相关模型的查全率比对比方法分别提高了11.1个百分点与5.29个百分点。     

基于词向量的越汉跨语言事件检索研究

为了解决越汉跨语言事件检索中的查询翻译问题,该文提出了一种基于词向量的越汉跨语言事件检索方法。首先利用词向量构建事件关键词的汉语语义特征向量,然后计算越语的事件关键词的特征翻译向量,最后通过计算语义特征向量之间的相似度完成跨语言关键词对齐,从而实现查询关键词的自动翻译,进而完成跨语言事件检索。在构建的南海话题相关越汉语料库上进行的实验证明了该方法的有效性。     

Structure-sensitive semantic matching for aggregate question answering over knowledge base

Aggregate question answering essentially returns answers for given questions by obtaining query graphs with unique dependencies between values and corresponding objects. Word order dependency, as the key to uniquely identify dependency of the query graph, reflects the dependencies between the words in the question. However, due to the semantic gap caused by the expression difference between questions encoded with word vectors and query graphs represented with logical formal elements, it is not trivial to match the correct query graph for the question. Most existing approaches design more expressive query graphs for complex questions and rank them just by directly calculating their similarities, ignoring the semantic gap between them. In this paper, we propose a novel Structure-sensitive Semantic Matching(SSM) approach that learns aligned representations of dependencies in questions and query graphs to eliminate their gap. First, we propose a cross-structure matching module to bridge the gap between two modalities(i.e., textual question and query graph). Then, we propose an entropy-based gated AQG filter to remove the structural noise caused by the uncertainty of dependencies. Finally, we present a two-channel query graph representation that fuses the semantics of abstract structure and grounding content of the query graph explicitly. Experimental results show that SSM could learn aligned representations of questions and query graphs to eliminate the gaps between their dependencies, and improves up to 12% (F1 score) on aggregation questions of two benchmark datasets.     

On graphs of central episturmian words

Episturmian sequences are a natural extension of Sturmian sequences to the case of finite alphabets of arbitrary cardinality. In this paper, we are interested in central episturmian words, or simply, epicentral words, i.e., the palindromic prefixes of standard episturmian sequences. An epicentral word admits a variety of faithful representations including as a directive word, as a certain type of period vector, as a Parikh vector, as a certain type of Fine and Wilf extremal word, as a suitable modular matrix, and as a labeled graph. Various interconnections between the different representations of an epicentral word are analyzed. In particular, we investigate the structure of the graphs of epicentral words proving some curious and surprising properties.     

Neural network modelling of word production in Finnish: coding semantic and non-semantic features

The objective of our research is to computationally model word production and its disorders by means of artificial neural networks. In the current study we develop and analyze an algorithm that generates a distributed semantic coding from a given semantic tree-structure classification of words. With the algorithm it is possible to generate semantic representations that are compact and easy to modify. This renders the coding method suitable for our multilayer perceptron-based neural network model of word production. The model is shown to be able to account for a variety of performance patterns observed in four Finnish aphasia patients suffering from word-finding difficulties.     

概念驱动的小样本判别特征学习方法

小样本学习旨在让模型能够在仅有少量标记数据的新类中进行分类。基于度量学习的方法是小样本学习的一种有效方法,该类方法利用有标签的支持集样本构建类表示,再基于查询样本和类表示的相似性进行分类。因此,如何构建判别性更强的类表示是这类方法的关键所在。多数工作在构建类表示时,忽略了类概念相关信息的挖掘,这样容易引入样本中类别无关信息,从而降低类表示的判别性。为此本文提出一种概念驱动的小样本判别特征学习方法。该方法首先利用类别的语义信息来指导模型挖掘样本中类概念相关信息,进而构建更具判别性的类表示。其次,设计了随机掩码混合机制增加样本的多样性和识别难度,进一步提升类表示的质量。最后对处于决策边界附近的查询样本赋予更大的权重,引导模型关注难样本,从而更好地进行类表示学习。大量实验的结果表明本文提出的方法能够有效提升小样本分类任务的准确率,并且在多个数据集上优于当前先进的算法。     

基于频繁词义序列的检索结果聚类算法研究

目前大多搜索引擎结果聚类算法针对用户查询生成的网页摘要进行聚类,由于网页摘要较短且质量良莠不齐,聚类效果难以保证。提出了一种基于频繁词义序列的检索结果聚类算法,利用WordNet结合句法和语义特征对搜索结果构建聚类及标签。不像传统的基于向量空间模型的聚类算法,考虑了词语在文档中的序列模式。算法首先对文本进行预处理,生成压缩文档以降低文本数据维度,构建广义后缀树,挖掘出最大频繁项集,然后获取频繁词义序列。从文档中获取的有序频繁项集可以更好地反映文档的主题,把相同主题的搜索结果聚类在一起,与用户查询相关度高的优先排序。实验表明,该算法可以获得与查询相关的高质量聚类及基于语义的聚类标签,具有更高的聚类准确度和更高的运行效率,并且可扩展性良好。     

利用词的分布式表示改进作文跑题检测

作文跑题检测任务的核心问题是文本相似度计算。传统的文本相似度计算方法一般基于向量空间模型,即把文本表示成高维向量,再计算文本之间的相似度。这种方法只考虑文本中出现的词项(词袋模型),而没有利用词项的语义信息。该文提出一种新的文本相似度计算方法:基于词扩展的文本相似度计算方法,将词袋模型(Bag-of-Words)方法与词的分布式表示相结合,在词的分布式表示向量空间中寻找与文本出现的词项语义上相似的词加入到文本表示中,实现文本中单词的扩展。然后对扩展后的文本计算相似度。该文将这种方法运用到英文作文的跑题检测中,构建一套跑题检测系统,并在一个真实数据中进行测试。实验结果表明该文的跑题检测系统能有效识别跑题作文,性能明显高于基准系统。
     

Sense representations for Portuguese: experiments with sense embeddings and deep neural language models

Sense representations have gone beyond word representations like Word2Vec, GloVe and FastText and achieved innovative performance on a wide range of natural language processing tasks. Although very useful in many applications, the traditional approaches for generating word embeddings have a strict drawback: they produce a single vector representation for a given word ignoring the fact that ambiguous words can assume different meanings. In this paper, we explore unsupervised sense representations which, different from traditional word embeddings, are able to induce different senses of a word by analyzing its contextual semantics in a text. The unsupervised sense representations investigated in this paper are: sense embeddings and deep neural language models. We present the first experiments carried out for generating sense embeddings for Portuguese. Our experiments show that the sense embedding model (Sense2vec) outperformed traditional word embeddings in syntactic and semantic analogies task, proving that the language resource generated here can improve the performance of NLP tasks in Portuguese. We also evaluated the performance of pre-trained deep neural language models (ELMo and BERT) in two transfer learning approaches: feature based and fine-tuning, in the semantic textual similarity task. Our experiments indicate that the fine tuned Multilingual and Portuguese BERT language models were able to achieve better accuracy than the ELMo model and baselines.

     

基于GloVe模型的词向量改进方法

使用词向量表示方法能够很好的捕捉词语的语法和语义信息,为了能够提高词向量语义信息表示的准确性,本文通过分析GloVe模型共现矩阵的特点,利用分布式假设,提出了一种基于GloVe词向量训练模型的改进方法.该方法主要通过对维基百科统计词频分析,总结出过滤共现矩阵中无关词和噪声词的一般规律,最后给出了词向量在词语类比数据集和词语相关性数据集的评估结果.实验表明,在相同的实验环境中,本文的方法能够有效的缩短词向量的训练时间,并且在词语语义类比实验中准确率得到提高.     

连续时间区间内的频繁词序列挖掘算法

查询文本中频繁出现的短语可快速掌握文本内容,然而传统频繁词序列挖掘算法面向挖掘任务时的时间复杂度较高,无法满足频繁更换查询条件及快速获得反馈的查询需求。利用基于频率树的快速频繁词序列挖掘算法（TS_Mining）,在保持后缀树线性构造时间的情况下实现文本集合中频繁词序列的查询,并采用树型索引结构避免多次扫描文本集合,降低算法时间复杂度。针对连续时间区间内的频繁词序列查询问题,提出改进的剪枝挖掘算法（TS_Pruning）,通过减少频率树的扫描范围进一步提高挖掘效率。实验结果表明,TS_Mining与TS_Pruning算法的运行时间相比经典Apriori挖掘算法约减少了2个数量级,具有更高的频繁词序列挖掘效率。     

一种基于本体语义的信息检索模型

对基于向量空间模型的检索方法进行改进,提出基于本体语义的信息检索模型。将WordNet词典作为参照本体来计算概念之间的语义相似度,依据查询中标引项之间的相似度,对查询向量中的标引项进行权值调整,并参照Word-Net本体对标引项进行同义和上下位扩展,在此基础上定义查询与文档间的相似度。与传统的基于词形的信息检索方法相比,该方法可以提高语义层面上的检索精度。