基于半监督CRF的跨领域中文分词

中文分词是中文信息处理领域的一项关键基础技术。随着中文信息处理应用的发展,专业领域中文分词需求日益增大。然而,现有可用于训练的标注语料多为通用领域(或新闻领域)语料,跨领域移植成为基于统计的中文分词系统的难点。在跨领域分词任务中,由于待分词文本与训练文本构词规则和特征分布差异较大,使得全监督统计学习方法难以获得较好的效果。该文在全监督CRF中引入最小熵正则化框架,提出半监督CRF分词模型,将基于通用领域标注文本的有指导训练和基于目标领域无标记文本的无指导训练相结合。同时,为了综合利用各分词方法的优点,该文将加词典的方法、加标注语料的方法和半监督CRF模型结合起来,提高分词系统的领域适应性。实验表明,半监督CRF较全监督CRF OOV召回率提高了3.2个百分点,F-值提高了1.1个百分点;将多种方法混合使用的分词系统相对于单独在CRF模型中添加标注语料的方法OOV召回率提高了2.9个百分点,F-值提高了2.5个百分点。     

A semi-supervised regression model for mixed numerical and categorical variables

In this paper, we develop a semi-supervised regression algorithm to analyze data sets which contain both categorical and numerical attributes. This algorithm partitions the data sets into several clusters and at the same time fits a multivariate regression model to each cluster. This framework allows one to incorporate both multivariate regression models for numerical variables (supervised learning methods) and k-mode clustering algorithms for categorical variables (unsupervised learning methods). The estimates of regression models and k-mode parameters can be obtained simultaneously by minimizing a function which is the weighted sum of the least-square errors in the multivariate regression models and the dissimilarity measures among the categorical variables. Both synthetic and real data sets are presented to demonstrate the effectiveness of the proposed method.     

Learning from partially supervised data using mixture models and belief functions

This paper addresses classification problems in which the class membership of training data are only partially known. Each learning sample is assumed to consist of a feature vector x_i∈X and an imprecise and/or uncertain “soft” label m_i defined as a Dempster-Shafer basic belief assignment over the set of classes. This framework thus generalizes many kinds of learning problems including supervised, unsupervised and semi-supervised learning. Here, it is assumed that the feature vectors are generated from a mixture model. Using the generalized Bayesian theorem, an extension of Bayes’ theorem in the belief function framework, we derive a criterion generalizing the likelihood function. A variant of the expectation maximization (EM) algorithm, dedicated to the optimization of this criterion is proposed, allowing us to compute estimates of model parameters. Experimental results demonstrate the ability of this approach to exploit partial information about class labels.     

Semi-supervised change detection using modified self-organizing feature map neural network

In the present article, semi-supervised learning is integrated with an unsupervised context-sensitive change detection technique based on modified self-organizing feature map (MSOFM) network. In the proposed methodology, training of the MSOFM network is initially performed using only a few labeled patterns. Thereafter, the membership values, in both the classes, for each unlabeled pattern are determined using the concept of fuzzy set theory. The soft class label for each of the unlabeled patterns is then estimated using the membership values of its K nearest neighbors. Here, training of the network using the unlabeled patterns along with a few labeled patterns is carried out iteratively. A heuristic method has been suggested to select some patterns from the unlabeled ones for training. To check the effectiveness of the proposed methodology, experiments are conducted on three multi-temporal and multi-spectral data sets. Performance of the proposed work is compared with that of two unsupervised techniques, a supervised technique and two semi-supervised techniques. Results are also statistically validated using paired t-test. The proposed method produced promising results.     

Deep and fast: Deep learning hashing with semi-supervised graph construction

Learning-based hashing methods are becoming the mainstream for approximate scalable multimedia retrieval. They consist of two main components: hash codes learning for training data and hash functions learning for new data points. Tremendous efforts have been devoted to designing novel methods for these two components, i.e., supervised and unsupervised methods for learning hash codes, and different models for inferring hashing functions. However, there is little work integrating supervised and unsupervised hash codes learning into a single framework. Moreover, the hash function learning component is usually based on hand-crafted visual features extracted from the training images. The performance of a content-based image retrieval system crucially depends on the feature representation and such hand-crafted visual features may degrade the accuracy of the hash functions. In this paper, we propose a semi-supervised deep learning hashing (DLH) method for fast multimedia retrieval. More specifically, in the first component, we utilize both visual and label information to learn an optimal similarity graph that can more precisely encode the relationship among training data, and then generate the hash codes based on the graph. In the second stage, we apply a deep convolutional network to simultaneously learn a good multimedia representation and a set of hash functions. Extensive experiments on five popular datasets demonstrate the superiority of our DLH over both supervised and unsupervised hashing methods.     

跨语言词向量研究综述

随着人们对互联网多语言信息需求的日益增长,跨语言词向量已成为一项重要的基础工具,并成功应用到机器翻译、信息检索、文本情感分析等自然语言处理领域。跨语言词向量是单语词向量的一种自然扩展,词的跨语言表示通过将不同的语言映射到一个共享的低维向量空间,在不同语言间进行知识转移,从而在多语言环境下对词义进行准确捕捉。近几年跨语言词向量模型的研究成果比较丰富,研究者们提出了较多生成跨语言词向量的方法。该文通过对现有的跨语言词向量模型研究的文献回顾,综合论述了近年来跨语言词向量模型、方法、技术的发展。按照词向量训练方法的不同,将其分为有监督学习、无监督学习和半监督学习三类方法,并对各类训练方法的原理和代表性研究进行总结以及详细的比较;最后概述了跨语言词向量的评估及应用,并分析了所面临的挑战和未来的发展方向。     

Noise reduction and feature extraction based on low-rank representation and pairwise constraint preserving for hyperspectral images

ABSTRACT

Feature extraction (FE) methods play a central role in the classification of hyperspectral images (HSIs). However, all traditional FE methods work in original feature space (OFS), OFS may suffer from noise, outliers and poorly discriminative features. This paper presents a feature space enriching technique to address the problems of noise, outliers and poorly discriminative features which may exist in OFS. The proposed method is based on low-rank representation (LRR) with the capability of pairwise constraint preserving (PCP) termed LRR-PCP. LRR-PCP does not change the dimension of OFS and only can be used as an appropriate preprocessing procedure for any classification algorithm or DR methods. The proposed LRR-PCP aims to enrich the OFS and obtain extracted feature space (EFS) which results in features richer than OFS. The problems of noise and outliers can be decreased using LRR. But, LRR cannot preserve the intrinsic local structure of the original data and only capture the global structure of data. Therefore, two additional penalty terms are added into the objective function of LRR to keep the local discriminative ability and also preserve the data diversity. LRR-PCP method not only can be used in supervised learning but also in unsupervised and semi-supervised learning frameworks. The effectiveness of LRR-PCP is investigated on three HSI data sets using some existing DR methods and as a denoising procedure before the classification task. All experimental results and quantitative analysis demonstrate that applying LRR-PCP on OFS improves the performance of the classification and DR methods in supervised, unsupervised, and semi-supervised conditions.     

拉普拉斯阶梯网络

阶梯网络不仅是一种基于深度学习的特征提取器，而且能够应用于半监督学习中.深度学习在实现了复杂函数逼近的同时，也缓解了多层神经网络易陷入局部最小化的问题.传统的自编码、玻尔兹曼机等方法易忽略高维数据的低维流形结构信息，使用这些方法往往会获得无意义的特征表示，这些特征不能有效地嵌入到后续的预测或识别任务中.从流形学习的角度出发，提出一种基于阶梯网络的深度表示学习方法，即拉普拉斯阶梯网络LLN （Laplacian ladder network）.拉普拉斯阶梯网络在训练的过程中不仅对每一编码层嵌入噪声并进行重构，而且在各重构层引入图拉普拉斯约束，将流形结构嵌入到多层特征学习中，以提高特征提取的鲁棒性和判别性.在有限的有标签数据情况下，拉普拉斯阶梯网络将监督学习损失和非监督损失融合到了统一的框架进行半监督学习.在标准手写数据数据集MNIST和物体识别数据集CIFAR-10上进行了实验，结果表明，相对于阶梯网络和其他半监督方法，拉普拉斯阶梯网络都得到了更好的分类效果，是一种有效的半监督学习算法.     

Applying semi-supervised learning in hierarchical multi-label classification

In classification problems with hierarchical structures of labels, the target function must assign labels that are hierarchically organized and it can be used either for single-label (one label per instance) or multi-label classification problems (more than one label per instance). In parallel to these developments, the idea of semi-supervised learning has emerged as a solution to the problems found in a standard supervised learning procedure (used in most classification algorithms). It combines labelled and unlabelled data during the training phase. Some semi-supervised methods have been proposed for single-label classification methods. However, very little effort has been done in the context of multi-label hierarchical classification. Therefore, this paper proposes a new method for supervised hierarchical multi-label classification, called HMC-RAkEL. Additionally, we propose the use of semi-supervised learning, self-training, in hierarchical multi-label classification, leading to three new methods, called HMC-SSBR, HMC-SSLP and HMC-SSRAkEL. In order to validate the feasibility of these methods, an empirical analysis will be conducted, comparing the proposed methods with their corresponding supervised versions. The main aim of this analysis is to observe whether the semi-supervised methods proposed in this paper have similar performance of the corresponding supervised versions.     

基于监督学习的稀疏编码及在数据表示中的应用

针对稀疏编码在数据表示时没有利用样本类别信息的问题,提出一种基于监督学习的稀疏编码算法,并应用于数据表示.首先利用样本的类别信息构建图,直接提取样本的鉴别结构信息;然后利用基向量拟合鉴别结构特性向量,进而在基向量中嵌入样本的鉴别信息;最后对样本逐个进行稀疏表示.在COIL20和PIE图像库的实验结果表明,相比几种无监督矩阵分解算法,所提出的算法更利于样本的表示和分类.     

基于采样的半监督支持向量机软件缺陷预测方法

软件缺陷预测有助于提高软件开发质量,保证测试资源有效分配。针对软件缺陷预测研究中类标签数据难以获取和类不平衡分布问题,提出基于采样的半监督支持向量机预测模型。该模型采用无监督的采样技术,确保带标签样本数据中缺陷样本数量不会过低,使用半监督支持向量机方法,在少量带标签样本数据基础上利用无标签数据信息构建预测模型;使用公开的NASA软件缺陷预测数据集进行仿真实验。实验结果表明提出的方法与现有半监督方法相比,在综合评价指标[F]值和召回率上均优于现有方法;与有监督方法相比,能在学习样本较少的情况下取得相当的预测性能。     

Identification of non-functional requirements in textual specifications: A semi-supervised learning approach

ContextEarly detection of non-functional requirements (NFRs) is crucial in the evaluation of architectural alternatives starting from initial design decisions. The application of supervised text categorization strategies for requirements expressed in natural language has been proposed in several works as a method to help analysts in the detection and classification of NFRs concerning different aspects of software. However, a significant number of pre-categorized requirements are needed to train supervised text classifiers, which implies that analysts have to manually assign categories to numerous requirements before being able of accurately classifying the remaining ones.ObjectiveWe propose a semi-supervised text categorization approach for the automatic identification and classification of non-functional requirements. Therefore, a small number of requirements, possibly identified by the requirement team during the elicitation process, enable learning an initial classifier for NFRs, which could successively identify the type of further requirements in an iterative process. The goal of the approach is the integration into a recommender system to assist requirement analysts and software designers in the architectural design process.MethodDetection and classification of NFRs is performed using semi-supervised learning techniques. Classification is based on a reduced number of categorized requirements by taking advantage of the knowledge provided by uncategorized ones, as well as certain properties of text. The learning method also exploits feedback from users to enhance classification performance.ResultsThe semi-supervised approach resulted in accuracy rates above 70%, considerably higher than the results obtained with supervised methods using standard collections of documents.ConclusionEmpirical evidence showed that semi-supervision requires less human effort in labeling requirements than fully supervised methods, and can be further improved based on feedback provided by analysts. Our approach outperforms previous supervised classification proposals and can be further enhanced by exploiting feedback provided by analysts.     

Semi-supervised clustering with discriminative random fields

Semi-supervised clustering exploits a small quantity of supervised information to improve the accuracy of data clustering. In this paper, a framework for semi-supervised clustering is proposed. This framework is capable of integrating with a traditional clustering algorithm seamlessly, and particularly useful for the application where a traditional clustering is designated to use.In the proposed framework, discriminative random fields (DRFs) are employed to model the consistency between the result of a traditional clustering algorithm and the supervised information with the assumption of semi-supervised learning. The semi-supervised clustering problem is thus formulated as finding the label configuration with the maximum a posteriori (MAP) probability of the DRF. A procedure based on the iterated conditional modes algorithm and a metric-learning algorithm is developed to find a suboptimal MAP solution of the DRF. The proposed approach has been tested against various data sets. Experimental results demonstrate that our approach can enhance the clustering accuracy, and thus prove the feasibility of the proposed approach.     

基于k个标记样本的弱监督学习框架

聚类是机器学习领域中的一个研究热点,弱监督学习是半监督学习中一个重要的研究方向,有广泛的应用场景.在对聚类与弱监督学习的研究中,提出了一种基于k个标记样本的弱监督学习框架.该框架首先用聚类及聚类置信度实现了标记样本的扩展.其次,对受限玻尔兹曼机的能量函数进行改进,提出了基于k个标记样本的受限玻尔兹曼机学习模型.最后,完成了对该模型的推理并设计相关算法.为了完成对该框架和模型的检验,选择公开的数据集进行对比实验,实验结果表明,基于k个标记样本的弱监督学习框架实验效果较好.     

Active hashing and its application to image and text retrieval

In recent years, hashing-based methods for large-scale similarity search have sparked considerable research interests in the data mining and machine learning communities. While unsupervised hashing-based methods have achieved promising successes for metric similarity, they cannot handle semantic similarity which is usually given in the form of labeled point pairs. To overcome this limitation, some attempts have recently been made on semi-supervised hashing which aims at learning hash functions from both metric and semantic similarity simultaneously. Existing semi-supervised hashing methods can be regarded as passive hashing since they assume that the labeled pairs are provided in advance. In this paper, we propose a novel framework, called active hashing, which can actively select the most informative labeled pairs for hash function learning. Specifically, it identifies the most informative points to label and constructs labeled pairs accordingly. Under this framework, we use data uncertainty as a measure of informativeness and develop a batch mode algorithm to speed up active selection. We empirically compare our method with a state-of-the-art passive hashing method on two benchmark data sets, showing that the proposed method can reduce labeling cost as well as overcome the limitations of passive hashing.     

监督与半监督学习下的数据流集成分类综述

在监督或半监督学习的条件下对数据流集成分类进行研究是一个很有意义的方向.从基分类器、关键技术、集成策略等三个方面进行介绍,其中,基分类器主要介绍了决策树、神经网络、支持向量机等;关键技术从增量、在线等方面介绍;集成策略主要介绍了boosting、stacking等.对不同集成方法的优缺点、对比算法和实验数据集进行了总结与分析.最后给出了进一步研究方向,包括监督和半监督学习下对于概念漂移的处理、对于同质集成和异质集成的研究,无监督学习下的数据流集成分类等.     

SemiBoost: Boosting for Semi-Supervised Learning

Semi-supervised learning has attracted a significant amount of attention in pattern recognition and machine learning. Most previous studies have focused on designing special algorithms to effectively exploit the unlabeled data in conjunction with labeled data. Our goal is to improve the classification accuracy of any given supervised learning algorithm by using the available unlabeled examples. We call this as the Semi-supervised improvement problem, to distinguish the proposed approach from the existing approaches. We design a metasemi-supervised learning algorithm that wraps around the underlying supervised algorithm and improves its performance using unlabeled data. This problem is particularly important when we need to train a supervised learning algorithm with a limited number of labeled examples and a multitude of unlabeled examples. We present a boosting framework for semi-supervised learning, termed as SemiBoost. The key advantages of the proposed semi-supervised learning approach are: 1) performance improvement of any supervised learning algorithm with a multitude of unlabeled data, 2) efficient computation by the iterative boosting algorithm, and 3) exploiting both manifold and cluster assumption in training classification models. An empirical study on 16 different data sets and text categorization demonstrates that the proposed framework improves the performance of several commonly used supervised learning algorithms, given a large number of unlabeled examples. We also show that the performance of the proposed algorithm, SemiBoost, is comparable to the state-of-the-art semi-supervised learning algorithms.     

基于半监督学习和规则相结合的中医古籍命名实体识别研究

目前针对中医古籍实体识别研究较少,且大多使用有监督学习方法。但古籍数字化程度低、标注语料稀少,且其语言多为文言文,专业术语也不断发展,现有方法无法有效解决以上问题。故而,该文在构建了中医古籍语料库的基础上,通过对中医古籍中实体名的分析研究,提出了一种基于半监督学习和规则相结合的中医古籍实体识别方法。以条件随机场模型为基本框架,在引入词、词性、词典等有监督特征的同时也引入了通过词向量获得的无监督语义特征,对比不同特征组合的识别性能,确定最优的半监督学习模型,并与其他模型进行了对比。之后,结合古籍语言学特点构建规则库对其进行基于规则的后处理。实验结果中最终F值达到83.18%,证明了该方法的有效性。     

基于半监督学习的中文社交文本事件聚类方法

面向社交媒体的事件聚类旨在根据事件特征实现短文本聚类.目前,事件聚类模型主要分为无监督模型和有监督模型.无监督模型聚类效果较差,有监督聚类模型依赖大量标注数据.基于此,该文提出了一种半监督事件聚类模型(SemiEC),该模型在小规模标注数据的基础上,利用LSTM表征事件,并基于线性模型计算文本相似度,进行增量聚类.然后...     

Text classification method based on self-training and LDA topic models

Supervised text classification methods are efficient when they can learn with reasonably sized labeled sets. On the other hand, when only a small set of labeled documents is available, semi-supervised methods become more appropriate. These methods are based on comparing distributions between labeled and unlabeled instances, therefore it is important to focus on the representation and its discrimination abilities. In this paper we present the ST LDA method for text classification in a semi-supervised manner with representations based on topic models. The proposed method comprises a semi-supervised text classification algorithm based on self-training and a model, which determines parameter settings for any new document collection. Self-training is used to enlarge the small initial labeled set with the help of information from unlabeled data. We investigate how topic-based representation affects prediction accuracy by performing NBMN and SVM classification algorithms on an enlarged labeled set and then compare the results with the same method on a typical TF-IDF representation. We also compare ST LDA with supervised classification methods and other well-known semi-supervised methods. Experiments were conducted on 11 very small initial labeled sets sampled from six publicly available document collections. The results show that our ST LDA method, when used in combination with NBMN, performed significantly better in terms of classification accuracy than other comparable methods and variations. In this manner, the ST LDA method proved to be a competitive classification method for different text collections when only a small set of labeled instances is available. As such, the proposed ST LDA method may well help to improve text classification tasks, which are essential in many advanced expert and intelligent systems, especially in the case of a scarcity of labeled texts.