BASSUM: A Bayesian semi-supervised method for classification feature selection

Feature selection is an important preprocessing step for building efficient, generalizable and interpretable classifiers on high dimensional data sets. Given the assumption on the sufficient labelled samples, the Markov Blanket provides a complete and sound solution to the selection of optimal features, by exploring the conditional independence relationships among the features. In real-world applications, unfortunately, it is usually easy to get unlabelled samples, but expensive to obtain the corresponding accurate labels on the samples. This leads to the potential waste of valuable classification information buried in unlabelled samples.In this paper, we propose a new BAyesian Semi-SUpervised Method, or BASSUM in short, to exploit the values of unlabelled samples on classification feature selection problem. Generally speaking, the inclusion of unlabelled samples helps the feature selection algorithm on (1) pinpointing more specific conditional independence tests involving fewer variable features and (2) improving the robustness of individual conditional independence tests with additional statistical information. Our experimental results show that BASSUM enhances the efficiency of traditional feature selection methods and overcomes the difficulties on redundant features in existing semi-supervised solutions.     

Spatial-spectral locality preserving projection for hyperspectral image classification with limited training samples

Locality preserving projection (LPP) is a popular unsupervised feature extraction (FE) method. In this paper, the spatial-spectral LPP (SSLPP) method is proposed, which uses both the spectral and spatial information of hyperspectral image (HSI) for FE. The proposed method consists of two parts. In the first part, unlabelled samples are selected in a spatially homogeneous neighbourhood from filtered HSI. In the second part, the transformation matrix is calculated by an LPP-based method and by using the spectral and spatial information of the selected unlabelled samples. Experimental results on Indian Pines (IP), Kennedy Space Center (KSC), and Pavia University (PU) datasets show that the performance of SSLPP is superior to spectral unsupervised, supervised, and semi-supervised FE methods in small and large sample size situations. Moreover, the proposed method outperforms other spatial-spectral semi-supervised FE methods for PU dataset, which has high spatial resolution. For IP and KSC datasets, spectral regularized local discriminant embedding (SSRLDE) has the best performance by using spectral and spatial information of labelled and unlabelled samples, and SSLPP is ranked just behind it. Experiments show that SSLPP is an efficient unsupervised FE method, which does not use training samples as preparation of them is so difficult, costly, and sometimes impractical. SSLPP results are much better than LPP. Also, it decreases the storage and calculation costs using less number of unlabelled samples.     

A novel active learning approach for the classification of hyperspectral imagery using quasi-Newton multinomial logistic regression

In this article, a novel active learning approach is proposed for the classification of hyperspectral imagery using quasi-Newton multinomial logistic regression/Davidon, Fletcher, and Powell selective variance (MLR-DFP-SV). The proposed approach consists of two main steps: (1) a fast solution for the MLR classifier, where the logistic regressors are obtained by the use of the quasi-Newton algorithm; and (2) selection of the most informative unlabelled samples. The SV method is applied to select the most informative unlabelled samples, based on the posterior density distributions. Experiments on two real hyperspectral data sets confirmed that the proposed approach can effectively select the most informative unlabelled samples and improve the classification accuracy. Three different methods – the maximum information (MI), breaking ties (BT), and minimum error (ME) methods – were also used to obtain the most informative unlabelled samples, and it was found that the new sample selection method – SV – can select more informative samples than the BT, MI, and ME methods.     

基于高维映射Fisher判别分析的图像分割

为提高Fisher判别分析的质量,对图像中各像素本身的灰度值及其邻域平均灰度值特征进行两步聚类分析,根据聚类结果选取Fisher判别分析所需的训练样本,同时为了尽可能降低判别分析过程中有用信息的损失,将所得到的原训练样本集进行非线性变换,使其映射到高维空间中,利用映射后的训练样本求得Fisher判别规则。实验结果表明,与基于原训练样本的Fisher判别分析和基于寻找更多样本特征的Fisher判别分析方法生成结果相比,该方法能够获得更好的图像分割精度。     

结合主动学习与置信度投票的集成自训练方法

基于集成学习的自训练算法是一种半监督算法,不少学者通过集成分类器类别投票或平均置信度的方法选择可靠样本。基于置信度的投票策略倾向选择置信度高的样本或置信度低但投票却一致的样本进行标记,后者这种情形可能会误标记靠近决策边界的样本,而采用异构集成分类器也可能会导致各基分类器对高置信度样本的类别标记不同,从而无法将其有效加入到有标记样本集。提出了结合主动学习与置信度投票策略的集成自训练算法用来解决上述问题。该算法合理调整了投票策略,选择置信度高且投票一致的无标记样本加以标注,同时利用主动学习对投票不一致而置信度较低的样本进行人工标注,以弥补集成自训练学习只关注置信度高的样本,而忽略了置信度低的样本的有用信息的缺陷。在UCI数据集上的对比实验验证了该算法的有效性。     

Fault diagnosis of a railway device using semi-supervised independent factor analysis with mixing constraints

Independent factor analysis (IFA) defines a generative model for observed data that are assumed to be linear mixtures of some unknown non-Gaussian, mutually independent latent variables (also called sources or independent components). The probability density function of each individual latent variable is modelled by a mixture of Gaussians. Learning in the context of this model is usually performed within an unsupervised framework in which only unlabelled samples are used. Both the mixing matrix and the parameters of latent variable densities are learned from the observed data. This paper investigates the possibility of estimating an IFA model in a noiseless setting when two kinds of prior information are incorporated, namely constraints on the mixing process and partial knowledge on the cluster membership of some training samples. Semi-supervised or partially supervised learning frameworks can thus be handled. The investigation of these two kinds of prior information was motivated by a real-world application concerning the fault diagnosis of railway track circuits. Simulated data, resulting from both these applications, are provided to demonstrate the capacity of our approach to enhance estimation accuracy and remove the indeterminacy commonly encountered in unsupervised IFA, such as source permutations.     

Wrapper positive Bayesian network classifiers

In the information retrieval framework, there are problems where the goal is to recover objects of a particular class from big sets of unlabelled objects. In some of these problems, only examples from the class we want to recover are available. For such problems, the machine learning community has developed algorithms that are able to learn binary classifiers in the absence of negative examples. Among them, we can find the positive Bayesian network classifiers, algorithms that induce Bayesian network classifiers from positive and unlabelled examples. The main drawback of these algorithms is that they require some previous knowledge about the a priori probability distribution of the class. In this paper, we propose a wrapper approach to tackle the learning when no such information is available, setting this probability at the optimal value in terms of the recovery of positive examples. The evaluation of classifiers in positive unlabelled learning problems is a non-trivial question. We have also worked on this problem, and we have proposed a new guiding metric to be used in the search for the optimal a priori probability of the positive class that we have called the pseudo F. We have empirically tested the proposed metric and the wrapper classifiers on both synthetic and real-life datasets. The results obtained in this empirical comparison show that the wrapper Bayesian network classifiers provide competitive results, particularly when the actual a priori probability of the positive class is high.     

Optimum tradeoff between class-selective rejection error and average number of classes

The paper first reviews the recently proposed optimum class-selective rejection rule. This rule provides an optimum tradeoff between the error rate and the average number of (selected) classes. Then, a new general relation between the error rate and the average number of classes is presented. The error rate can be directly computed from the class-selective reject function, which in turn can be estimated from unlabelled patterns, by simply counting the rejected classes. Theoretical as well as practical implications are discussed, and some future research directions are proposed.     

Scene-adaptive accurate and fast vertical crowd counting via joint using depth and color information

Reliable and real-time crowd counting is one of the most important tasks in intelligent visual surveillance systems. Most previous works only count passing people based on color information. Owing to the restrictions of color information influences themselves for multimedia processing, they will be affected inevitably by the unpredictable complex environments (e.g. illumination, occlusion, and shadow). To overcome this bottleneck, we propose a new algorithm by multimodal joint information processing for crowd counting. In our method, we use color and depth information together with a ordinary depth camera (e.g. Microsoft Kinect). Specifically, we first detect each head of the passing or still person in the surveillance region with adaptive modulation ability to varying scenes on depth information. Then, we track and count each detected head on color information. The characteristic advantage of our algorithm is that it is scene adaptive, which means the algorithm can be applied into all kinds of different scenes directly without additional conditions. Based on the proposed approach, we have built a practical system for robust and fast crowd counting facing complicated scenes. Extensive experimental results show the effectiveness of our proposed method.     

基于主动学习的文档分类

In the field of text categorization,the number of unlabeled documents is generally much gretaer than that of labeled documents. Text categorization is the problem of categorization in high-dimension vector space, and more training samples will generally improve the accuracy of text classifier. How to add the unlabeled documents of training set so as to expand training set is a valuable problem. The theory of active learning is introducted and applied to the field of text categorization in this paper ,exploring the method of using unlabeled documents to improve the accuracy oftext classifier. It is expected that such technology will improve text classifier's accuracy through adopting relativelylarge number of unlabelled documents samples. We brought forward an active learning based algorithm for text categorization,and the experiments on Reuters news corpus showed that when enough training samples available,it′s effective for the algorithm to promote text classifier's accuracy through adopting unlabelled document samples.     

Pattern Matching based Classification using Ant Colony Optimization based Feature Selection

Classification is a method of accurately predicting the target class for an unlabelled sample by learning from instances described by a set of attributes and a class label. Instance based classifiers are attractive due to their simplicity and performance. However, many of these are susceptible to noise and become unsuitable for real world problems. This paper proposes a novel instance based classification algorithm called Pattern Matching based Classification (PMC). The underlying principle of PMC is that it classifies unlabelled samples by matching for patterns in the training dataset. The advantage of PMC in comparison with other instance based methods is its simple classification procedure together with high performance. To improve the classification accuracy of PMC, an Ant Colony Optimization based Feature Selection algorithm based on the idea of PMC has been proposed. The classifier is evaluated on 35 datasets. Experimental results demonstrate that PMC is competent with many instance based classifiers. The results are also validated using nonparametric statistical tests. Also, the evaluation time of PMC is less when compared to the gravitation based methods used for classification.     

Classification on defective items using unidentified samples

A set of unlabelled items is used to establish a decision rule to classify defective items. The lifetime of an item has an exponential distribution. It is known that the Bayes decision rule, which classifies good and defective items, gives a minimum probability of misclassification. The Bayes decision rule needs to know the prior probability (defective percentage) and two mean lifetimes. In the set of unidentified samples, the defective percentage and two mean lifetimes are unknown. Hence, before we can use the Bayes decision rule, we have to estimate the three unknown parameters. In this study, a set of unlabelled samples is used to estimate the three unknown parameters. The Bayes decision rule with these estimated parameters is an empirical Bayes (EB) decision rule. A stochastic approximation procedure using the set of unidentified samples is established to estimate the three unknown parameters. When the size of unlabelled items increases, the estimates computed by the procedure converge to the real parameters and hence gradually adapt our EB decision rule to be a better classifier until it becomes the Bayes decision rule. The results of a Monte Carlo simulation study are presented to demonstrate the convergence of the correct classification rates made by the EB decision rule to the highest correct classification rates given by the Bayes decision rule.     

生物序列数据K-mer频次统计问题的算法

生物序列的k-mer频次统计是生物信息处理中一个非常基础且重要的问题. 本文针对多序列在对齐模式下,不同偏移处一段长度范围内的k-mer频次统计问题进行了研究. 提出了一种逆向遍历k-mer计数算法BTKC. 该算法能够充分利用长度的k-mer统计信息,快速得到长度的k-mer统计信息,从而避免了统计任意长度的k-mer频次信息时都需要对所有序列进行遍历. 算法的时间复杂度分析及实验结果表明,相比于传统的前向遍历FTKC算法,BTKC算法性能提升非常明显,且其时间复杂度与k-mer长度的变化范围无关,非常适合于在k-mer长度变化范围较大的情况下使用.     

基于基因库的新型检测器生成方法

提出一种新的检测器生成方法。由于非我样本中存在着关于非我空间的信息,提出通过统计非我样本中各属性的分布情况来构建基因库,并应用基因库来生成检测器的方法来检测入侵。应用KDDCup1999数据集,通过实验证明该方法能够生成检测率更高的检测器集。     

六元一阶相关免疫函数的新计数算法

若布尔函数的输出不泄漏其输入值的有关信息,则该函数是相关免疫的。该文基于列平衡矩阵研究相关免疫函数的计数问题,利用穷举和统计相结合的方法对2k×6(0≤k≤16)阶定序列平衡矩阵进行计数,给出一种新的六元一阶相关免疫函数的计数算法。与同类算法相比,新算法的复杂度降为(224),大大提高了一阶相关免疫函数的计数效率。     

Simple recurrent networks learn context-free and context-sensitive languages by counting

It has been shown that if a recurrent neural network (RNN) learns to process a regular language, one can extract a finite-state machine (FSM) by treating regions of phase-space as FSM states. However, it has also been shown that one can construct an RNN to implement Turing machines by using RNN dynamics as counters. But how does a network learn languages that require counting? Rodriguez, Wiles, and Elman (1999) showed that a simple recurrent network (SRN) can learn to process a simple context-free language (CFL) by counting up and down. This article extends that to show a range of language tasks in which an SRN develops solutions that not only count but also copy and store counting information. In one case, the network stores information like an explicit storage mechanism. In other cases, the network stores information more indirectly in trajectories that are sensitive to slight displacements that depend on context. In this sense, an SRN can learn analog computation as a set of interdependent counters. This demonstrates how SRNs may be an alternative psychological model of language or sequence processing.     

基于互信息量的分类模型

针对分类数据集中属性之间的相关性及每个属性取值对属性权值的贡献程度的差别,提出基于互信息量的分类模型以及影响因子与样本预测信息量的计算公式,并利用样本预测信息量预测分类标号。经实验证明,基于互信息量的分类模型可以有效地提高分类算法的预测精度和准确率。     

动作识别训练数据的扩展研究

动作识别是康复中心研究领域的一个热门话题。机器学习是动作识别的一个重要方面。由于样本标注需要付出诸多人工努力,所以被标注的样本数量是有限的。而未被标注样本数量是庞大缘于它容易获取,无需人为注解。训练数据是基于半监督学习动作识别的核心。文章将着重强调数据选择策略和扩展度,这也是训练数据选择的基础。文章结合已标注的有限样本,利用未被标注样本来提高动作识别的精度。