Fusion of probabilistic knowledge-based classification rules and learning automata for automatic recognition of digital images

In this paper, the fusion of probabilistic knowledge-based classification rules and learning automata theory is proposed and as a result we present a set of probabilistic classification rules with self-learning capability. The probabilities of the classification rules change dynamically guided by a supervised reinforcement process aimed at obtaining an optimum classification accuracy. This novel classifier is applied to the automatic recognition of digital images corresponding to visual landmarks for the autonomous navigation of an unmanned aerial vehicle (UAV) developed by the authors. The classification accuracy of the proposed classifier and its comparison with well-established pattern recognition methods is finally reported.     

基于SOM-PNN分类器的体数据概率分类及绘制

概率分类是三维医学体数据绘制必不可少的预处理环节。本文提出的ＳＯＭ－ＰＮＮ分类器，以贝叶斯置信度为基础，给出概率分类结果，并用于三维体制制，得到了良好的图像质量和较高的分类效率。     

基于FCM-PNN分类器的说话人识别

说话人识别的本质就是模式分类。传统分类器算法中参数模型方法的主要缺点是预先假定的概率分布函数形式不一定符合待分类的数据。非参数模型方法,如PNN分类器,可以有效地克服参数模型的缺点,但其巨大的内存开销与低的分类速度使得PNN作为大量和高维的数据样本分类几乎不可行。FCM虽具有良好的模糊聚类能力,但无法直接给出概率分类结果。该文提出的FCM-PNN分类器,在FCM聚类的基础上,以贝叶斯置信度为基础,利用PNN进行概率分类。它结合了FCM聚类和PNN概率分类的优势,同时克服了传统参数模型分类和FCM聚类的局限性。实验结果证实了FCM-PNN分类器具有分类精度高、速度快及揭示细节的能力。     

A relational approach to probabilistic classification in a transductive setting

Transduction is an inference mechanism adopted from several classification algorithms capable of exploiting both labeled and unlabeled data and making the prediction for the given set of unlabeled data only. Several transductive learning methods have been proposed in the literature to learn transductive classifiers from examples represented as rows of a classical double-entry table (or relational table). In this work we consider the case of examples represented as a set of multiple tables of a relational database and we propose a new relational classification algorithm, named TRANSC, that works in a transductive setting and employs a probabilistic approach to classification. Knowledge on the data model, i.e., foreign keys, is used to guide the search process. The transductive learning strategy iterates on a k-NN based re-classification of labeled and unlabeled examples, in order to identify borderline examples, and uses the relational probabilistic classifier Mr-SBC to bootstrap the transductive algorithm. Experimental results confirm that TRANSC outperforms its inductive counterpart (Mr-SBC).     

On estimating simple probabilistic discriminative models with subclasses

Discriminative subclass models can provide good estimates of complex ‘continuous to discrete’ conditional probabilities for hybrid Bayesian network models. However, the conventional approach of specifying deterministic ‘hard’ subclasses via unsupervised clustering can lead to inaccurate models. The multimodal softmax (MMS) model is presented as a new probabilistic discriminative subclass model that overcomes this unreliability. By invoking fully probabilistic latent ‘soft’ subclasses, MMS permits learning via standard statistical methods without requiring explicit clustering/relabeling of data. MMS is also shown to be closely related to the mixture of experts model and the generative Gaussian mixture classifier. Synthetic and benchmark classification results demonstrate the MMS model’s correctness and usefulness for hybrid probabilistic modeling.     

A hierarchical multiple classifier learning algorithm

This paper addresses the classification problem for applications with extensive amounts of data and a large number of features. The learning system developed utilizes a hierarchical multiple classifier scheme and is flexible, efficient, highly accurate and of low cost. The system has several novel features: (1) It uses a graph-theoretic clustering algorithm to group the training data into possibly overlapping cluster, each representing a dense region in the data space; (2) component classifiers trained on these dense regions are specialists whose probabilistic outputs are gated inputs to a super-classifier. Only those classifiers whose training clusters are most related to an unknown data instance send their outputs to the super-classifier; and (3) sub-class labelling is used to improve the classification of super-classes. The learning system achieves the goals of reducing the training cost and increasing the prediction accuracy compared to other multiple classifier algorithms. The system was tested on three large sets of data, two from the medical diagnosis domain and one from a forest cover classification problem. The results are superior to those obtained by several other learning algorithms.     

Monitoring the Formation of Kernel-Based Topographic Maps in a Hybrid SOM-kMER Model

A new lattice disentangling monitoring algorithm for a hybrid self-organizing map-kernel-based maximum entropy learning rule (SOM-kMER) model is proposed. It aims to overcome topological defects owing to a rapid decrease of the neighborhood range over the finite running time in topographic map formation. The empirical results demonstrate that the proposed approach is able to accelerate the formation of a topographic map and, at the same time, to simplify the monitoring procedure.     

Supervised classification using probabilistic decision graphs

A new model for supervised classification based on probabilistic decision graphs is introduced. A probabilistic decision graph (PDG) is a graphical model that efficiently captures certain context specific independencies that are not easily represented by other graphical models traditionally used for classification, such as the Naïve Bayes (NB) or Classification Trees (CT). This means that the PDG model can capture some distributions using fewer parameters than classical models. Two approaches for constructing a PDG for classification are proposed. The first is to directly construct the model from a dataset of labelled data, while the second is to transform a previously obtained Bayesian classifier into a PDG model that can then be refined. These two approaches are compared with a wide range of classical approaches to the supervised classification problem on a number of both real world databases and artificially generated data.     

Robust supervised classification with mixture models: Learning from data with uncertain labels

In the supervised classification framework, human supervision is required for labeling a set of learning data which are then used for building the classifier. However, in many applications, human supervision is either imprecise, difficult or expensive. In this paper, the problem of learning a supervised multi-class classifier from data with uncertain labels is considered and a model-based classification method is proposed to solve it. The idea of the proposed method is to confront an unsupervised modeling of the data with the supervised information carried by the labels of the learning data in order to detect inconsistencies. The method is able afterward to build a robust classifier taking into account the detected inconsistencies into the labels. Experiments on artificial and real data are provided to highlight the main features of the proposed method as well as an application to object recognition under weak supervision.     

PARTICLE SWARM OPTIMIZATION-BASED FEATURE SELECTION AND PARAMETER OPTIMIZATION FOR POWER SYSTEM DISTURBANCES CLASSIFICATION

In many data mining applications that address classification problems, feature and model selection are considered as key tasks. The appropriate input features of the classifier are selected from a given set of possible features, and the structure parameters of the classifier are adapted with respect to these features and a given dataset. This paper describes the particle swarm optimization algorithm (PSO) that performs feature and model selection simultaneously for the probabilistic neural network (PNN) classifier for power system disturbances. The probabilistic neural network is one of the successful classifiers used to solve many classification problems. However, the computational effort and storage requirement of the PNN method will prohibitively increase as the number of patterns used in the training set increases. An important issue that has not been given enough attention is the selection of a “spread parameter,” also called a “smoothing parameter,” in the PNN classifier. PSO is a powerful meta-heuristic technique in the artificial intelligence field; therefore, this study proposes a PSO-based approach, called PSO-PNN, to specify the beneficial features and the value of spread parameter to enhance the performance of PNN. The experimental results indicate that the proposed PSO-based approach significantly improves the classification accuracy with the discriminating input features for PNN.     

Classification in the presence of class noise using a probabilistic Kernel Fisher method

In machine learning, class noise occurs frequently and deteriorates the classifier derived from the noisy data set. This paper presents two promising classifiers for this problem based on a probabilistic model proposed by Lawrence and Schölkopf (2001). The proposed algorithms are able to tolerate class noise, and extend the earlier work of Lawrence and Schölkopf in two ways. First, we present a novel incorporation of their probabilistic noise model in the Kernel Fisher discriminant; second, the distribution assumption previously made is relaxed in our work. The methods were investigated on simulated noisy data sets and a real world comparative genomic hybridization (CGH) data set. The results show that the proposed approaches substantially improve standard classifiers in noisy data sets, and achieve larger performance gain in non-Gaussian data sets and small size data sets.     

基于图谱分解和概率神经网络的图像分类

为了准确地对不同学习样本数的图像进行分类,首先讨论了对属于不同类别的图像序列如何进行正确有监督分类的问题,由于解决这类问题首先要选取适合图像分类的图像特征作为分类的依据,为此先用图像角点来构成Delaunay图,然后将由Delaunay图的谱特征形成的分类特征矢量作为分类的依据;其次,由于分类器的选取也直接影响分类结果,为此采用了学习效率高的概率神经网络分类器来进行分类.经过大量分类实验表明,图谱特征很好地保持了图像的结构特征,是理想的图像分类特征;经过与其他相关分类器的分类比较实验表明,基于概率神经网络的分类器可以准确地进行图像分类;通过不同学习样本数的比较,证实了概率神经网络在进行图像分类时,对于学习样本数并不敏感,并具有一定稳定性.     

基于在线增量学习的自适应聚焦爬虫研究

在原始分类器聚焦爬虫的基础上设计并实现在线增量学习的自适应聚焦爬虫.该聚焦爬虫包括一个基础网页分类器和一个在线增量学习自适应链接分类器.基础页面分类器根据领域知识对抓取到的页面内容主题相关性进行分类.在线增量学习自适应链接分类器能即时根据爬虫爬得网页和网页链接信息作出分类模型调整以更合理方式计算链接的主题相关度.系统中链接排序模块采用TopicalRank主题相关度计算方法分析链接优先抓取顺序.把基于增量学习的自适应聚焦爬虫应用到农业领域,实验结果和分析证明在线增量学习的自适应聚焦爬虫在农业领域爬行性能比仅基于网页相关性和链接重要度的原始分类器聚焦爬虫具有更好的性能.     

基于朴素贝叶斯的半监督学习遥感影像分类

为提高分类器识别率,减少标注样本使用数量,提出一种基于朴素贝叶斯的半监督学习方法。研究基于该方法的分类器分类效果,采用遥感影像数据作为训练和测试集,与基于朴素贝叶斯的全监督学习分类器分类效果作比较。实验结果表明,当标注样本与非标注样本比例在1:2~1:9时,半监督学习可以利用比全监督学习更少的标注样本,达到更高的分类精度。     

Efficient Venn predictors using random forests

Successful use of probabilistic classification requires well-calibrated probability estimates, i.e., the predicted class probabilities must correspond to the true probabilities. In addition, a probabilistic classifier must, of course, also be as accurate as possible. In this paper, Venn predictors, and its special case Venn-Abers predictors, are evaluated for probabilistic classification, using random forests as the underlying models. Venn predictors output multiple probabilities for each label, i.e., the predicted label is associated with a probability interval. Since all Venn predictors are valid in the long run, the size of the probability intervals is very important, with tighter intervals being more informative. The standard solution when calibrating a classifier is to employ an additional step, transforming the outputs from a classifier into probability estimates, using a labeled data set not employed for training of the models. For random forests, and other bagged ensembles, it is, however, possible to use the out-of-bag instances for calibration, making all training data available for both model learning and calibration. This procedure has previously been successfully applied to conformal prediction, but was here evaluated for the first time for Venn predictors. The empirical investigation, using 22 publicly available data sets, showed that all four versions of the Venn predictors were better calibrated than both the raw estimates from the random forest, and the standard techniques Platt scaling and isotonic regression. Regarding both informativeness and accuracy, the standard Venn predictor calibrated on out-of-bag instances was the best setup evaluated. Most importantly, calibrating on out-of-bag instances, instead of using a separate calibration set, resulted in tighter intervals and more accurate models on every data set, for both the Venn predictors and the Venn-Abers predictors.

     

基于GAN-AdaBoost-DT不平衡分类算法的信用卡欺诈分类

针对传统单个分类器在不平衡数据上分类效果有限的问题，基于对抗生成网络（GAN）和集成学习方法，提出一种新的针对二类不平衡数据集的分类方法——对抗生成网络-自适应增强-决策树（GAN-AdaBoost-DT）算法。首先，利用GAN训练得到生成模型，生成模型生成少数类样本，降低数据的不平衡性；其次，将生成的少数类样本代入自适应增强（AdaBoost）模型框架，更改权重，改进AdaBoost模型，提升以决策树（DT）为基分类器的AdaBoost模型的分类性能。使用受测者工作特征曲线下面积（AUC）作为分类评价指标，在信用卡诈骗数据集上的实验分析表明，该算法与合成少数类样本集成学习相比，准确率提高了4.5%，受测者工作特征曲线下面积提高了6.5%；对比改进的合成少数类样本集成学习，准确率提高了4.9%，AUC值提高了5.9%；对比随机欠采样集成学习，准确率提高了4.5%，受测者工作特征曲线下面积提高了5.4%。在UCI和KEEL的其他数据集上的实验结果表明，该算法在不平衡二分类问题上能提高总体的准确率，优化分类器性能。     

基于贝叶斯学习的集成流量分类方法

NB方法条件独立性假设和BAN方法小训练集难以建模。为此,提出一种基于贝叶斯学习的集成流量分类方法。构造单独的NB和BAN分类器,在此基础上利用验证集得到各分类器的权重,通过加权平均组合各分类器的输出,实现网络流量分类。以Moore数据集为实验数据,并与NB方法和BAN方法相比较,结果表明,该方法具有更高的分类准确率和稳定性。     

面向概念漂移数据流的自适应分类算法

数据流分类是数据挖掘中最重要的任务之一,而数据流的概念漂移特性给分类算法带来了巨大的挑战.基于极限学习机算法进行优化是解决数据流分类问题的一个热门方向,但目前大多数算法都采用提前指定模型参数的方式进行学习,这种做法使得分类模型只能在特定的数据集上才能发挥较好的性能.针对这一问题,提出了一种简单有效的处理概念漂移的算法——自适应在线顺序极限学习机分类算法.算法通过引入自适应模型复杂度机制,从而具有更好的分类性能.然后通过引入自适应遗忘因子与概念漂移检测机制,能够根据动态变化的数据流进行自适应学习,从而可以更好地适应概念漂移.进一步还引入异常点检测机制,避免分类决策边界被异常点破坏.仿真实验表明,所提出算法比同类算法具有更好的稳定性、分类准确性以及概念漂移适应能力.此外,还通过消融实验证实了算法所引入3个机制的有效性.     

Time integration and reject options for probabilistic output of pairwise LVQ

Learning vector quantization (LVQ) constitutes a very popular machine learning technology with applications, for example, in biomedical data analysis, predictive maintenance/quality as well as product individualization. Albeit probabilistic LVQ variants exist, its deterministic counterparts are often preferred due to their better efficiency. The latter do not allow an immediate probabilistic interpretation of its output; hence, a rejection of classification based on confidence values is not possible. In this contribution, we investigate different schemes how to extend and integrate pairwise LVQ schemes to an overall probabilistic output, in comparison with a recent heuristic surrogate measure for the security of the classification, which is directly based on LVQ’s multi-class classification scheme. Furthermore, we propose a canonic way how to fuse these values over a given time window in case a possibly disrupted measurement is taken over a longer time interval to counter the uncertainty of a single point in time. Experimental results indicate that an explicit probabilistic treatment often yields superior results as compared to a standard deterministic LVQ method, but metric learning is able to annul this difference. Fusion over a short time period is beneficial in case of an unclear classification.

     

CNN-ELM混合短文本分类模型

针对目前自然语言处理研究中,使用卷积神经网络（CNN）进行短文本分类任务时可以结合不同神经网络结构与分类算法以提高分类性能的问题,提出了一种结合卷积神经网络与极速学习机的CNN-ELM混合短文本分类模型。使用词向量训练构成文本矩阵作为输入数据,然后使用卷积神经网络提取特征并使用Highway网络进行特征优化,最后使用误差最小化极速学习机(EM-ELM)作为分类器完成短文本分类任务。与其他模型相比,该混合模型能够提取更具代表性的特征并能快速准确地输出分类结果。在多种英文数据集上的实验结果表明提出的CNN-ELM混合短文本分类模型比传统机器学习模型与深度学习模型更适合完成短文本分类任务。