一种基于属性分割的产生式/判别式混合分类器

为了利用产生式和判别式方法各自的优势,研究了基于属性分割的产生式/判别式混合分类模型框架,提出了一种基于属性分割的产生式/判别式混合分类器学习算法GDGA。其利用遗传算法,将属性集X划分为两个子集XG和XD,并相应地将训练集D垂直分割为两个子集DG和DD,在两个训练子集上分别学习产生式分类器和判别式分类器;最后将两个分类器合并形成一个混合分类器。实验结果表明,在大多数数据集上,混合分类器的分类正确率优于其成员分类器。在训练数据不足或数据属性分布不清楚的情况下,该混合分类器具有特别的优势。     

贝叶斯分类器的判别式参数学习

为了提高贝叶斯分类器的分类性能,针对贝叶斯网络分类器的构成特征,提出一种基于参数集成的贝叶斯分类器判别式参数学习算法PEBNC。该算法将贝叶斯分类器的参数学习视为回归问题,将加法回归模型应用于贝叶斯网络分类器的参数学习,实现贝叶斯分类器的判别式参数学习。实验结果表明,在大多数实验数据上,PEBNC能够明显提高贝叶斯分类器的分类准确率。此外,与一般的贝叶斯集成分类器相比,PEBNC不必存储成员分类器的参数,空间复杂度大大降低。     

基于标准数据集的分类器融合学习模型

判别式分类器通过生成不同复杂度的指示函数去调节算法与所解决问题的适应性,能有效地避免过拟合现象。分类器融合方法就是应用单个分类器对特定样本预报的特异性来提高模型的整体预测精度,应用支持向量机(SVM)对乳腺癌数据进行建模,通过选取不同的模型参数(径向基核函数参数gamma和正则化约束参数cost)构建9个单分类器,通过投票策略在单分类器上构建融合分类器,融合模型对乳腺癌数据的预测精度为98.59%,相比单分类模型对此数据集的预测精度97.72%有明显的竞争力,试验结果表明融合模型能有效提升分类器的泛化能力。     

基于遗传算法的多扰动多K近邻分类器系统

为改善维数灾难对K近邻分类器的影响,提出一种基于遗传算法(GA)的多扰动的K近邻融合算法,简称GA-MKNNC算法。目标扰动将所识别的问题划分成多个子分类问题进行单独识别。针对不同子分类问题,数据扰动选取相关的数据,特征扰动确定相关的特征,参数扰动明确相关参数值。数据扰动由Bagging算法确定。特征扰动和参数扰动通过GA学习得到。多个子分类问题的决策通过最大融合得到最终决策。实验结果表明,该算法的性能优于K近邻分类器及多数融合算法,且选用的子分类器数目少于FASBIR算法。     

基于K近邻统计的非线性AdaBoost算法

AdaBoost是数据挖掘领域最常见的提升算法之一。对传统AdaBoost将各个基分类器线性相加所存在的不足进行分析,并针对AdaBoost各个弱分类器的加权方式提出新的改进,将传统的线性相加改为非线性组合,把从学习过程得到的固定不变的权重系数改为由预测阶段的具体实例决定的动态参数,该参数基于待测实例K近邻的分类结果统计,从而使各个基分类器的权重更贴近当前待测实例的实际可靠度。实验结果表明,与传统AdaBoost相比,提出的非线性改进算法对不同数据集均有不同程度提升,提升最高的达到了7个百分点。由此证明,提出的改进是一种更加准确的分类算法,对绝大多数数据集均能得到更高的分类准确率。     

一种基于信息熵的多分类器动态组合方法

为提高数据分类的性能,提出了一种基于信息熵[1]的多分类器动态组合方法（EMDA）。此方法在多个UCI标准数据集上进行了测试,并与由集成学习算法—AdaBoost,训练出的各个基分类器的分类效果进行比较,证明了该算法的有效性。     

基于线性判别分析的Choquet积分的符号模糊测度提取

在解决分类问题时,建立在Choquet积分上的分类器以其非线性和不可加性的特点,扮演着越来越重要的角色。由于Choquet积分中的符号模糊测度可以描述各特征对结果的影响,因此Choquet积分在解决数据分类及融合 问题方面具有显著的优势。但是,关于Choquet积分符号模糊测度值的求解,学术界一直缺乏有效的方法。目前最常用的方法是遗传算法,但是遗传算法在解决符号模糊测度值的优化问题时存在算法较为复杂、耗时较长等缺陷。由于符号模糊测度值在Choquet积分分类器中是决定性的重要参数,因此设计出一种有效的符号模糊测度提取方法十分必要。文中提出基于线性判别分析的Choquet积分符号模糊测度的提取方法,推导出在分类问题下Choquet积分的符号模糊测度值的解析式表达,其能够有效、快速地得出关键性参数。分别在人工数据集及基准实际数据集上进行测试与验证,实验结果表明所提方法能有效解决Choquet积分分类器中符号模糊测度的优化问题。     

一种混合遗传算法的设计及其应用研究

针对线性分类器这一狭义模式识别问题,分析影响基于感知器的梯度算法的线性分类器的收敛性问题,提出一种遗传算法和梯度算法相结合的权值训练方法,用于线性分类器的参数设计,给出一种衡量算法优劣的标准,并进行仿真研究。     

基于单类分类器的半监督学习

提出一种结合单类学习器和集成学习优点的Ensemble one-class半监督学习算法.该算法首先为少量有标识数据中的两类数据分别建立两个单类分类器.然后用建立好的两个单类分类器共同对无标识样本进行识别,利用已识别的无标识样本对已建立的两个分类面进行调整、优化.最终被识别出来的无标识数据和有标识数据集合在一起训练一个基分类器,多个基分类器集成在一起对测试样本的测试结果进行投票.在5个UCI数据集上进行实验表明,该算法与tri-training算法相比平均识别精度提高4.5%,与仅采用纯有标识数据的单类分类器相比,平均识别精度提高8.9%.从实验结果可以看出,该算法在解决半监督问题上是有效的.     

不平衡数据的集成分类算法综述

集成学习是通过集成多个基分类器共同决策的机器学习技术,通过不同的样本集训练有差异的基分类器,得到的集成分类器可以有效地提高学习效果。在基分类器的训练过程中,可以通过代价敏感技术和数据采样实现不平衡数据的处理。由于集成学习在不平衡数据分类的优势,针对不平衡数据的集成分类算法得到广泛研究。详细分析了不平衡数据集成分类算法的研究现状,比较了现有算法的差异和各自存在的优点及问题,提出和分析了有待进一步研究的问题。     

Semisupervised learning for a hybrid generative/discriminative classifier based on the maximum entropy principle

This paper presents a method for designing semi-supervised classifiers trained on labeled and unlabeled samples. We focus on probabilistic semi-supervised classifier design for multi-class and single-labeled classification problems, and propose a hybrid approach that takes advantage of generative and discriminative approaches. In our approach, we first consider a generative model trained by using labeled samples and introduce a bias correction model, where these models belong to the same model family, but have different parameters. Then, we construct a hybrid classifier by combining these models based on the maximum entropy principle. To enable us to apply our hybrid approach to text classification problems, we employed naive Bayes models as the generative and bias correction models. Our experimental results for four text data sets confirmed that the generalization ability of our hybrid classifier was much improved by using a large number of unlabeled samples for training when there were too few labeled samples to obtain good performance. We also confirmed that our hybrid approach significantly outperformed generative and discriminative approaches when the performance of the generative and discriminative approaches was comparable. Moreover, we examined the performance of our hybrid classifier when the labeled and unlabeled data distributions were different.     

A hybrid discriminative/generative approach to protein fold recognition

There are two standard approaches to the classification task: generative, which use training data to estimate a probability model for each class, and discriminative, which try to construct flexible decision boundaries between the classes. An ideal classifier should combine these two approaches. In this paper a classifier combining the well-known support vector machine (SVM) classifier with regularized discriminant analysis (RDA) classifier is presented. The hybrid classifier is used for protein structure prediction which is one of the most important goals pursued by bioinformatics. The obtained results are promising, the hybrid classifier achieves better result than the SVM or RDA classifiers alone. The proposed method achieves higher recognition ratio than other methods described in the literature.     

On the generative–discriminative tradeoff approach: Interpretation, asymptotic efficiency and classification performance

The interpretation of generative, discriminative and hybrid approaches to classification is discussed, in particular for the generative–discriminative tradeoff (GDT), a hybrid approach. The asymptotic efficiency of the GDT, relative to that of its generative or discriminative counterpart, is presented theoretically and, by using linear normal discrimination as an example, numerically. On real and simulated datasets, the classification performance of the GDT is compared with those of normal-based linear discriminant analysis (LDA) and linear logistic regression (LLR). Four arguments are made as follows. First, the GDT is a generative model integrating both discriminative and generative learning. It is therefore subject to model misspecification of the data-generating process and hindered by complex optimisation. Secondly, among the three approaches being compared, the asymptotic efficiency of the GDT is higher than that of the discriminative approach but lower than that of the generative approach, when no model misspecification occurs. Thirdly, without model misspecification, LDA performs the best; with model misspecification, LLR or the GDT with an optimal, large weight on its discriminative component may perform the best. Finally, LLR is affected by the imbalance between groups of data.     

Learning semantic concepts from image database with hybrid generative/discriminative approach

Semantic gap has become a bottleneck of content-based image retrieval in recent years. In order to bridge the gap and improve the retrieval performance, automatic image annotation has emerged as a crucial problem. In this paper, a hybrid approach is proposed to learn the semantic concepts of images automatically. Firstly, we present continuous probabilistic latent semantic analysis (PLSA) and derive its corresponding Expectation–Maximization (EM) algorithm. Continuous PLSA assumes that elements are sampled from a multivariate Gaussian distribution given a latent aspect, instead of a multinomial one in traditional PLSA. Furthermore, we propose a hybrid framework which employs continuous PLSA to model visual features of images in generative learning stage and uses ensembles of classifier chains to classify the multi-label data in discriminative learning stage. Therefore, the framework can learn the correlations between features as well as the correlations between words. Since the hybrid approach combines the advantages of generative and discriminative learning, it can predict semantic annotation precisely for unseen images. Finally, we conduct the experiments on three baseline datasets and the results show that our approach outperforms many state-of-the-art approaches.     

混合生成式和判别式模型的图像自动标注

目的 由于图像检索中存在着低层特征和高层语义之间的“语义鸿沟”,图像自动标注成为当前的关键性问题.为缩减语义鸿沟,提出了一种混合生成式和判别式模型的图像自动标注方法.方法 在生成式学习阶段,采用连续的概率潜在语义分析模型对图像进行建模,可得到相应的模型参数和每幅图像的主题分布.将这个主题分布作为每幅图像的中间表示向量,那么图像自动标注的问题就转化为一个基于多标记学习的分类问题.在判别式学习阶段,使用构造集群分类器链的方法对图像的中间表示向量进行学习,在建立分类器链的同时也集成了标注关键词之间的上下文信息,因而能够取得更高的标注精度和更好的检索效果.结果 在两个基准数据集上进行的实验表明,本文方法在Corel5k数据集上的平均精度、平均召回率分别达到0.28和0.32,在IAPR-TC12数据集上则达到0.29和0.18,其性能优于大多数当前先进的图像自动标注方法.此外,从精度—召回率曲线上看,本文方法也优于几种典型的具有代表性的标注方法.结论 提出了一种基于混合学习策略的图像自动标注方法,集成了生成式模型和判别式模型各自的优点,并在图像语义检索的任务中表现出良好的有效性和鲁棒性.本文方法和技术不仅能应用于图像检索和识别的领域,经过适当的改进之后也能在跨媒体检索和数据挖掘领域发挥重要作用.     

Hybrid Generative-Discriminative Visual Categorization

Learning models for detecting and classifying object categories is a challenging problem in machine vision. While discriminative approaches to learning and classification have, in principle, superior performance, generative approaches provide many useful features, one of which is the ability to naturally establish explicit correspondence between model components and scene features—this, in turn, allows for the handling of missing data and unsupervised learning in clutter. We explore a hybrid generative/discriminative approach, using ‘Fisher Kernels’ (Jaakola, T., et al. in Advances in neural information processing systems, Vol. 11, pp. 487–493, 1999), which retains most of the desirable properties of generative methods, while increasing the classification performance through a discriminative setting. Our experiments, conducted on a number of popular benchmarks, show strong performance improvements over the corresponding generative approach. In addition, we demonstrate how this hybrid learning paradigm can be extended to address several outstanding challenges within computer vision including how to combine multiple object models and learning with unlabeled data.     

A Tale of Two Classifier Systems

This paper describes two classifier systems that learn. These are rule-based systems that use genetic algorithms, which are based on an analogy with natural selection and genetics, as their principal learning mechanism, and an economic model as their principal mechanism for apportioning credit. CFS-C is a domain-independent learning system that has been widely tested on serial computers. CFS is a parallel implementation of CFS-C that makes full use of the inherent parallelism of classifier systems and genetic algorithms, and that allows the exploration of large-scale tasks that were formerly impractical. As with other approaches to learning, classifier systems in their current form work well for moderately-sized tasks but break down for larger tasks. In order to shed light on this issue, we present several empirical studies of known issues in classifier systems, including the effects of population size, the actual contribution of genetic algorithms, the use of rule chaining in solving higher-order tasks, and issues of task representation and dynamic population convergence. We conclude with a discussion of some major unresolved issues in learning classifier systems and some possible approaches to making them more effective on complex tasks.     

Object classification by fusing SVMs and Gaussian mixtures

We present a new technique that employs support vector machines (SVMs) and Gaussian mixture densities (GMDs) to create a generative/discriminative object classification technique using local image features. In the past, several approaches to fuse the advantages of generative and discriminative approaches were presented, often leading to improved robustness and recognition accuracy. Support vector machines are a well known discriminative classification framework but, similar to other discriminative approaches, suffer from a lack of robustness with respect to noise and overfitting. Gaussian mixtures, on the contrary, are a widely used generative technique. We present a method to directly fuse both approaches, effectively allowing to fully exploit the advantages of both. The fusion of SVMs and GMDs is done by representing SVMs in the framework of GMDs without changing the training and without changing the decision boundary. The new classifier is evaluated on the PASCAL VOC 2006 data. Additionally, we perform experiments on the USPS dataset and on four tasks from the UCI machine learning repository to obtain additional insights into the properties of the proposed approach. It is shown that for the relatively rare cases where SVMs have problems, the combined method outperforms both individual ones.     

Aerial scene classification via an ensemble extreme learning machine classifier based on discriminative hybrid convolutional neural networks features

Identifying a discriminative feature can effectively improve the classification performance of aerial scene classification. Deep convolutional neural networks (DCNN) have been widely used in aerial scene classification for its learning discriminative feature ability. The DCNN feature can be more discriminative by optimizing the training loss function and using transfer learning methods. To enhance the discriminative power of a DCNN feature, the improved loss functions of pretraining models are combined with a softmax loss function and a centre loss function. To further improve performance, in this article, we propose hybrid DCNN features for aerial scene classification. First, we use DCNN models with joint loss functions and transfer learning from pretrained deep DCNN models. Second, the dense DCNN features are extracted, and the discriminative hybrid features are created using linear connection. Finally, an ensemble extreme learning machine (EELM) classifier is adopted for classification due to its general superiority and low computational cost. Experimental results based on the three public benchmark data sets demonstrate that the hybrid features obtained using the proposed approach and classified by the EELM classifier can result in remarkable performance.