Latent Classification Models

One of the simplest, and yet most consistently well-performing set of classifiers is the Naïve Bayes models. These models rely on two assumptions: (i) All the attributes used to describe an instance are conditionally independent given the class of that instance, and (ii) all attributes follow a specific parametric family of distributions. In this paper we propose a new set of models for classification in continuous domains, termed latent classification models. The latent classification model can roughly be seen as combining the Naïve Bayes model with a mixture of factor analyzers, thereby relaxing the assumptions of the Naïve Bayes classifier. In the proposed model the continuous attributes are described by a mixture of multivariate Gaussians, where the conditional dependencies among the attributes are encoded using latent variables. We present algorithms for learning both the parameters and the structure of a latent classification model, and we demonstrate empirically that the accuracy of the proposed model is significantly higher than the accuracy of other probabilistic classifiers.Editors: Pedro Larrañaga, Jose A. Lozano, Jose M. Peña and Iñaki Inza     

基于模拟退火遗传算法的贝叶斯分类

朴素贝叶斯分类器是一种简单而高效的分类器，但是其属性独立性假设限制了对实际数据的应用。文章提出一种新的算法，该算法为避免数据预处理时的属性约简对分类效果的直接影响，在训练集上通过随机属性选取生成若干属性子集，以这些子集构建相应的朴素贝叶斯分类器，采用模拟退火遗传算法进行优选。实验表明，与传统的朴素贝叶斯方法相比，该方法具有更好的性能。     

Supervised classification using probabilistic decision graphs

A new model for supervised classification based on probabilistic decision graphs is introduced. A probabilistic decision graph (PDG) is a graphical model that efficiently captures certain context specific independencies that are not easily represented by other graphical models traditionally used for classification, such as the Naïve Bayes (NB) or Classification Trees (CT). This means that the PDG model can capture some distributions using fewer parameters than classical models. Two approaches for constructing a PDG for classification are proposed. The first is to directly construct the model from a dataset of labelled data, while the second is to transform a previously obtained Bayesian classifier into a PDG model that can then be refined. These two approaches are compared with a wide range of classical approaches to the supervised classification problem on a number of both real world databases and artificially generated data.     

基于多种特征选择的NB组合文本分类器设计

针对朴素贝叶斯（NB）分类器在分类过程中存在诸如分类模型对样本具有敏感性、分类精度难以提高等缺陷,提出一种基于多种特征选择方法的NB组合文本分类器方法。依据Boosting分类算法,采用多种不同的特征选择方法建立文本的特征词集,训练NB分类器作为Boosting迭代过程的基分类器,通过对基分类器的加权投票生成最终的NB组合文本分类器。实验结果表明,该组合分类器较单NB文本分类器具有更好的分类性能。     

基于余弦相似度和实例加权改进的贝叶斯算法

面对大量样本特征时很多分类器无法取得较好的分类效果,样本数有限导致贝叶斯算法无法获得精确的联合概率分布估计,在样本局部构建高质量分类器需要有效的样本相似性度量指标. 针对以上问题,提出了一种基于余弦相似度进行实例加权改进的朴素贝叶斯分类算法. 算法考虑特征对分类的决策权重不同,使用余弦相似度度量样本的相似性,选出最优训练样本子集,用相似度值作为训练样本的权值来训练修正后的贝叶斯模型进行分类. 基于UCI数据集的对比实验结果表明,提出的改进算法易于实现且具有更高的平均分类准确率.     

基于朴素贝叶斯模型的新闻故事分割方法

提出一种基于朴素贝叶斯模型的新闻视频故事分割方法。通过对新闻视频进行镜头检测,获得候选故事边界点,从候选边界点周围镜头提取多模态中级特征,形成属性集合作为输入,应用朴素贝叶斯模型对候选边界点进行分类后对结果进行后处理,得到新闻故事。实验结果表明,该方法获得了较高的查准率和查全率,对不同类型的新闻节目有良好的适应性。     

A genetic algorithm-based rule extraction system

Individual classifiers predict unknown objects. Although, these are usually domain specific, and lack the property of scaling up prediction while handling data sets with huge size and high-dimensionality or imbalance class distribution. This article introduces an accuracy-based learning system called DTGA (decision tree and genetic algorithm) that aims to improve prediction accuracy over any classification problem irrespective to domain, size, dimensionality and class distribution. More specifically, the proposed system consists of two rule inducing phases. In the first phase, a base classifier, C4.5 (a decision tree based rule inducer) is used to produce rules from training data set, whereas GA (genetic algorithm) in the next phase refines them with the aim to provide more accurate and high-performance rules for prediction. The system has been compared with competent non-GA based systems: neural network, Naïve Bayes, rule-based classifier using rough set theory and C4.5 (i.e., the base classifier of DTGA), on a number of benchmark datasets collected from UCI (University of California at Irvine) machine learning repository. Empirical results demonstrate that the proposed hybrid approach provides marked improvement in a number of cases.     

Incorporating label dependency into the binary relevance framework for multi-label classification

In multi-label classification, examples can be associated with multiple labels simultaneously. The task of learning from multi-label data can be addressed by methods that transform the multi-label classification problem into several single-label classification problems. The binary relevance approach is one of these methods, where the multi-label learning task is decomposed into several independent binary classification problems, one for each label in the set of labels, and the final labels for each example are determined by aggregating the predictions from all binary classifiers. However, this approach fails to consider any dependency among the labels. Aiming to accurately predict label combinations, in this paper we propose a simple approach that enables the binary classifiers to discover existing label dependency by themselves. An experimental study using decision trees, a kernel method as well as Naïve Bayes as base-learning techniques shows the potential of the proposed approach to improve the multi-label classification performance.     

Latent classification models for binary data

One of the simplest, and yet most consistently well-performing set of classifiers is the naïve Bayes models (a special class of Bayesian network models). However, these models rely on the (naïve) assumption that all the attributes used to describe an instance are conditionally independent given the class of that instance. To relax this independence assumption, we have in previous work proposed a family of models, called latent classification models (LCMs). LCMs are defined for continuous domains and generalize the naïve Bayes model by using latent variables to model class-conditional dependencies between the attributes. In addition to providing good classification accuracy, the LCM has several appealing properties, including a relatively small parameter space making it less susceptible to over-fitting. In this paper we take a first step towards generalizing LCMs to hybrid domains, by proposing an LCM for domains with binary attributes. We present algorithms for learning the proposed model, and we describe a variational approximation-based inference procedure. Finally, we empirically compare the accuracy of the proposed model to the accuracy of other classifiers for a number of different domains, including the problem of recognizing symbols in black and white images.     

A hierarchical Naïve Bayes model for approximate identity matching

Organizations often manage identity information for their customers, vendors, and employees. Identity management is critical to various organizational practices ranging from customer relationship management to crime investigation. The task of searching for a specific identity is difficult because disparate identity information may exist due to the issues related to unintentional errors and intentional deception. In this paper we propose a hierarchical Naïve Bayes model that improves existing identity matching techniques in terms of searching effectiveness. Experiments show that our proposed model performs significantly better than the exact-match based matching technique. With 50% training instances labeled, the proposed semi-supervised learning achieves a performance comparable to the fully supervised record comparison algorithm. The semi-supervised learning greatly reduces the efforts of manually labeling training instances without significant performance degradation.     

针对文本分类的神经网络模型

文本分类是自然语言处理领域的一项重要任务,具有广泛的应用场景,比如知识问答、文本主题分类、文本情感分析等.解决文本分类任务的方法有很多,如支持向量机（Support Vector Machines,SVM）模型和朴素贝叶斯（Naïve Bayes）模型,现在被广泛使用的是以循环神经网络（Recurrent Neural Network,RNN）和文本卷积网络（TextConventional Neural Network,TextCNN）为代表的神经网络模型.本文分析了文本分类领域中的序列模型和卷积模型,并提出一种组合序列模型和卷积模型的混合模型.在公开数据集上对不同模型进行性能上的对比,验证了组合模型的性能要优于单独的模型.     

CPU load prediction using neuro-fuzzy and Bayesian inferences

Ensuring adequate use of the computing resources for highly fluctuating availability in multi-user computational environments requires effective prediction models, which play a key role in achieving application performance for large-scale distributed applications. Predicting the processor availability for scheduling a new process or task in a distributed environment is a basic problem that arises in many important contexts. The present paper aims at developing a model for single-step-ahead CPU load prediction that can be used to predict the future CPU load in a dynamic environment. Our prediction model is based on the control of multiple Local Adaptive Network-based Fuzzy Inference Systems Predictors (LAPs) via the Naïve Bayesian Network inference between clusters states of CPU load time points obtained by the C-means clustering process. Experimental results show that our model performs better and has less overhead than other approaches reported in the literature.     

Enhanced default risk models with SVM+

Default risk models have lately raised a great interest due to the recent world economic crisis. In spite of many advanced techniques that have extensively been proposed, no comprehensive method incorporating a holistic perspective has hitherto been considered. Thus, the existing models for bankruptcy prediction lack the whole coverage of contextual knowledge which may prevent the decision makers such as investors and financial analysts to take the right decisions. Recently, SVM+ provides a formal way to incorporate additional information (not only training data) onto the learning models improving generalization. In financial settings examples of such non-financial (though relevant) information are marketing reports, competitors landscape, economic environment, customers screening, industry trends, etc. By exploiting additional information able to improve classical inductive learning we propose a prediction model where data is naturally separated into several structured groups clustered by the size and annual turnover of the firms. Experimental results in the setting of a heterogeneous data set of French companies demonstrated that the proposed default risk model showed better predictability performance than the baseline SVM and multi-task learning with SVM.     

基于置信优势关系的粗糙集近似模型

不完备有序信息处理是现实生活中的常见问题. 多种拓展优势关系及其粗糙集模型被提出并用于解决不完全的偏好决策问题, 但均未考虑序关系特性, 与现实语义存在矛盾. 对此, 提出一种置信优势关系及其粗糙集近似模型, 讨论了基于置信优势关系的粗糙集模型与已有模型的关系. 与现有的拓展关系相比, 该置信优势关系满足序关系特性, 避免了语义上的矛盾. 定理证明和实例分析表明, 置信优势关系粗糙集近似模型的近似精度和分类精度更优.

     

Building contextual classifiers by integrating fuzzy rule based classification technique and k-nn method for credit scoring

Credit-risk evaluation is a very challenging and important problem in the domain of financial analysis. Many classification methods have been proposed in the literature to tackle this problem. Statistical and neural network based approaches are among the most popular paradigms. However, most of these methods produce so-called “hard” classifiers, those generate decisions without any accompanying confidence measure. In contrast, “soft” classifiers, such as those designed using fuzzy set theoretic approach; produce a measure of support for the decision (and also alternative decisions) that provides the analyst with greater insight. In this paper, we propose a method of building credit-scoring models using fuzzy rule based classifiers. First, the rule base is learned from the training data using a SOM based method. Then the fuzzy k-nn rule is incorporated with it to design a contextual classifier that integrates the context information from the training set for more robust and qualitatively better classification. Further, a method of seamlessly integrating business constraints into the model is also demonstrated.     

Multi-class classification using hybrid soft decision model for agriculture crop selection

A hybrid soft decision model has been developed in this paper to take decision on agriculture crop that can be cultivated in a given experimental land by integrating few soft computing techniques. The proposed model comprises of three parts, namely weight calculation, classification and prediction. Twenty-seven input criteria were categorized into seven broad criteria, namely soil (11 sub-criteria), water (2 sub-criteria), season (no sub-criterion), input (6 sub-criteria), support (2 sub-criteria), facilities (3 sub-criteria) and risk (2 sub-criteria). In the proposed model, relative weights of main criteria were calculated using Shannon’s Entropy method and relative weights of sub-criteria in each main criterion were calculated using rough set approach. As VIKOR method is effective in sorting the alternatives, it is used to determine the ranking index of main criteria in this study. A soft decision system was constructed from the results of rough set method, VIKOR method and Shannon’s Entropy method. Classification rules were generated for five agriculture crops, namely paddy, groundnut, sugarcane, cumbu and ragi based on the soft decision system using bijective soft set approach. The developed model predicts each site in the validation dataset into one of the five crops. The performance of the proposed model has been sanity checked by agriculture experts.

     

Predicting Corporate Acquisitions: An Application of Uncertain Reasoning Using Rule Induction

Artificial Intelligence (AI)-based rule induction techniques such as IXL and ID3 are powerful tools that can be used to classify firms as acquisition candidates or not, based on financial and other data. The purpose of this paper is to develop an expert system that employs uncertainty representation and predicts acquisition targets. We outline in this paper, the features of IXL, a machine learning technique that we use to induce rules. We also discuss how uncertainty is handled by IXL and describe the use of confidence factors. Rules generated by IXL are incorporated into a prototype expert system, ACQTARGET, which evaluates corporate acquisitions. The use of confidence factors in ACQTARGET allows investors to specifically incorporate uncertainties into the decision making process. A set of training examples comprising 65 acquired and 65 non-acquired real world firms is used to generate the rules and a separate holdout sample containing 32 acquired and 32 non-acquired real world firms is used to validate the expert system results. The performance of the expert system is also compared with a conventional discriminant analysis model and a logit model using the same data. The results show that the expert system, ACQTARGET, performs as well as the statistical models and is a useful evaluation tool to classify firms into acquisition and non-acquisition target categories. This rule induction technique can be a valuable decision aid to help financial analysts and investors in their buy/sell decisions.     

Stochastic dominance-based rough set model for ordinal classification

In order to discover interesting patterns and dependencies in data, an approach based on rough set theory can be used. In particular, dominance-based rough set approach (DRSA) has been introduced to deal with the problem of ordinal classification with monotonicity constraints (also referred to as multicriteria classification in decision analysis). However, in real-life problems, in the presence of noise, the notions of rough approximations were found to be excessively restrictive. In this paper, we introduce a probabilistic model for ordinal classification problems with monotonicity constraints. Then, we generalize the notion of lower approximations to the stochastic case. We estimate the probabilities with the maximum likelihood method which leads to the isotonic regression problem for a two-class (binary) case. The approach is easily generalized to a multi-class case. Finally, we show the equivalence of the variable consistency rough sets to the specific empirical risk-minimizing decision rule in the statistical decision theory.     

Pruning least objective contribution in KMSE

Although kernel minimum squared error (KMSE) is computationally simple, i.e., it only needs solving a linear equation set, it suffers from the drawback that in the testing phase the computational efficiency decreases seriously as the training samples increase. The underlying reason is that the solution of Naïve KMSE is represented by all the training samples in the feature space. Hence, in this paper, a method of selecting significant nodes for KMSE is proposed. During each calculation round, the presented algorithm prunes the training sample making least contribution to the objective function, hence called as PLOC-KMSE. To accelerate the training procedure, a batch of so-called nonsignificant nodes is pruned instead of one by one in PLOC-KMSE, and this speedup algorithm is named MPLOC-KMSE for short. To show the efficacy and feasibility of the proposed PLOC-KMSE and MPLOC-KMSE, the experiments on benchmark data sets and real-world instances are reported. The experimental results demonstrate that PLOC-KMSE and MPLOC-KMSE require the fewest significant nodes compared with other algorithms. That is to say, their computational efficiency in the testing phase is best, thus suitable for environments having a strict demand of computational efficiency. In addition, from the performed experiments, it is easily known that the proposed MPLOC-KMSE accelerates the training procedure without sacrificing the computational efficiency of testing phase to reach the almost same generalization performance. Finally, although PLOC and MPLOC are proposed in regression domain, they can be easily extended to classification problem and other algorithms such as kernel ridge regression.     

Multiknowledge for decision making

The representation of knowledge has an important effect on automated decision-making. In this paper, vector spaces are used to describe a condition space and a decision space, and knowledge is represented by a mapping from the condition space to the decision space. Many such mappings can be obtained from a training set. A set of mappings, which are created from multiple reducts in the training set, is defined as multiknowledge. In order to get a good reduct and find multiple reducts, the WADF (worst-attribute-drop-first) algorithm is developed through analysis of the properties of decision systems using rough set theory. An approach that combines multiknowledge and the naïve Bayes classifier is applied to make decisions for unseen instances or for instances with missing attribute values. Benchmark data sets from the UCI Machine Learning Repository are used to test the algorithms. The experimental results are encouraging; the prediction accuracy for unseen instances by using the algorithms is higher than by using other approaches based on a single body of knowledge.