Binary naive possibilistic classifiers: Handling uncertain inputs

Possibilistic networks are graphical models particularly suitable for representing and reasoning with uncertain and incomplete information. According to the underlying interpretation of possibilistic scales, possibilistic networks are either quantitative (using product‐based conditioning) or qualitative (using min‐based conditioning). Among the multiple tasks, possibilitic models can be used for, classification is a very important one. In this paper, we address the problem of handling uncertain inputs in binary possibilistic‐based classification. More precisely, we propose an efficient algorithm for revising possibility distributions encoded by a naive possibilistic network. This algorithm is suitable for binary classification with uncertain inputs since it allows classification in polynomial time using several efficient transformations of initial naive possibilistic networks. © 2009 Wiley Periodicals, Inc.     

Interventions and belief change in possibilistic graphical models

Causality and belief change play an important role in many applications. This paper focuses on the main issues of causality and interventions in possibilistic graphical models. We show that interventions, which are very useful for representing causal relations between events, can be naturally viewed as a belief change process. In particular, interventions can be handled using a possibilistic counterpart of Jeffrey's rule of conditioning under uncertain inputs. This paper also addresses new issues that are arisen in the revision of graphical models when handling interventions. We first argue that the order in which observations and interventions are introduced is very important. Then we show that in order to correctly handle sequences of observations and interventions, one needs to change the structure of possibilistic networks. Lastly, an efficient procedure for revising possibilistic causal trees is provided.     

An Approximate Algorithm for Min‐Based Possibilistic Networks

Min‐based (or qualitative) possibilistic networks are important tools to efficiently and compactly represent and analyze uncertain information. Inference is a crucial task in min‐based networks, which consists of propagating information through the network structure to answer queries. Exact inference computes posteriori possibility distributions, given some observed evidence, in a time proportional to the number of nodes of the network when it is simply connected (without loops). On multiply connected networks (with loops), exact inference is known as a hard problem. This paper proposes an approximate algorithm for inference in min‐based possibilistic networks. More precisely, we adapt the well‐known approximate algorithm Loopy Belief Propagation (LBP) on qualitative possibilistic networks. We provide different experimental results that analyze the convergence of possibilistic LBP.     

Modeling uncertainty reasoning with possibilistic Petri nets

Manipulation of perceptions is a remarkable human capability in a wide variety of physical and mental tasks under fuzzy or uncertain surroundings. Possibilistic reasoning can be treated as a mechanism that mimics human inference mechanisms with uncertain information. Petri nets are a graphical and mathematical modeling tool with powerful modeling and analytical ability. The focus of this paper is on the integration of Petri nets with possibilistic reasoning to reap the benefits of both formalisms. This integration leads to a possibilistic Petri nets model (PPN) with the following features. A possibilistic token carries information to describe an object and its corresponding possibility and necessity measures. Possibilistic transitions are classified into four types: inference transitions, duplication transitions, aggregation transitions, and aggregation-duplication transitions. A reasoning algorithm, based on possibilistic Petri nets, is also presented to improve the efficiency of possibilistic reasoning and an example related to diagnosis of cracks in reinforced concrete structures is used to illustrate the proposed approach.     

Fusion and normalization of quantitative possibilistic networks

The problem of merging multiple-source uncertain information is a crucial issue in many applications. This paper proposes an analysis of possibilistic merging operators where uncertain information is encoded by means of product-based (or quantitative) possibilistic networks. We first show that the product-based merging of possibilistic networks having the same DAG structures can be easily achieved in a polynomial time. We then propose solutions to merge possibilistic networks having different structures and where the union of their graphs is free of cycles. Then we show how to deal with merged networks having cycles. Lastly, we handle the sub-normalization problem which reflects the presence of conflicts between different sources.     

Jeffrey’s rule of conditioning in a possibilistic framework

Conditioning, belief update and revision are important tasks for designing intelligent systems. Possibility theory is among the powerful uncertainty theories particularly suitable for representing and reasoning with uncertain and incomplete information. This paper addresses an important issue related to the possibilistic counterparts of Jeffrey’s rule of conditioning. More precisely, it addresses the existence and uniqueness of the solutions computed using the possibilistic counterparts of the so-called kinematics properties underlying Jeffrey’s rule of conditioning. We first point out that like the probabilistic framework, in the quantitative possibilistic setting, there exists a unique solution for revising a possibility distribution given the uncertainty bearing on a set of exhaustive and mutually exclusive events. However, in the qualitative possibilistic framework, the situation is different. In particular, the application of Jeffrey’s rule of conditioning does not guarantee the existence of a solution. We provide precise conditions where the uniqueness of the revised possibility distribution exists.     

具有多值决策过程的广义可能性计算树逻辑模型检测

模型检测是一种自动验证软硬件系统行为的有效技术。为了对包含非确定性信息、不一致信息的并发系统进行形式化验证,在可能性理论、多值逻辑的基础上,研究了具有多值决策过程的广义可能性多值计算树逻辑模型检测算法,及其在检验非确定性系统中的具体应用。首先构造了多值决策过程作为系统模型,用多值计算树逻辑描述系统属性。然后给出具有多值决策过程的广义可能性多值计算树逻辑的模型检测算法,该算法将模型检测的具体问题转换为多项式时间内的模糊矩阵运算。最后就包含非确定性选择的多值系统的模型检测问题,给出一个具体的应用实例。     

Knowledge integration in a multiple classifier system

This paper introduces a knowledge integration framework based on Dempster-Shafer's mathematical theory of evidence for integrating classification results derived from multiple classifiers. This framework enables us to understand in which situations the classifiers give uncertain responses, to interpret classification evidence, and allows the classifiers to compensate for their individual deficiencies. Under this framework, we developed algorithms to model classification evidence and combine classification evidence form difference classifiers, we derived inference rules from evidential intervals for reasoning about classification results. The algorithms have been implemented and tested. Implementation issues, performance analysis and experimental results are presented.     

Possibilistic uncertainty handling for answer set programming

In this work, we introduce a new framework able to deal with a reasoning that is at the same time non monotonic and uncertain. In order to take into account a certainty level associated to each piece of knowledge, we use possibility theory to extend the non monotonic semantics of stable models for logic programs with default negation. By means of a possibility distribution we define a clear semantics of such programs by introducing what is a possibilistic stable model. We also propose a syntactic process based on a fix-point operator to compute these particular models representing the deductions of the program and their certainty. Then, we show how this introduction of a certainty level on each rule of a program can be used in order to restore its consistency in case of the program has no model at all. Furthermore, we explain how we can compute possibilistic stable models by using available softwares for Answer Set Programming and we describe the main lines of the system that we have developed to achieve this goal.     

Interval regression analysis by quadratic programming approach

When we use linear programming in possibilistic regression analysis, some coefficients tend to become crisp because of the characteristic of linear programming. On the other hand, a quadratic programming approach gives more diverse spread coefficients than a linear programming one. Therefore, to overcome the crisp characteristic of linear programming, we propose interval regression analysis based on a quadratic programming approach. Another advantage of adopting a quadratic programming approach is to be able to integrate both the property of central tendency in least squares and the possibilistic property in fuzzy regression. By changing the weights of the quadratic function, we can analyze the given data from different viewpoints. For data with crisp inputs and interval outputs, the possibility and necessity models can be considered. Therefore, the unified quadratic programming approach obtaining the possibility and necessity regression models simultaneously is proposed. Even though there always exist possibility estimation models, the existence of necessity estimation models is not guaranteed if we fail to assume a proper function fitting to the given data as a regression model. Thus, we consider polynomials as regression models since any curve can be represented by the polynomial approximation. Using polynomials, we discuss how to obtain approximation models which fit well to the given data where the measure of fitness is newly defined to gauge the similarity between the possibility and the necessity models. Furthermore, from the obtained possibility and necessity regression models, a trapezoidal fuzzy output can be constructed     

一种基于多进化神经网络的分类方法

分类问题是目前数据挖掘和机器学习领域的重要内容.提出了一种基于多进化神经网络的分类方法CABEN(classification approach based on evolutionary neural networks).利用改进的进化策略和Levenberg-Marquardt方法对多个三层前馈神经网络同时进行训练.训练好各个分类模型以后,将待识别数据分别输入,最后根据绝对多数投票法决定最终分类结果.实验结果表明,该方法可以较好地进行数据分类,而且与传统的神经网络方法以及贝叶斯方法和决策树方法相比,在     

A relational approach to probabilistic classification in a transductive setting

Transduction is an inference mechanism adopted from several classification algorithms capable of exploiting both labeled and unlabeled data and making the prediction for the given set of unlabeled data only. Several transductive learning methods have been proposed in the literature to learn transductive classifiers from examples represented as rows of a classical double-entry table (or relational table). In this work we consider the case of examples represented as a set of multiple tables of a relational database and we propose a new relational classification algorithm, named TRANSC, that works in a transductive setting and employs a probabilistic approach to classification. Knowledge on the data model, i.e., foreign keys, is used to guide the search process. The transductive learning strategy iterates on a k-NN based re-classification of labeled and unlabeled examples, in order to identify borderline examples, and uses the relational probabilistic classifier Mr-SBC to bootstrap the transductive algorithm. Experimental results confirm that TRANSC outperforms its inductive counterpart (Mr-SBC).     

隐含概念漂移的不确定数据流集成分类算法

近年来,数据流分类问题已经逐渐成为数据挖掘领域的一个研究热点,然而传统的数据流分类算法大多只能处理数据项已知并且为精确值的数据流,无法有效地应用于现实应用中普遍存在的不确定数据流。为建立适应数据不确定性的分类模型,提高不确定数据流分类准确率,提出一种针对不确定数据流的集成分类算法,该算法将不确定数据用区间及其概率分布函数表示,用C4.5决策树分类方法和朴素贝叶斯分类方法训练基分类器,在合理处理数据流中不确定性的同时,还能有效解决数据流中隐含的概念漂移问题。实验结果表明,所提算法在处理不确定数据流的分类时具有较好的鲁棒性,并且具有较高的分类准确率。     

Approximate maximum entropy joint feature inference consistent with arbitrary lower-order probability constraints: application to statistical classification

We propose a new learning method for discrete space statistical classifiers. Similar to Chow and Liu (1968) and Cheeseman (1983), we cast classification/inference within the more general framework of estimating the joint probability mass function (p.m.f.) for the (feature vector, class label) pair. Cheeseman's proposal to build the maximum entropy (ME) joint p.m.f. consistent with general lower-order probability constraints is in principle powerful, allowing general dependencies between features. However, enormous learning complexity has severely limited the use of this approach. Alternative models such as Bayesian networks (BNs) require explicit determination of conditional independencies. These may be difficult to assess given limited data. Here we propose an approximate ME method, which, like previous methods, incorporates general constraints while retaining quite tractable learning. The new method restricts joint p.m.f. support during learning to a small subset of the full feature space. Classification gains are realized over dependence trees, tree-augmented naive Bayes networks, BNs trained by the Kutato algorithm, and multilayer perceptrons. Extensions to more general inference problems are indicated. We also propose a novel exact inference method when there are several missing features.     

Genetic tuning of fuzzy inference within fuzzy classifier systems

Abstract: In generating a suitable fuzzy classifier system, significant effort is often placed on the determination and the fine tuning of the fuzzy sets. However, in such systems little thought is given to the way in which membership functions are combined within the fuzzy rules. Often traditional fuzzy inference strategies are used which consequently provide no control over how strongly or weakly the inference is applied within these rules. Furthermore such strategies will allow no interaction between grades of membership. A number of theoretical fuzzy inference operators have been proposed for both regression and classification problems but they have not been investigated in the context of real-world applications. In this paper we propose a novel genetic algorithm framework for optimizing the strength of fuzzy inference operators concurrently with the tuning of membership functions for a given fuzzy classifier system. Each fuzzy system is generated using two well-established decision tree algorithms: C4.5 and CHAID. This will enable both classification and regression problems to be addressed within the framework. Each solution generated by the genetic algorithm will produce a set of fuzzy membership functions and also determine how strongly the inference will be applied within each fuzzy rule. We investigate several theoretical proven fuzzy inference techniques (T-norms) in the context of both classification and regression problems. The methodology proposed is applied to a number of real-world data sets in order to determine the effects of the simultaneous tuning of membership functions and inference parameters on the accuracy and robustness of fuzzy classifiers.     

The entailment principle for dempster—shafer granules

We discuss the rule of inference called the entailment principle which plays a significant role in the possibilistic type reasoning used in the theory of approximate reasoning. We extend this principle to situations in which the knowledge is a type of combination of possibilistic and probabilistic information which we call Dempster—Shafer granules. We discuss the conjunction of these D—S granules and show that Dempster's rule of combination is a special application of conjunction followed by a particular implementation of the entailment principle.     

An intelligent web-page classifier with fair feature-subset selection

The explosion of on-line information has given rise to many manually constructed topic hierarchies (such as Yahoo!!). But with the current growth rate in the amount of information, manual classification in topic hierarchies results in an immense information bottleneck. Therefore, developing an automatic classifier is an urgent need. However, classifiers suffer from enormous dimensionality, since the dimensionality is determined by the number of distinct keywords in a document corpus. More seriously, most classifiers are either working slowly or they are constructed subjectively without any learning ability. In this paper, we address these problems with a fair feature-subset selection (FFSS) algorithm and an adaptive fuzzy learning network (AFLN) for classification. The FFSS algorithm is used to reduce the enormous dimensionality. It not only gives fair treatment to each category but also has ability to identify useful features, including both positive and negative features. On the other hand, the AFLN provides extremely fast learning ability to model the uncertain behavior for classification so as to correct the fuzzy matrix automatically. Experimental results show that both FFSS algorithm and the AFLN lead to a significant improvement in document classification, compared to alternative approaches.     

多分类器系统的泛组合规则研究与应用

现有的多分类器系统采用固定的组合算子,适用性较差。将泛逻辑的柔性化思想引入多分类器系统中,应用泛组合运算模型建立了泛组合规则。泛组合规则采用遗传算法进行参数估计,对并行结构的多分类器系统具有良好的适用性。在时间序列数据集上的分类实验结果表明,泛组合规则的分类性能优于乘积规则、均值规则、中值规则、最大规则、最小规则、投票规则等固定组合规则。