基于加权信息熵的直觉模糊信息系统的三支决策

讨论直觉模糊信息系统上的三支决策问题.首先,定义一个由模糊因子、均值因子和概率因子3部分组成的相似度函数,从而建立直觉模糊信息系统上的三支决策模型,并指出该模型从理论上统一了各种双论域模型;其次,考虑论域对象的评价值不同,提出一种基于评价值的划分测度:加权信息熵,并且证明划分越细,加权信息熵越大;最后,基于加权信息熵的性质,给出最优三划分的合理解释,从而提出一种新的阈值求解方法.     

Decision trees using model ensemble-based nodes

Decision trees recursively partition the instance space by generating nodes that implement a decision function belonging to an a priori specified model class. Each decision may be univariate, linear or nonlinear. Alternatively, in omnivariate decision trees, one of the model types is dynamically selected by taking into account the complexity of the problem defined by the samples reaching that node. The selection is based on statistical tests where the most appropriate model type is selected as the one providing significantly better accuracy than others. In this study, we propose the use of model ensemble-based nodes where a multitude of models are considered for making decisions at each node. The ensemble members are generated by perturbing the model parameters and input attributes. Experiments conducted on several datasets and three model types indicate that the proposed approach achieves better classification accuracies compared to individual nodes, even in cases when only one model class is used in generating ensemble members.     

基于最大熵选取示例的增量决策树归纳

设A是一训练集,B是A的一个子集,B是选择A中部分有代表性的示例而生成的。得到了这样一个结论,即对于适当选取的B,由B训练出的决策树其泛化精度优于由A训练出的决策树的泛化精度。进一步,设计实现了一种如何从A中挑选有代表性的示例来生成B的算法,并从数据分布和信息熵理论角度分析了该算法的设计原理。     

Using genetic algorithms to determine nonnegative monotone set functions for information fusion in environments with random perturbation

Due to some inherent interactions among diverse information sources, the classical weighted average method is not adequate for information fusion in many real problems. To describe the interactions, an intuitive and effective way is to use an appropriate nonadditive set function. Instead of the weighted average method, which is essentially the Lebesgue integral, we should thus use the Choquet integral or some other nonlinear integrals. To apply this alternative, more realistic approach to information fusion, we need to determine the nonadditive set function from given input-output data, viewing the nonlinear integral as a multi-input one-output system. In this paper, we employ an adaptive genetic algorithm to construct an approximate optimal nonnegative monotone set function from given input-output data in an environment with random perturbation. An example for diverse strengths of random perturbation is shown to demonstrate the efficiency of this algorithm. ©1999 John Wiley & Sons, Inc.     

基于信息熵的地学空间数据挖掘模型

从信息熵的基本概念出发,认为地学空间数据子集划分产生的互信息或熵减源于子集划分,使得各个子集的不确定性或模型糊降低,并且子集之间的差异性增在 最大熵减的子集划分方案代表一定的地学模式和地不规律。以此为基础分别探讨了地学数据属性要素的子集划分产生多维属性关联规则,以 间和时间的子集分割来进行了聚类的方法。     

基于离散度的决策树构造方法

在构造决策树的过程中,属性选择将影响到决策树的分类精度.对此,讨论了基于信息熵方法和WMR方法的局限性,提出了信息系统中条件属性集的离散度的概念.利用该概念在决策树构造过程中选择划分属性,设计了基于离散度的决策树构造算法DSD.DSD算法可以解决WMR方法在实际应用中的局限性.在UCI数据集上的实验表明,该方法构造的决策树精度与基于信息熵的方法相近,而时间复杂度则优于基于信息熵的方法.     

基于信息熵的核属性增量式高效更新算法

针对基于信息熵求核算法效率不理想的情况,给出信息观下的二进制差别矩阵定义,理论上证明基于信息熵的核属性与基于二进制差别矩阵的核属性等价;并将决策表划分为相容的对象集和不相容的对象集,缩小求核算法的搜索空间;然后针对动态的决策表,研究核属性的增量更新机制,由此构造一种基于信息熵的核属性增量式高效更新算法。实例分析与实验结果验证文中算法优于同类求解算法。     

Decision forest: Twenty years of research

A decision tree is a predictive model that recursively partitions the covariate’s space into subspaces such that each subspace constitutes a basis for a different prediction function. Decision trees can be used for various learning tasks including classification, regression and survival analysis. Due to their unique benefits, decision trees have become one of the most powerful and popular approaches in data science. Decision forest aims to improve the predictive performance of a single decision tree by training multiple trees and combining their predictions. This paper provides an introduction to the subject by explaining how a decision forest can be created and when it is most valuable. In addition, we are reviewing some popular methods for generating the forest, fusion the individual trees’ outputs and thinning large decision forests.     

Fuzzy min-max neural networks. I. Classification.

A supervised learning neural network classifier that utilizes fuzzy sets as pattern classes is described. Each fuzzy set is an aggregate (union) of fuzzy set hyperboxes. A fuzzy set hyperbox is an n-dimensional box defined by a min point and a max point with a corresponding membership function. The min-max points are determined using the fuzzy min-max learning algorithm, an expansion-contraction process that can learn nonlinear class boundaries in a single pass through the data and provides the ability to incorporate new and refine existing classes without retraining. The use of a fuzzy set approach to pattern classification inherently provides a degree of membership information that is extremely useful in higher-level decision making. The relationship between fuzzy sets and pattern classification is described. The fuzzy min-max classifier neural network implementation is explained, the learning and recall algorithms are outlined, and several examples of operation demonstrate the strong qualities of this new neural network classifier.     

Approaches to knowledge reduction of covering decision systems based on information theory

In this paper, we propose some new approaches for attribute reduction in covering decision systems from the viewpoint of information theory. Firstly, we introduce information entropy and conditional entropy of the covering and define attribute reduction by means of conditional entropy in consistent covering decision systems. Secondly, in inconsistent covering decision systems, the limitary conditional entropy of the covering is proposed and attribute reductions are defined. And finally, by the significance of the covering, some algorithms are designed to compute all the reducts of consistent and inconsistent covering decision systems. We prove that their computational complexity are polynomial. Numerical tests show that the proposed attribute reductions accomplish better classification performance than those of traditional rough sets. In addition, in traditional rough set theory, MIBARK-algorithm [G.Y. Wang, H. Hu, D. Yang, Decision table reduction based on conditional information entropy, Chinese J. Comput., 25 (2002) 1-8] cannot ensure the reduct is the minimal attribute subset which keeps the decision rule invariant in inconsistent decision systems. Here, we solve this problem in inconsistent covering decision systems.     

基于改进关系积的最小属性约简算法

属性约简是粗糙集理论的核心内容之一。针对现有关系积理论不能求解不一致决策表中最小属性约简的缺陷,提出一种基于知识联合划分的改进关系积和关系积约简概念,分析了关系积约简、正区域和负区域之间的性质,提出一种基于改进关系积的最小属性约筒算法。理论分析和实例计算结果表明,该算法具有可行性,能求取一致或不一致决策表中所有最小属性约简。     

DP-PMK: an improved pyramid matching kernel for approximating correspondences in high dimensions

As the feature dimension increases, the original pyramid matching kernel suffers from distortion factors that increase linearly with the feature dimension. This paper proposes a new method by consistently dividing the feature space into two subspaces while generating several levels. In each subspace of the level, the original pyramid matching is used. Then, a weighted sum of every subspace at each level is made as the final measurement of similarity. Experiments on data set Caltech-101 and ETH-80 show that compared with other related algorithms which need hundreds of times of original computation time, dimension partition pyramid matching kernel only needs about 4–6 times less of original computation time to obtain the similar accuracy.     

一种协调目标信息系统中决策规则的融合方法

决策规则获取是目标信息系统中的一个重要研究内容。引入了一种集合向量空间上的加权包含度,并基于该包含度提出了一种协调目标信息系统中决策规则的融合方法,可以得到全部决策规则,实例表明了该方法的有效性。     

不完备数据中面向特征值更新的增量特征选择方法

实际应用中,数据常常表现出不完备性和动态性的特点。针对动态不完备数据中的特征选择问题,提出了一种基于相容粗糙集模型和信息熵理论的增量式特征选择方法。首先,建立了不完备信息系统中特征值动态更新时论域上条件划分与决策分类的动态更新模式,分析了作为特征重要度评价准则的不完备相容信息熵的增量计算机制,并将该机制引入到启发式最优特征子集搜索过程中特征重要度的迭代计算,进一步设计了不完备数据中面向特征值动态更新的增量式特征选择算法。最后,在标准UCI数据集上从分类精度、决策性能和计算效率3个方面对文中所提出的增量算法的有效性和高效性进行了实验验证。     

基于GA的多分类器融合算法

为了提高单一分类器的识别性能,在模式识别领域经常采用多分类器集成的方法。提出了一种基于GA的多分类器融合算法,首先通过GA算法对特征集的分割进行优化选择,形成了较优的成员分类器;然后通过对成员分类器分辨能力的度量,提出了一种加权系数矩阵的多分类器组合方法。在UCI数据库上进行了实验,结果表明所提出的算法具有较高的识别率。     

Marginal contribution representation of capacity-based multicriteria decision making

Integrals defined with respect to fuzzy measures (capacities) are powerful tools in multicriteria decision making. Monotonicity is a basic property of capacity, which means that the marginal contribution of any single criterion to any subset of criteria is always nonnegative. In this paper, we present the capacity-based decision making theory in terms of marginal contributions, which provides an alternative perspective to this widely used decision making strategy. We construct the marginal contribution representations of the equivalent transformations of capacities, some particular capacities, three types of nonlinear integrals, and discuss the capacity identification methods. We also introduce some new concepts and representations, such as nonadditivity and nonmodularity indices, 0 to 1 variables-based linear constraints of k-maxitive capacity, a special representation of the Choquet integral and pan integral. We discuss constraints on marginal contributions which ensure supermodularity of capacities. Finally, an illustrative example is given to show the use of marginal contribution presentation in capacity-based decision making methods.     

FCM聚类算法中模糊加权指数m的优化

研究模糊加权指数m对FCM(Fuzzy c-means)算法的聚类性能的影响,从划分熵入手提出了变权划分熵的概念,并基于模糊决策理论提出了一种最优加权指数m*的选取方法.该方法利用小的目标函数值和小的变权划分熵对应好的数据分类结果这一特性,将m的确定转化为一个带约束的非线性规划问题,从而确定最佳取值m*.实验结果表明该方法是非常有效和灵敏的.     

Induction of multiple fuzzy decision trees based on rough set technique

The integration of fuzzy sets and rough sets can lead to a hybrid soft-computing technique which has been applied successfully to many fields such as machine learning, pattern recognition and image processing. The key to this soft-computing technique is how to set up and make use of the fuzzy attribute reduct in fuzzy rough set theory. Given a fuzzy information system, we may find many fuzzy attribute reducts and each of them can have different contributions to decision-making. If only one of the fuzzy attribute reducts, which may be the most important one, is selected to induce decision rules, some useful information hidden in the other reducts for the decision-making will be losing unavoidably. To sufficiently make use of the information provided by every individual fuzzy attribute reduct in a fuzzy information system, this paper presents a novel induction of multiple fuzzy decision trees based on rough set technique. The induction consists of three stages. First several fuzzy attribute reducts are found by a similarity based approach, and then a fuzzy decision tree for each fuzzy attribute reduct is generated according to the fuzzy ID3 algorithm. The fuzzy integral is finally considered as a fusion tool to integrate the generated decision trees, which combines together all outputs of the multiple fuzzy decision trees and forms the final decision result. An illustration is given to show the proposed fusion scheme. A numerical experiment on real data indicates that the proposed multiple tree induction is superior to the single tree induction based on the individual reduct or on the entire feature set for learning problems with many attributes.     

基于相对决策嫡的决策树算法及其在入侵检测中的应用

为了弥补传统决策树算法的不足,提出一种基于相对决策熵的决策树算法DTRDE。首先,将Shannon提出的信息熵引入到粗糙集理论中,定义一个相对决策熵的概念,并利用相对决策熵来度量属性的重要性;其次,在算法DTRDE中,采用基于相对决策熵的属性重要性以及粗糙集中的属性依赖性来选择分离属性,并且利用粗糙集中的属性约简技术来删除冗余的属性,旨在降低算法的计算复杂性;最后,将该算法应用于网络入侵检测。在KDD Cup99数据集上的实验表明,DTRDE算法比传统的基于信息熵的算法具有更高的检测率,而其计算开销则与传统方法接近。     

一种基于结构分解和因子分析的贝叶斯网络隐变量发现算法

通过研究粗糙集与图论的关系,提出了以集合为权的加权多重完全多部图的概念,定义了加权多重完全多部图的邻接矩阵,得到了加权完全多部图与决策表的映射关系;给出了粗糙集决策表信息系统的图论形式和决策表信息系统属性约简的图论方法,并根据图论理论对算法进行了优化;得到了在决策表信息系统中,属性的集合不可以约简的充分必要条件;并进一步提出了基于属性置信度的计算方法和多决策属性的处理方法。编程实验结果证明该方法能有效地降低时间和空间复杂度。