Information entropy,rough entropy and knowledge granulation in incomplete information systems

Rough set theory is a relatively new mathematical tool for use in computer applications in circumstances that are characterized by vagueness and uncertainty. Rough set theory uses a table called an information system, and knowledge is defined as classifications of an information system. In this paper, we introduce the concepts of information entropy, rough entropy, knowledge granulation and granularity measure in incomplete information systems, their important properties are given, and the relationships among these concepts are established. The relationship between the information entropy E(A) and the knowledge granulation GK(A) of knowledge A can be expressed as E(A)+GK(A) = 1, the relationship between the granularity measure G(A) and the rough entropy E _r(A) of knowledge A can be expressed as G(A)+E _r(A) = log₂|U|. The conclusions in Liang and Shi (2004 Liang, J.Y. and Shi, Z.Z. 2004. The information entropy, rough entropy and knowledge granulation in rough set theory. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 12(1): 37–46. [Crossref], [Web of Science ®] , [Google Scholar]) are special instances in this paper. Furthermore, two inequalities ? log₂ GK(A) ≤ G(A) and E _r(A) ≤ log₂(|U|(1 ? E(A))) about the measures GK, G, E and E _r are obtained. These results will be very helpful for understanding the essence of uncertainty measurement, the significance of an attribute, constructing the heuristic function in a heuristic reduct algorithm and measuring the quality of a decision rule in incomplete information systems.     

A fast approach to attribute reduction in incomplete decision systems with tolerance relation-based rough sets

Efficient attribute reduction in large, incomplete decision systems is a challenging problem; existing approaches have time complexities no less than O(∣C∣²∣U∣²). This paper derives some important properties of incomplete information systems, then constructs a positive region-based algorithm to solve the attribute reduction problem with a time complexity no more than O(∣C∣²∣U∣log∣U∣). Furthermore, our approach does not change the size of the original incomplete system. Numerical experiments show that the proposed approach is indeed efficient, and therefore of practical value to many real-world problems. The proposed algorithm can be applied to both consistent and inconsistent incomplete decision systems.     

Three-way decision reduction in neighborhood systems

Rough set reduction has been used as an important preprocessing tool for pattern recognition, machine learning and data mining. As the classical Pawlak rough sets can just be used to evaluate categorical features, a neighborhood rough set model is introduced to deal with numerical data sets. Three-way decision theory proposed by Yao comes from Pawlak rough sets and probability rough sets for trading off different types of classification error in order to obtain a minimum cost ternary classifier. In this paper, we discuss reduction questions based on three-way decisions and neighborhood rough sets. First, the three-way decision reducts of positive region preservation, boundary region preservation and negative region preservation are introduced into the neighborhood rough set model. Second, three condition entropy measures are constructed based on three-way decision regions by considering variants of neighborhood classes. The monotonic principles of entropy measures are proved, from which we can obtain the heuristic reduction algorithms in neighborhood systems. Finally, the experimental results show that the three-way decision reduction approaches are effective feature selection techniques for addressing numerical data sets.     

基于扩展容差关系的不完备信息系统属性约简

针对当前的邻域粗糙集多用于处理完备的信息系统,而非不完备的信息系统这一问题,提出了一种可用于处理不完备混合信息系统的扩展容差关系,并给出相关定义,使用容差完备度和邻域阈值作为限制条件计算扩展容差邻域,以此邻域为基础选择决策正域得到系统的属性重要性,并以该重要性作为启发因子给出基于扩展容差关系的属性约简算法。采用UCI数据集中的7组不同类型的数据集进行仿真实验,并分别与扩展邻域关系（EN）、容差邻域熵（TRE）、邻域粗糙集（NR）的方法进行比较,实验结果表明,该方法在保证分类精度的同时能够约简得到更少的属性。最后讨论了在扩展容差关系中改变邻域阈值对分类精度产生的影响。     

基于边界域的不完备信息系统属性约简方法

研究了用矩阵来计算不完备信息系统的属性约简方法,引入了容差关系矩阵等概念来计算决策表的上下近似集;然后给出了基于容差关系矩阵的决策表边界域的计算方法,再利用边界域的基数相等作为评价属性约简的准则,提出了基于边界域的启发式约简方法;最后,举例说明了属性约简的操作方法和所提算法的可行性。     

Approaches to knowledge reduction of covering decision systems based on information theory

In this paper, we propose some new approaches for attribute reduction in covering decision systems from the viewpoint of information theory. Firstly, we introduce information entropy and conditional entropy of the covering and define attribute reduction by means of conditional entropy in consistent covering decision systems. Secondly, in inconsistent covering decision systems, the limitary conditional entropy of the covering is proposed and attribute reductions are defined. And finally, by the significance of the covering, some algorithms are designed to compute all the reducts of consistent and inconsistent covering decision systems. We prove that their computational complexity are polynomial. Numerical tests show that the proposed attribute reductions accomplish better classification performance than those of traditional rough sets. In addition, in traditional rough set theory, MIBARK-algorithm [G.Y. Wang, H. Hu, D. Yang, Decision table reduction based on conditional information entropy, Chinese J. Comput., 25 (2002) 1-8] cannot ensure the reduct is the minimal attribute subset which keeps the decision rule invariant in inconsistent decision systems. Here, we solve this problem in inconsistent covering decision systems.     

基于疫苗提取及免疫优化的粗糙集属性约简

针对约简属性组合的爆炸问题,将RS属性核参数作为先验信息的免疫疫苗引入抗体编码,概率性对种群接种疫苗.将属性集合的分类近似标准作为抗体适应度,通过在免疫克隆选择过程中引入聚类竞争机制,提高抗体群分布的多样性及亲和力成熟,从而获得多个属性约简及最小约简的平衡.实验结果表明,这种粗糙集属性约简方法对于多维条件属性集是快速且有效的.     

基于近似决策熵的属性约简

粗糙集理论已被证明是一种有效的属性约简方法. 目前有许多启发式属性约简算法已被提出, 其中基于信息熵的属性约简算法受到了广泛的关注. 为此, 针对现有的基于信息熵的属性约简算法问题, 定义一种新的信息熵模型—–近似决策熵, 并提出一种基于近似决策熵的属性约简(ADEAR) 算法. 通过在多个UCI 数据集上的实验表明, 与现有算法相比, ADEAR算法能够获得较小的约简和较高的分类精度, 具有相对较低的计算开销.     

A new approach to attribute reduction of consistent and inconsistent covering decision systems with covering rough sets

Traditional rough set theory is mainly used to extract rules from and reduce attributes in databases in which attributes are characterized by partitions, while the covering rough set theory, a generalization of traditional rough set theory, does the same yet characterizes attributes by covers. In this paper, we propose a way to reduce the attributes of covering decision systems, which are databases characterized by covers. First, we define consistent and inconsistent covering decision systems and their attribute reductions. Then, we state the sufficient and the necessary conditions for reduction. Finally, we use a discernibility matrix to design algorithms that compute all the reducts of consistent and inconsistent covering decision systems. Numerical tests on four public data sets show that the proposed attribute reductions of covering decision systems accomplish better classification performance than those of traditional rough sets.     

基于局部可调节多粒度粗糙集的属性约简

经典的多粒度粗糙集模型采用多个等价关系(多粒度结构)来逼近目标集。根据乐观和悲观策略,常见的多粒度粗糙集分为两种类型:乐观多粒度粗糙集和悲观多粒度粗糙集。然而,这两个模型缺乏实用性,一个过于严格,另一个过于宽松。此外,多粒度粗糙集模型由于在逼近一个概念时需要遍历所有的对象,因此非常耗时。为了弥补这一缺点,进而扩大多粒度粗糙集模型的使用范围,首先在不完备信息系统中引入了可调节多粒度粗糙集模型,随后定义了局部可调节多粒度粗糙集模型。其次,证明了局部可调节多粒度粗糙集和可调节多粒度粗糙集具有相同的上下近似。通过定义下近似协调集、下近似约简、下近似质量、下近似质量约简、内外重要度等概念,提出了一种基于局部可调节多粒度粗糙集的属性约简方法。在此基础上,构造了基于粒度重要性的属性约简的启发式算法。最后,通过实例说明了该方法的有效性。实验结果表明,局部可调节多粒度粗糙集模型能够准确处理不完备信息系统的数据,降低了算法的复杂度。     

基于可辨识矩阵的启发式属性约简方法及其应用

在基于可辨识矩阵的属性约简算法的基础上，提出了基于可辨识矩阵的计算属性重要性的方法，并以此作为启发式知识来约简决策表中的冗余属性。这种方法直接源于评审数据，思路清晰，拟合结果表明本约简算法合理、可靠。     

Attribute selection with fuzzy decision reducts

Rough set theory provides a methodology for data analysis based on the approximation of concepts in information systems. It revolves around the notion of discernibility: the ability to distinguish between objects, based on their attribute values. It allows to infer data dependencies that are useful in the fields of feature selection and decision model construction. In many cases, however, it is more natural, and more effective, to consider a gradual notion of discernibility. Therefore, within the context of fuzzy rough set theory, we present a generalization of the classical rough set framework for data-based attribute selection and reduction using fuzzy tolerance relations. The paper unifies existing work in this direction, and introduces the concept of fuzzy decision reducts, dependent on an increasing attribute subset measure. Experimental results demonstrate the potential of fuzzy decision reducts to discover shorter attribute subsets, leading to decision models with a better coverage and with comparable, or even higher accuracy.     

A systematic study on attribute reduction with rough sets based on general binary relations

Attribute reduction is considered as an important preprocessing step for pattern recognition, machine learning, and data mining. This paper provides a systematic study on attribute reduction with rough sets based on general binary relations. We define a relation information system, a consistent relation decision system, and a relation decision system and their attribute reductions. Furthermore, we present a judgment theorem and a discernibility matrix associated with attribute reduction in each type of system; based on the discernibility matrix, we can compute all the reducts. Finally, the experimental results with UCI data sets show that the proposed reduction methods are an effective technique to deal with complex data sets.     

粗糙集属性约简算法研究

粗糙集理论是一个新的处理不确定性问题的数学工具，属性约简是粗糙集理论的核心问题之一。但求解最优约简已被证明是一个NP—hard问题。基于属性重要度的启发式算法在属性约简中应用的较多．文中分别介绍了基于区分矩阵、基于相关矩阵和基于信息量的属性约简算法。对其思想进行了剖析和总结。     

基于等价类矩阵的属性约简

由于不完备信息系统不能完全适用于粗糙集等价类模型,其合理的属性约简方法的研究在当前是一个备受关注的研究热点。文章给出不完备信息系统等价关系的矩阵表示,同时给出了关于等价类矩阵以及核属性的相关定理,给出了应用等价类矩阵进行属性约简的方法和应用举例,为不完备信息系统的属性约简提供了一种新的方法。     

在不完备信息系统中基于VPRSM的约简异常分析

属性约简问题的关键在于约简集合能否表达与原属性集完全一致的信息。分析了完备信息系统中基于变精度粗糙集模型属性约简异常出现的原因,并将其扩展到不完备信息系统中,在文献[1]的基础上,给出了不完备信息系统中基于集对分析的VPRSM的约简定义,既能保证在不完备信息系统中基于变精度约简的准确性,又能增加灵活性和容噪能力。     

采用布尔矩阵不完备信息系统的属性约简

对于不完备信息系统完备化问题,现在常采用的方法是数据补齐法和扩充法,在研究和分析了其优劣后,提出一种完全依据信息系统提供的信息来产生布尔可辨矩阵的方法。给出一个基于对布尔可辨矩阵进行化简的求属性约简（或最小属性约简）的高效算法。     

不完备决策系统下的多特定类广义决策约简

属性约简是粗糙集理论研究中最重要的领域之一。经典的不完备决策系统广义决策约简关注决策系统中的所有决策类,而在实际应用中,决策者往往只关注一个或者几个特定决策类。针对以上问题,提出基于多特定类的不完备决策系统广义决策约简理论框架。首先,定义了单特定类的不完备决策系统广义决策约简的相关概念,提出并证明相关定理,构造相应差别矩阵和区分函数。其次,将单特定类的广义决策约简推广到多特定类,提出基于差别矩阵的多特定类的不完备决策系统广义决策约简算法。最后,采用6组UCI数据集进行实验。实验结果表明,相对全部决策类数量,当选定特定类数量较少时,平均约简长度有不同程度的缩短,占用空间有所减小,约简效率有不同程度的提升。     

基于相似关系的集值决策信息系统的属性约简

属性约简是粗糙集理论中重要研究内容,由于现实应用领域中决策信息系统往往呈现出不完备性特点,为此,首先将不完备决策表转化为集值决策信息系统,并详细分析了集值决策信息系统下基于相似关系的分布约简和最大分布约简,在此基础上,构造了一种基于可区分对象集的属性约简算法,算法利用可区分对象集的集对,计算极小析取范式求解出所有的分布约简和最大分布约简。最后,利用实例分析验证了算法的有效性。