一种基于信息熵建立决策树的算法

信息论给出了信息的数学本质,提出了用热力学中的熵来度量信息量的大小。分类是一个重要的数据挖掘问题,在训练数据集上建立分类树的关键是如何选择决策树每一个内部结点的测试属性。传统的决策树建立算法利用信息论中熵的概念选择属性,具有偏向于取值较多属性的缺点。本文分析了信息论中有关熵的一些基本概念和含义,讨论了它们在挖掘分类树中的应用,利用互信息设计了一个建立分类树的算法,克服了传统算法的缺点。

An Algorithms to Construct Decision Tree Based on Information Entropy

Information theory provides us what mathematic essence of information is and borrows Entropy concept from thermodynamics to quantify information.Classification plays a big role in data mining.The key of constructing classifier tree based on training set is how to identify the test attribute for each internal node of the tree.Traditional algorithm of decision tree utilizes Entropy to address the issue,which trend to bias towards attributes with more different values.In this paper,we examine some concepts relating to Entropy and discuss their applications in classification tree problem of data mining, and design a decision tree based on mutual information,which eliminates the disadvantages with the old algorithms.