基于信息论方法的分类数据相似性度量

两实例的距离或相似性度量在数据挖掘和机器学习中扮演着重要的角色。常用的距离度量方法主要适用于数值数据，针对分类数据，本文提出一种数据驱动的相似性度量方法。该方法利用属性值与类标签的信息，将属性值的类条件概率结合信息论来度量分类数据的相似性。为了与已提出的相似性度量方法作比较，把各度量方法与k最近邻算法结合，对多个分类数据集进行分类，通过十折交叉验证比较结果的错误率。实验表明该度量结合k最近邻方法使分类具有较低的错误率。

Similarity Measure of Categorical Data Based on Information Theory

The measure of distance or similarity between two instances plays an important role in data mining and machine learning. The common distance measures are mainly suitable for numerical data, to the classification data, this paper proposes a data-driven similarity measure. This method uses the information of attribute values and class labels to measure the similarity of categorical data by combining the label’s conditional probability of attribute values with information theory. In order to compare with the proposed similarity measures, this paper combines 8 kinds of measure methods with k-nearest neighbor algorithm to classify a plurality of categorical data sets, and the error rates of the results are compared through ten-fold cross validation. Experiments show that this metric combined with k-nearest neighbor method makes a lower error classification rate.