Spark下的分布式粗糙集属性约简算法

属性约简（特征选择）作为数据预处理的重要环节，大多以属性依赖作为筛选属性子集的标准。设计了一种快速依赖计算方法FDC，通过直接寻找基于相对正域的对象来计算依赖度，而不需要预先求出相对正域，相比传统方法在速度上有明显的性能提升。另外，改进鲸鱼优化算法（WOA）使其能够有效应用于粗糙集属性约简。结合上述两个方法，提出一种基于Spark的分布式粗糙集属性约简算法SP-WOFRST，并在两组人工合成的大数据集上与另一种基于Spark的粗糙集属性约简算法SP-RST进行对比实验。实验结果表明所提出的SP-WOFRST算法在精度和速度上均优于SP-RST。

Distributed rough set attribute reduction algorithm under Spark

Attribute reduction （feature selection） is an important part of data preprocessing. Most of attribute reduction methods use attribute dependence as the criterion for filtering attribute subsets. A Fast Dependence Calculation （FDC） method was designed to calculate the dependence by directly searching for the objects based on relative positive domains. It is not necessary to find the relative positive domain in advance， so that the method has a significant performance improvement in speed compared with the traditional methods. In addition， the Whale Optimization Algorithm （WOA） was improved to make the calculation method effective for rough set attribute reduction. Combining the above two methods， a distributed rough set attribute reduction algorithm based on Spark named SP-WOFRST was proposed， which was compared with a Spark-based rough set attribute reduction algorithm named SP-RST on two synthetical large data sets. Experimental results show that the proposed SP-WOFRST algorithm is superior to SP-RST in accuracy and speed.