基于区间模糊匹配函数的数据清洗算法研究及其在问卷调查中的应用

数据清洗是保证数据质量的重要步骤. 由于人类的活动通常带有一定的主观性与情绪性,因此现实中部分数据往往存在不合理性甚至错误. 而此类不合理数据常具有不确定性、模糊性与隐藏性,这给数据清洗带来了困难. 传统的数据清洗方法对此类数据难以充分发挥作用. 结合区间值模糊集理论与匹配函数提出一种区间模糊匹配函数方法,构建区间模糊匹配算法来清洗数据、提高数据质量,并将其应用在问卷调查数据中. 实验结果表明本算法具有较高的准确度及运行效率,适应处理数据中的不合理数据.

Reasearch into Data Cleaning Algorithm Based on Interval FuzzyMatching Functions and Its Application to Questionnaire Data

Data cleaning is a very important step to ensure data quality. The real-world data often has some unreasonable data even error because of human activites usually with subjectivity and emotionality,such as the questionare data. However,there are some difficulties to process data cleaning due to these unreasonable data often being uncertainty,ambiguity and hidding. For this type of data,the traditional data cleaning methods have difficulty in handling the unreasonable data. Therefore,by combining the basic theories of interval-valued fuzzy set and mathcing function,we propose an interval fuzzy matching function method. Based on this method we construct a new algorithm to clean data and improve data quality,and then apply it to questionaire data. Experiments show that our algorithm have a good precision and running efficiency,and that it is adaptable to process the unreasonable data.