数据整合中异常检测算法研究

传统的数据整合方案^1]中存在结构上的不严谨性,在整合期间由于各种原因导致整合后的结果存在很多异常离群点,而且并没有有效的措施进行检测和避免.本文提出了基于角度的改进后的三阶段离群点检测算法,通过对数据整合后的结果进行检测,有效地消除了存在的大量疑似离群点.这种改进算法减小了传统算法中对离群点误判的可能性,考虑到数据动态变化的因素,二次验证疑似离群点的异常情况的真实性.本文以生产事故应急救援平台系统项目为背景.

Research on Anomaly Detection Algorithm in Data Integration

Traditional data integration solutions in the presence of the structure are not precise. During the integration period, the integrated result due to various reasons has many abnormal outliers, which cannot be detected and avoided with effective measures. This paper proposes an improved three stage outlier detection algorithm based on angle, which is mainly to detect the results after data integration, and effectively solve the problem of a large number of suspected outliers. This improved algorithm reduces the possibility of outliers in the traditional algorithm, taking into account the factors of dynamic changes in the data, verifying the abnormal real situation of suspected outliers for two times. This paper is backgrounded on the project of production accident emergency rescue system.