一种基于深度学习的异常数据清洗算法

在物联网(IoT)中采用合适的异常数据清洗算法能极大地提升数据质量.许多研究人员采用统计学方法或分类聚类等方法对时-空相关数据进行清洗.但这些方法需要额外的先验知识,会给汇聚节点带来额外的计算开销.该文根据低秩-稀疏矩阵分解模型,提出一种基于深度神经网络的快速异常数据清洗算法,来解决物联网中时-空相关数据的清洗问题.结...

An Outlier Cleaning Algorithm Based on Deep Learning

The use of appropriate abnormal data cleaning algorithms in the Internet of Things (IoT) can greatly improve data quality. Statistical methods or clustering methods are utilized to clean anomalies in Spatio-temporal data. However, these methods require additional prior knowledge, which will incur additional computational overhead for the sink node. In this paper, in line with the low-rank sparse matrix decomposition model, a fast anomaly cleaning algorithm based on a deep neural network is proposed to solve the Spatio-temporal data cleaning problem in IoT. Both the Spatio-temporal correlation of sensing data and the abnormal values' sparsity are considered in an optimization problem. The Iterative Shrinkage-Thresholding Algorithm (ISTA) is used to solve it. Then the ISTA is unfolded into a fixed-length deep neural network. The real-world dataset’s experimental results show that the proposed method can automatically update the thresholds faster and more accurately than the traditional ISTA.