基于kNN算法的智能电网5G海量接入数据异常检测

为了更好适应智能电网高维数据异常识别,提出了一种加权kNN数据异常值检测识别方法,该方法使用Z阶曲线来识别kNN。利用Z阶曲线,提出了一种加权kNN异常数据检测方法。用信息熵衡量所有属性的重要性,用Z阶曲线对高维数据进行编码并映射为Z值。实验结果表明,智能电网集群计算节点的数量越多,算法的运行速度就越短。发电数据异常检测准确率达到最高99.2%,较随机森林算法提高8.165%。且kNN算法的运行时间均优于随机森林算法运行时间,最小算法运行时间为4 s,进一步表明kNN算法可有效检测智能电网5G海量接入数据。     

基于K均值聚类与局部离群因子算法的故障检测研究

针对化工生产过程的多工况、数据多模态问题,提出一种基于K均值聚类的局部离群因子故障检测方法。首先利用K均值聚类算法对多模态工业数据进行聚类,将各个模态的数据分离出来,然后运用局部离群因子算法在各个模态下单独建立模型,并且确定各个模态下的局部离群因子控制限。检测时首先判断样本属于哪一类,然后在相应类别下求取局部离群因子值并与此类别下的控制限进行比较,确定是否为故障数据。将此方法运用到TE过程的多模态数据中,并且将此方法与单独应用局部离群因子算法做故障检测对比,结果表明:所提算法可以大幅提高故障的检测率。     

基于多证据融合决策的间歇过程测量数据异常检测方法

间歇过程测量数据的高维、非线性、非高斯分布特征直接影响过程测量数据异常检测的准确性，为了融合多源数据异常检测信息，提升间歇过程测量数据异常检测精度，提出了一种基于多证据融合决策的间歇过程测量数据异常检测方法，该方法通过引入证据理论（Dempster-Shafer，D-S），采用主焦元判别伪证据和重新计算证据权重改进冲突证据处理方法，减小了冲突证据对多证据融合决策结果的影响，提高了间歇过程测量数据异常检测的准确率。构建了基于多证据融合的测量数据异常检测模型并将其应用到间歇过程测量数据异常检测决策判决中。实验结果表明，该方法能够融合多证据信息，有效地处理冲突证据，实现了间歇过程测量数据异常检测，降低了误检和漏检率。     

基于LSTM时间序列重建的生产装置异常检测

工业生产装置通常设置传感器报警阈值进行报警，但是对处于报警阈值以下的时间序列异常难以及时捕捉。基于统计的传统检测方法在解决时间序列异常检测上存在很大挑战，因此提出基于long short term memory (LSTM)时间序列重建的方法进行生产装置的异常检测。该算法首先引入一层LSTM网络对传感器数据的时间序列进行向量表示，采用另一层LSTM网络对时间序列进行逆序重建，然后利用重建值与实际值之间的误差，通过极大似然估计方法对该段序列进行异常概率估计，最终通过学习异常报警阈值实现时间序列异常检测。采用ECG测试数据、能源数据与危险品储罐传感器数据进行了仿真实验，验证了所提方法在不同长度的数据上的有效性。     

状态估计与不良数据检测方法及其在大庆油田电网中的应用

为了解决常用的状态估计方法常易出现的发散现象,提出了基于改进粒子群算法的状态估计。同时由于系统存在不良数据,不能进行电力系统的常规计算,采用了基于检测法进行不良数据检测。把改进粒子群算法引入到状态估计求解非线性方程中,使得方程的解由不收敛和局部最优解转变为全局最优解。为了验证基于改进粒子群算法的状态估计的正确性和基于检测法进行不良数据检测的合理性,用IEEE9节点系统进行状态估计和不良数据检测的计算,并用编程语言实现了状态估计和不良数据检测的结果显示。结果表明基于改进粒子群算法的状态估计可以得到最优值,但时间较长,检测法可以检测出不良数据。然后对大庆油田电网的10个区分别进行了状态估计和不良数据检测的计算,以一区为例,选取了88个量测数据进行计算,得出了满意的结果,也证明了此方法的正确性。     

基于粒子群算法的多传感器数据融合

粒子群算法是一种有效的寻找函数极值的演化计算方法,它简便易行、收敛速度快,但存在收敛精度不高、易陷入局部极值点的缺点。本文对原有算法中的固定惯性权重进行改进,着重分析了惯性权值因子在粒子群优化(PSO)算法中的作用,在现有的线性递减权值方法上,提出一种非线性权值递减策略,并将其尝试性地运用到多传感器融合的领域,运用该算法对数据融合中的加权因子进行估计。实验结果表明,改进的PSO算法能近似最优地确定数据融合中各权值因子,使融合在信息源的可靠性、信息的冗余度/互补性以及进行融合的分级结构不确定的情况下,以近似最优的方式对传感器数据进行融合,有效地从各融合数据中提取有用信息,成功排除噪声干扰,取得了良好的融合结果。     

一种不等长的多模态间歇过程故障检测方法

提出一种不等长的多模态间歇过程故障检测方法。首先,运用局部加权算法对不等长批次数据进行预处理。在训练样本中确定不等长数据的最大可保留长度,利用k近邻信息,通过加权重构出不等长批次缺失的数据点。其次,对等长的训练集构造局部近邻标准化矩阵,运用K-means算法进行模态聚类,使用局部离群因子方法确定第一控制限,并剔除离群样本。最后,对各个模态建立MPCA模型并确定第二控制限。根据各个模态控制限的匹配系数计算统一的统计量和控制限,在统一的控制限下进行多模态故障检测。将提出方法应用于半导体工业过程,仿真结果表明,与传统的故障检测算法相比,本文算法提高了故障检测率,验证了该方法的有效性。     

基于DHSC的多模态间歇过程测量数据异常检测方法

多模态间歇过程测量数据异常直接影响数据驱动的多元统计分析过程建模的准确性，导致间歇过程的监控性能降低。针对多模态间歇过程测量数据异常问题，提出了一种基于动态超球结构变化（DHSC）的多模态间歇过程测量数据异常检测方法。该方法通过引入时序约束的模糊C均值聚类（SCFCM），利用隶属度变化划分多模态间歇过程的模态；针对不同模态，采用支持向量数据描述（SVDD）建立基于训练数据的静态超球体和基于待检数据的动态超球体，选择重要的支持向量作为球体结构，进而通过识别超球体发生结构变化实现过程测量数据异常检测。青霉素发酵过程仿真实验表明，所提出的方法能够实现多模态间歇过程的模态划分，减少了模态切换对过程测量数据异常检测精度的影响，并能够根据超球体结构变化检测过程测量数据异常，具有较高的检测精度，降低了误检率。     

城市排水管网GIS数据异常的检测与修复

数据是城市排水管网地理信息系统(GIS)的核心和基础。只有保证高质量的数据,才能使GIS系统真正发挥在排水管网信息化管理、建模分析等方面的作用。文章按照“拓扑检测-文本属性项归类-管径检测-高程检测”的处理步骤提出了排水管网数据异常检测与修复的方法,并将方法应用到ZH市的排水管网GIS数据中,结果表明该方法能有效识别拓扑、文本属性、管径和高程数据的异常,并为部分异常数据提供合理的修正参考值。     

一种改进的动态数据校正离群值检测法

Data reconciliation technology can decrease the level of corruption of process data due to measurement noise, but the presence of outliers caused by process peaks or unmeasured disturbances will smear the reconciled results. Based on the analysis of limitation of conventional outlier detection algorithms, a modified outlier detection method in dynamic data reconciliation （DDR） is proposed in this paper. In the modified method, the outliers of each variable are distinguished individually and the weight is modified accordingly. Therefore, the modified method can use more information of normal data, and can efficiently decrease the effect of outliers. Simulation of a continuous stirred tank reactor （CSTR） process verifies the effectiveness of the proposed algorithm.     

一种新型融合离群点的稳态检测方法

针对带有离群点的数据稳态检测,采用分布图法对离群点进行剔除;为了保持数据的完整性,提出用灰色预测值替代离群点值;最后用3δ法则进行稳态检验。如此,数据的稳态与非稳态便会区分开来。与现有稳态检测方法相比,分布图法快速有效地克服了离群点对稳态检测结果不准确的影响,降低了过程中个别异常数据带来的误诊率;灰色预测方法使离群点的替代值更贴近真实值,从而得到的过程数据比现有方法得到的数据更可靠。仿真结果证实了该方法的有效性和优越性。     

An improved methodology for outlier detection in dynamic datasets

A time series Kalman filter (TSKF) is proposed that successfully handles outlier detection in dynamic systems, where normal process changes often mask the existence of outliers. The TSKF method combines a time series model fitting procedure with a modified Kalman filter to deal with additive outlier and innovational outlier detection problems in dynamic process dataset. Compared with current outlier detection methods, the new method enjoys the following advantages: (a) no prior knowledge of the process model is needed; (b) it is easy to tune; (c) it can be applied to both univariate and multivariate outlier detection; (d) it is applicable to both on‐line and off‐line operation; (e) it cleans outliers while maintains the integrity of the original dataset. © 2014 American Institute of Chemical Engineers AIChE J, 61: 419–433, 2015     

Empirical Likelihood for Outlier Detection and Estimation in Autoregressive Time Series

Identification and estimation of outliers in time series is proposed by using empirical likelihood methods. Theory and applications are developed for stationary autoregressive models with outliers distinguished in the usual additive and innovation types. Some other useful outlier types are considered as well. A simulation experiment is used for studying the behaviour of the empirical likelihood‐based method in finite samples and indicates that the proposed methods are preferable when dealing with the non‐Gaussian data. Our simulations suggest that the usual sequential procedure for multiple outlier detection is suitable also for the methods based on empirical likelihood.     

Two Simple Procedures for Testing for a Unit Root When There are Additive Outliers

This paper presents some results on testing for a unit root in the presence of additive outliers. Two procedures are proposed. The first procedure is to ignore the possibility of additive outliers and use modified Phillips–Perron statistics. The second procedure uses a new and very simple outlier detection statistic to identify outliers and then properly adjust standard Dickey–Fuller unit root tests. Simulations show that these procedures are robust to additive outliers in terms of size and power.     

Partial least squares with outlier detection in spectral analysis: A tool to predict gasoline properties

The aim of this study is to propose a novel partial least squares with outlier detection (PLS_OD) calibration method and show its usefulness in calibration successfully with data containing outlying objects. We apply this method in gasoline spectral analysis to predict gasoline properties. In particular, a comparative study of PLS_OD and other five methods is presented. The performances of the proposed method are illustrated on spectral data set with and without outliers. The obtained results suggest that the proposed method can be used for constructing satisfactory gasoline prediction model whether there are some outliers or not.     

SEARCHING FOR ADDITIVE OUTLIERS IN NONSTATIONARY TIME SERIES*

Abstract. Recently, Vogelsang (1999) proposed a method to detect outliers which explicitly imposes the null hypothesis of a unit root. It works in an iterative fashion to select multiple outlier in a given series. We show, via simulations, that, under the null hypothesis of no outliers, it has the right size in finite samples to detect a single outlier but, when applied in an iterative fashion to select multiple outliers, it exhibits severe size distortions towards finding an excessive number of outliers. We show that his iterative method is incorrect and derive the appropriate limiting distribution of the test at each step of the search. Whether corrected or not, we also show that the outliers need to be very large for the method to have any decent power. We propose an alternative method based on first‐differenced data that has considerably more power. We also show that our method to identify outliers leads to unit root tests with more accurate finite sample size and robustness to departures from a unit root. The issues are illustrated using two US/Finland real‐exchange rate series.     

A SCORE TEST FOR DETECTION OF TIME SERIES OUTLIERS

Abstract. Two characterizations, the aberrant observation and innovation models, for outliers in time series are considered. A procedure based on the well-known score-test is discussed for detection of outliers and distinguishing between the outlier types. Significance levels of the tests are also obtained and the method is illustrated with simulated examples.