An information theoretic sparse kernel algorithm for online learning

Kernel-based algorithms have been proven successful in many nonlinear modeling applications. However, the computational complexity of classical kernel-based methods grows superlinearly with the increasing number of training data, which is too expensive for online applications. In order to solve this problem, the paper presents an information theoretic method to train a sparse version of kernel learning algorithm. A concept named instantaneous mutual information is investigated to measure the system reliability of the estimated output. This measure is used as a criterion to determine the novelty of the training sample and informative ones are selected to form a compact dictionary to represent the whole data. Furthermore, we propose a robust learning scheme for the training of the kernel learning algorithm with an adaptive learning rate. This ensures the convergence of the learning algorithm and makes it converge to the steady state faster. We illustrate the performance of our proposed algorithm and compare it with some recent kernel algorithms by several experiments.     

A modified quantized kernel least mean square algorithm for prediction of chaotic time series

A modified quantized kernel least mean square (M-QKLMS) algorithm is proposed in this paper, which is an improvement of quantized kernel least mean square (QKLMS) and the gradient descent method is used to update the coefficient of filter. Unlike the QKLMS method which only considers the prediction error, the M-QKLMS method uses both the new training data and the prediction error for coefficient adjustment of the closest center in the dictionary. Therefore, the proposed method completely utilizes the knowledge hidden in the new training data, and achieves a better accuracy. In addition, the energy conservation relation and a sufficient condition for mean-square convergence of the proposed method are obtained. Simulations on prediction of chaotic time series show that the M-QKLMS method outperforms the QKLMS method in terms of steady-state mean square errors.     

Adaptive kernel approach to the time series prediction

This short article describes two kernel algorithms of the regression function estimation. One of them is called HASKE and has its own heuristic of the h parameter evaluation. The second is a hybrid algorithm that connects the SVM and HASKE in such a way that the definition of the local neighborhood is based on the definition of the h-neighborhood from HASKE. Both of them are used as predictors for time series.     

Speeding up dynamic time warping distance for sparse time series data

Dynamic time warping (DTW) distance has been effectively used in mining time series data in a multitude of domains. However, in its original formulation DTW is extremely inefficient in comparing long sparse time series, containing mostly zeros and some unevenly spaced nonzero observations. Original DTW distance does not take advantage of this sparsity, leading to redundant calculations and a prohibitively large computational cost for long time series. We derive a new time warping similarity measure (AWarp) for sparse time series that works on the run-length encoded representation of sparse time series. The complexity of AWarp is quadratic on the number of observations as opposed to the range of time of the time series. Therefore, AWarp can be several orders of magnitude faster than DTW on sparse time series. AWarp is exact for binary-valued time series and a close approximation of the original DTW distance for any-valued series. We discuss useful variants of AWarp: bounded (both upper and lower), constrained, and multidimensional. We show applications of AWarp to three data mining tasks including clustering, classification, and outlier detection, which are otherwise not feasible using classic DTW, while producing equivalent results. Potential areas of application include bot detection, human activity classification, search trend analysis, seismic analysis, and unusual review pattern mining.     

A novel double incremental learning algorithm for time series prediction

Neural Computing and Applications - Based on support vector machine (SVM), incremental SVM was proposed, which has a strong ability to deal with various classification and regression problems....     

一种时间序列数据的动态密度聚类算法

传统的聚类算法多是针对某个时间片上的静态数据集合进行的聚类分析,但事实上大部分数据存在时间序列上的连续动态演变过程.本文对时间序列数据及其类结构的演变过程进行了分析,发现在一定条件下相邻时间片间的数据集间存在较强的关联性,并且类簇结构间则存在一定的继承性.故本文得出新的思想,在前一时间片聚类结果的基础上,通过对部分变化数据的计算和类簇结构的局部调整就有望获得对后一时间片上数据进行完全聚类相同的效果,且运算量会显著下降.基于此思想提出了一种时间序列数据的动态密度聚类算法(DDCA/TSD).仿真实验中使用6种数据集对所提出算法进行了实验验证.结果显示DDCA/TSD在保证聚类准确性的基础上相对传统聚类算法有明显的时间效率提升,并能更有效地发现数据点的属性变化及类簇结构的演变过程.     

一种时间序列相似匹配新算法

为提高时间序列相似匹配的精度和效率，提出一种基于小波包变换的时间序列相似匹配算法．首先利用小波包可对信号进行精细分析的特点，对时间序列进行维数约简，用变换后的低频系数和部分高频均值系数作为特征向量表示原始序列；然后用多维索引结构R树存储这些特征向量，将欧几里德距离作为相似尺度，在此基础上实现了范围查询和k近邻查询，对电力负荷时间序列数据的仿真实验结果表明了算法的有效性。     

一种支持时间序列数据的CBR检索算法

探讨了如何为CBR（基于范例的推理）增加对一种特殊的范例类型——时间序列数据的支持．分析了基于谱分析的时间序列相似度比较算法不适用于CBR检索的缺点，并在此基础上设计了一种综合性能很好的CBR检索算法．思路是把时间序列相似度比较转化成一个卷积问题，并用DFT来简化这个卷积的计算．通过对这种CBR检索算法进行了深入的理论分析和认真的实验，结果证明，提出的算法是一个高效的算法．在这个检索算法的基础上，CBR就能够席用到时序数据的分析推理中，具有广阔的应用前景．     

A framework based on sparse representation model for time series prediction in smart city

Smart city driven by Big Data and Internet of Things(loT)has become a most promising trend of the future.As one important function of smart city,event alert based on time series prediction is faced with the challenge of how to extract and represent discriminative features of sensing knowledge from the massive sequential data generated by IoT devices.In this paper,a framework based on sparse representa-tion model(SRM)for time series prediction is proposed as an efficient approach to tackle this challenge.After dividing the over-complete dictionary into upper and lower parts,the main idea of SRM is to obtain the sparse representation of time series based on the upper part firstly,and then realize the prediction of future values based on the lower part.The choice of different dictionaries has a significant impact on the performance of SRM.This paper focuses on the study of dictionary construction strategy and summarizes eight variants of SRM.Experimental results demonstrate that SRM can deal with different types of time series prediction flexibly and effectively.     

MultiTL-KELM: A multi-task learning algorithm for multi-step-ahead time series prediction

Time series prediction for higher future horizons is of great importance and has increasingly aroused interest among both scholars and practitioners. Compared to one-step-ahead prediction, multi-step-ahead prediction encounters higher dose of uncertainty arising from various facets, including accumulation of errors and lack of information. Many existing studies draw attention to the former issue, while relatively overlook the latter one. Inspired by this discovery, a new multi-task learning algorithm, called the MultiTL-KELM algorithm for short, is proposed for multi-step-ahead time series prediction in this work, where the long-ago data is utilized to provide more information for the current prediction task. The time-varying quality of time-series data usually gives rise to a wide variability between data over long time span, making it difficult to ensure the assumption of identical distribution. How to make the most of, rather than discard the abundant old data, and transfer more useful knowledge to current prediction is one of the main concerns of our proposed MultiTL-KELM algorithm. Besides, unlike typical iterated or direct strategies, MultiTL-KELM regards predictions of different horizons as different tasks. Knowledge from one task can benefit others, enabling it to explore the relatedness among horizons. Based upon its design scheme, MultiTL-KELM alleviates the accumulation error problem of iterated strategy and the time consuming of direct strategies. The proposed MultiTL-KELM algorithm has been compared with several other state-of-the-art algorithms, and its effectiveness has been numerically confirmed by the experiments we conducted on four synthetic and two real-world benchmark time series datasets.     

A forward-constrained regression algorithm for sparse kernel density estimation.

Using the classical Parzen window (PW) estimate as the target function, the sparse kernel density estimator is constructed in a forward-constrained regression (FCR) manner. The proposed algorithm selects significant kernels one at a time, while the leave-one-out (LOO) test score is minimized subject to a simple positivity constraint in each forward stage. The model parameter estimation in each forward stage is simply the solution of jackknife parameter estimator for a single parameter, subject to the same positivity constraint check. For each selected kernels, the associated kernel width is updated via the Gauss-Newton method with the model parameter estimate fixed. The proposed approach is simple to implement and the associated computational cost is very low. Numerical examples are employed to demonstrate the efficacy of the proposed approach.     

基于卷积的时态CBR快速检索算法及应用

探讨了如何增强CBR对一种常见的时态信息,即时间序列数据的检索能力;分析了已有的基于傅里叶频谱分析的时间序列检索算法应用于CBR时遇到的问题,并根据时态CBR检索的需要,提出了一种新的基于循环卷积和傅里叶变换时间序列检索算法.理论分析和数值实验结果都证明,提出的算法在检索效率上有一定的优势.将采取这种检索方法的时态CBR应用于时间序列的预测问题中,取得了较好的预测效果且具有较高的预测效率.     

时间序列数据流预测模型应用研究

时间序列数据流中蕴含了大量潜在信息,可以作为智能决策的依据。研究时间序列数据流的变化趋势,预测其未来一段时间的可能值,能够为当前的决策提供重要的支持。提出用链式可重写窗口技术代替传统的滑动窗口技术,并结合经验模式分解和径向基神经网络建立时间序列数据流在线预测模型——Online_DSPM。实验结果表明,与单一时间序列数据流预测模型相比,该模型具有较高的预测精度和校好的模型适应性。     

基于组合模型的非线性时间序列预测算法

为解决复杂系统中非线性时间序列预测模型构建效率低、预测精度低的问题,提出基于组合模型的HURST-EMD预测算法.采用EMD算法将非线性时间序列分解为代表原始序列特征的各个IMF,然后引入赫斯特(Hurst)指数将同类的IMF整合为新的分量,最后选用LS-SVR-ARIMA模型进行组合预测.在该算法中,设计了序列分类整合等过程,优化了建模的计算量,构建了高效精准的预测模型.为验证模型的有效性,采用上证指数公共数据集和真实交通流数据进行检验,实验结果表明,改进的基于组合模型的HURST-EMD预测算法在提高预测效率的同时具有更好的预测精度.     

基于核稀疏表示的属性选择算法

为解决高维数据在分类时造成的“维数灾难”问题,提出一种新的将核函数与稀疏学习相结合的属性选择算法。具体地,首先将每一维属性利用核函数映射到核空间,在此高维核空间上执行线性属性选择,从而实现低维空间上的非线性属性选择;其次,对映射到核空间上的属性进行稀疏重构,得到原始数据集的一种稀疏表达方式;接着利用L 1范数构建属性评分选择机制,选出最优属性子集;最后,将属性选择后的数据用于分类实验。在公开数据集上的实验结果表明,该算法能够较好地实现属性选择,与对比算法相比分类准确率提高了约3%。     

利用自回归模型的平稳时序数据快速辨识算法

由于自回归模型的参数估计可归结为求解一个线性方程组的问题,故其在平稳时序数据的辨识过程中具有广泛的应用场合。提出了一种基于自回归模型的快速辨识算法,首先,以递推的方式对平稳时序数据自相关函数矩阵的秩的下界值进行估计,然后,以该估计值作为自回归模型的起始阶数对系统进行依次的递阶辨识,最后,基于F检验对相邻阶次的拟合误差的变化趋势进行显著性检验,并以检验结果作为算法的结束条件。新算法在保证较高辨识精度的条件下,其计算效能及辨识精度的稳定性均优于现有的自回归模型辨识算法,实验结果验证了新算法的有效性和先进性。     

Period-aware local modelling and data selection for time series prediction

The paper tackles with local models (LM) for periodical time series (TS) prediction. A novel prediction method is introduced, which achieves high prediction accuracy by extracting relevant data from historical TS for LMs training. According to the proposed method, the period of TS is determined by using autocorrelation function and moving average filter. A segment of relevant historical data is determined for each time step of the TS period. The data for LMs training are selected on the basis of the k-nearest neighbours approach with a new hybrid usefulness-related distance. The proposed definition of hybrid distance takes into account usefulness of data for making predictions at a given time step. During the training procedure, only the most informative lags are taken into account. The number of most informative lags is determined in accordance with the Kraskov's mutual information criteria. The proposed approach enables effective applications of various machine learning (ML) techniques for prediction making in expert and intelligent systems. Effectiveness of this approach was experimentally verified for three popular ML methods: neural network, support vector machine, and adaptive neuro-fuzzy inference system. The complexity of LMs was reduced by TS preprocessing and informative lags selection. Experiments on synthetic and real-world datasets, covering various application areas, confirm that the proposed period aware method can give better prediction accuracy than state-of-the-art global models and LMs. Moreover, the data selection reduces the size of training dataset. Hence, the LMs can be trained in a shorter time.     

一种时序数据的离群数据挖掘新算法

离群数据挖掘是数据挖掘的重要内容，针对时序数据进行离群数据挖掘方法的研究。首先通过对时序数据进行离散傅立叶变换将其从时域空间变换到频域空间，将时序数据映射为多维空间的点，在此基础上，提出一种新的基于距离的离群数据挖掘算法。对某钢铁企业电力负荷时序数据进行仿真实验，结果表明了算法的有效性。     

回声状态网络LM算法及混沌时间序列预测

回声状态网络（ESN）学习算法中可能存在解的奇异问题,在时间序列预测时易导致病态解问题,且伴随着具有较大幅值的输出权值,尤其是当训练样本个数小于输出权值维数时,ESN的解必为奇异的.鉴于此,考虑使用LM（Levenberg Marquardt）算法代替常用的线性回归方法,自适应选择LM参数,从而有效地控制输出权值的幅值,提高ESN的预测性能.通过Lorenz混沌时间序列进行预测研究,对大连月平均气温实际数据进行仿真研究,取得了较好的预测效果.