On efficient multistep non-linear time series prediction in chaotic diode resonator circuits by optimizing the combination of non-linear time series analysis and neural networks

An improved novel non-linear time series prediction method is presented based on optimizing the combination of non-linear signal analysis and deterministic chaos techniques with Artificial Neural Networks of the Multilayer Perceptron (MLP) type. The proposed methodology has been applied to the non-linear time series produced by a diode resonator chaotic circuit. Multisim is used to simulate the circuit and show the presence of chaos. The first stage of the proposed approach employs a non-linear time series analysis module applying the method proposed by Grasberger and Procaccia, involving estimation of the correlation and minimum embedding dimension as well as of the corresponding largest Lyapunov exponent in combination with a nearest neighbour-based non-linear signal predictor. The two previously mentioned modules are used to construct the first stage of a one-step/multistep predictor while a back-propagation MLP is involved in the second stage to enhance prediction results. The novelty of the proposed two-stage predictor lies on that the back-propagation MLP is employed as an error predictor of the nearest neighbour-based first-stage non-linear signal forecasting application following an efficient strategy for optimizing the combination of nearest neighbour prediction based on deterministic chaos techniques and MLP neural networks. This novel two-stage predictor is evaluated through an extensive experimental study and is favourably compared with rival approaches.     

Prediction of multivariate chaotic time series with local polynomial fitting

To improve the prediction accuracy of complex multivariate chaotic time series, a novel scheme formed on the basis of multivariate local polynomial fitting with the optimal kernel function is proposed. According to Takens Theorem, a chaotic time series is reconstructed into vector data, multivariate local polynomial regression is used to fit the predicted complex chaotic system, then the regression model parameters with the least squares method based on embedding dimensions are estimated,and the prediction value is calculated. To evaluate the results, the proposed multivariate chaotic time series predictor based on multivariate local polynomial model is compared with a univariate predictor with the same numerical data. The simulation results obtained by the Lorenz system show that the prediction mean squares error of the multivariate predictor is much smaller than the univariate one, and is much better than the existing three methods. Even if the last half of the training data are used in the multivariate predictor, the prediction mean squares error is smaller than that of the univariate predictor.     

Chaotic Time Series Prediction Using a Neuro-Fuzzy System with Time-Delay Coordinates

This paper presents an investigation into the use of the delay coordinate embedding technique in the multi-input- multioutput-adaptive-network-based fuzzy inference system (MANFIS) for chaotic time series prediction. The inputs to the MANFIS are embedded-phase-space (EPS) vectors preprocessed from the time series under test, while the output time series is extracted from the output EPS vectors from the MANFIS. A moving root-mean-square error is used to monitor the error over the prediction horizon and to tune the membership functions in the MANFIS. With the inclusion of the EPS preprocessing step, the prediction performance of the MANFIS is improved significantly. The proposed method has been tested with one periodic function and two chaotic functions including Mackey-Glass chaotic time series and Duffing forced-oscillation system. The prediction performances with and without EPS preprocessing are statistically compared by using the t-test method. The results show that EPS preprocessing can help improve the prediction performance of a MANFIS significantly.     

Local prediction of non-linear time series using support vector regression

Prediction on complex time series has received much attention during the last decade. This paper reviews least square and radial basis function based predictors and proposes a support vector regression (SVR) based local predictor to improve phase space prediction of chaotic time series by combining the strength of SVR and the reconstruction properties of chaotic dynamics. The proposed method is applied to Hénon map and Lorenz flow with and without additive noise, and also to Sunspots time series. The method provides a relatively better long term prediction performance in comparison with the others.     

遗传算法在混沌时间序列预测中的应用研究*

在混沌时间序列研究中,相空间重构和预测模型参数优化是影响预测性能的关键步骤,利用两者之间的相互联系来提高混沌时间序列预测模型的整体性能,提出一种基于遗传算法的混沌时间序列参数同步优化方法。同步优化方法将相空间重构和最小二乘支持向量机参数作为遗传算法的染色体,预测精度作为遗传算法的适应度函数值,通过遗传算法对参数同步优化问题进行求解。通过混沌时间数据对同步优化方法进行了验证性实验。实验结果表明,相对于单独参数优化方法,同步优化方法不仅提高了混沌时间序列的预测精度,同时降低了计算时间的复杂度。     

Fuzzy prediction of chaotic time series based on fuzzy clustering

The main purpose of this paper is to study a new method to model and predict a chaotic time series using a fuzzy model. First, the GK fuzzy clustering method is used to confirm the input space of the fuzzy model. The goal is to divide the training patterns into representative groups so that patterns within one cluster are more similar than those belonging to other clusters. Then, the Kalman filtering algorithm with singular value decomposition is applied to estimate the consequent parameters of the fuzzy model in order to avoid error delivery and error accumulation. The effectiveness of the proposed method is evaluated through simulated examples, including Mackey‐Glass time series and Lorenz chaotic systems. The results show that the proposed method provides effective and accurate prediction. Copyright © 2011 John Wiley and Sons Asia Pte Ltd and Chinese Automatic Control Society     

交通流量VNNTF神经网络模型多步预测研究

研究了VNNTF 神经网络（Volterra neural network trafficflow model,VNNTF） 交通流量混沌时间序列多步预测问题. 通过分析比较交通流量混沌时间序列相空间重构的嵌入维数和Volterra 离散模型之间的关系,给出了确定交通流量Volterra 级数模型截断阶数和截断项数的方法,并在此基础上建立了VNNTF 神经网络交通流量时间序列模型;设计了交通流量Volterra 神经网络的快速学习算法;最后,利用交通流量混沌时间序列对VNNTF 网络模型,Volterra 预测滤波器和BP 网络进行了多步预测实验,比较了多步预测结果的仿真图、绝对误差的柱状图以及归一化后的方均根;实验结果表明VNNTF 神经网络的多步预测性能明显优于Volterra 预测滤波器和BP 神经网络.     

Support vector regression with chaos-based firefly algorithm for stock market price forecasting

Due to the inherent non-linearity and non-stationary characteristics of financial stock market price time series, conventional modeling techniques such as the Box–Jenkins autoregressive integrated moving average (ARIMA) are not adequate for stock market price forecasting. In this paper, a forecasting model based on chaotic mapping, firefly algorithm, and support vector regression (SVR) is proposed to predict stock market price. The forecasting model has three stages. In the first stage, a delay coordinate embedding method is used to reconstruct unseen phase space dynamics. In the second stage, a chaotic firefly algorithm is employed to optimize SVR hyperparameters. Finally in the third stage, the optimized SVR is used to forecast stock market price. The significance of the proposed algorithm is 3-fold. First, it integrates both chaos theory and the firefly algorithm to optimize SVR hyperparameters, whereas previous studies employ a genetic algorithm (GA) to optimize these parameters. Second, it uses a delay coordinate embedding method to reconstruct phase space dynamics. Third, it has high prediction accuracy due to its implementation of structural risk minimization (SRM). To show the applicability and superiority of the proposed algorithm, we selected the three most challenging stock market time series data from NASDAQ historical quotes, namely Intel, National Bank shares and Microsoft daily closed (last) stock price, and applied the proposed algorithm to these data. Compared with genetic algorithm-based SVR (SVR-GA), chaotic genetic algorithm-based SVR (SVR-CGA), firefly-based SVR (SVR-FA), artificial neural networks (ANNs) and adaptive neuro-fuzzy inference systems (ANFIS), the proposed model performs best based on two error measures, namely mean squared error (MSE) and mean absolute percent error (MAPE).     

基于异常序列剔除的多变量时间序列结构化预测

针对传统多变量时间序列预测方法未考虑变量间依赖关系从而影响预测效果的问题,提出了一种基于异常序列剔除的多变量时间序列预测算法.该算法旨在利用多维支持向量回归机（Multi-dimensional support vector regression,M-SVR）内在的结构化输出特性,对选取到具有相似性的多个变量序列进行联合预测.首先,对已知序列进行基于模糊熵的层次聚类,实现对相似序列的初步划分;其次,求出类中所有序列的主曲线,根据序列到主曲线的距离计算各个序列的异常因子,从而进一步剔除聚类结果中的异常序列;最后,将选取到的相似变量序列作为输入,利用M-SVR进行预测.通过理论分析,证明本文算法在理论上存在信息损失上界与可靠度下界,从而说明本文算法的合理性与可行性.采用混沌时间序列数据与多个实际数据集进行对比实验,结果表明,与现有多个代表性方法相比,本文算法可有效挖掘多变量时间序列的内在结构信息,预测精度更高,数值稳定性更好.     

基于模糊信息粒化与混合神经网络的混沌时间序列预测

为了提高对混沌时间序列预测的精准度,提出了一种基于模糊信息粒化和注意力机制的混合神经网络预测模型。首先对数据进行归一化处理,利用模糊信息粒化对数据的复杂度进行简化;然后将经过相空间重构后的样本输入卷积神经网络(CNN)提取空间特征;再利用长短期记忆神经网络(LSTM)进一步提取时间特征;最后将融合特征传递给注意力机制提取关键特征,得出预测结果。选取Logistic、洛伦兹和太阳黑子混沌时间序列进行实验,并与CNN-LSTM-Att模型、CNN-LSTM模型、FIG-CNN模型、FIG-LSTM模型、CNN模型、LSTM模型、支持向量机(SVM)及误差逆传播(BP)模型进行对比分析。结果表明,所提的预测模型预测精度更高,误差更小。     

基于支持向量机的混沌时间序列预测分析

混沌的特性决定了混沌系统很难长期预测,支持向量机有强大的学习能力,根据相空间重构理论用支持向量机建立预测模型对混沌时间序列进行短期预测。预测输出构建混沌吸引子来定性评价预测模型性能,同时与BP神经网络RBF神经网络构建的预测模型比较,计算预测模型的均方根误差定量地评价模型的性能。仿真结果表明,该方法具有较高的预测精度和泛化能力。     

High-precision forecast using grey models

In recent years the grey theorem has been successfully used in many prediction applications. The proposed Markov-Fourier grey model prediction approach uses a grey model to predict roughly the next datum from a set of most recent data. Then, a Fourier series is used to fit the residual error produced by the grey model. With the Fourier series obtained, the error produced by the grey model in the next step can be estimated. Such a Fourier residual correction approach can have a good performance. However, this approach only uses the most recent data without considering those previous data. In this paper, we further propose to adopt the Markov forecasting method to act as a longterm residual correction scheme. By combining the short-term predicted value by a Fourier series and a long-term estimated error by the Markov forecasting method, our approach can predict the future more accurately. Three time series are used in our demonstration. They are a smooth functional curve, a curve for the stock market and the Mackey-Glass chaotic time series. The performance of our approach is compared with different prediction schemes, such as back-propagation neural networks and fuzzy models. All these methods are one-step-ahead forecasting. The simulation results show that our approach can predict the future more accurately and also use less computational time than other methods do.     

Prediction of noisy chaotic time series using an optimal radialbasis function neural network

This paper considers the problem of optimum prediction of noisy chaotic time series using a basis function neural network, in particular, the radial basis function (RBF) network. In the noiseless environment, predicting a chaotic time series is equivalent to approximating a nonlinear function. The optimal generalization is achieved when the number of hidden units of a RBF predictor approaches infinity. When noise exists, it is shown that an optimal RBF predictor should use a finite number of hidden units. To determine the structure of an optimal RBF predictor, we propose a new technique called the cross-validated subspace method to estimate the optimum number of hidden units. While the subspace technique is used to identify a suitable number of hidden units by detecting the dimension of the subspace spanned by the signal eigenvectors, the cross validation method is applied to prevent the problem of overfitting. The effectiveness of this new method is evaluated using simulated noisy chaotic time series as well as real-life oceanic radar signals. Results show that the proposed method can find the correct number of hidden units of an RBF network for an optimal prediction.     

FOR NONLINEARITY IN THE HEARTBEAT TIME SERIES

Hearbeat time series obtained by Holter monitoring of 10 healthy subjects are studied using a nonlinear predictor S map . The normalized prediction error was calculated as a function of the model control parameter in order to get information on the amount of nonlinearity in the data. Moreover, to search for possible chaotic behavior, the linear correlation coefficient between predicted and real values was calculated as a function of the prediction time. The results of this analysis reveal no clear evidence of chaoticness or nonlinearity in the data. Moreover, for 8 subjects out of 10, the predictability during sleep is better than during the daytime.     

Gradient radial basis function networks for nonlinear andnonstationary time series prediction

We present a method of modifying the structure of radial basis function (RBF) network to work with nonstationary series that exhibit homogeneous nonstationary behavior. In the original RBF network, the hidden node's function is to sense the trajectory of the time series and to respond when there is a strong correlation between the input pattern and the hidden node's center. This type of response, however, is highly sensitive to changes in the level and trend of the time series. To counter these effects, the hidden node's function is modified to one which detects and reacts to the gradient of the series. We call this new network the gradient RBF (GRBF) model. Single and multistep predictive performance for the Mackey-Glass chaotic time series were evaluated using the classical RBF and GRBF models. The simulation results for the series without and with a tine-varying mean confirm the superior performance of the GRBF predictor over the RBF predictor.     

Multiple predictors hiding scheme using asymmetric histograms

In recent years, many data hiding techniques have been proposed, and they can be generally classified into two types according to the reversibility of the image; these two types are reversible and irreversible data hiding. This study focused on reversible data hiding, which makes recovering the cover image possible after the secret data has been extracted. In 2013, Chen et al. proposed an asymmetric-histogram reversible data hiding method. In their scheme, two prediction error histograms (maximum and minimum error histograms) were used to embed the secret message. Two histograms were shifted in opposite directions. Hence, some stego-pixels were shifted to their original values. The complementary embedding strategy is effective. However, the predictor in the method is rough. Only neighboring pixels were used to generate the prediction errors, thereby resulting in poor prediction efficiency. To enhance the prediction efficiency, this paper combines several well-known predictors such as gradient adjusted gap (GAP), median edge detect, and interpolation by neighboring pixel (INP) to generate prediction errors. Different predictors along with the asymmetric-histogram method can achieve better results. The predictor GAP used more neighboring pixels to obtain the prediction value; therefore, it is suitable for complex images. However, the predictor INP only considers that closer pixels can achieve great results for smooth images. Hence, the proposed scheme combines GAP and asymmetric histogram for complex images. However, the predictor INP along with asymmetric histogram is used for smooth images. Experimental results showed that the PSNR value of the proposed method is greater than that of the asymmetric-histogram shifting method and other recent approaches.     

Chaotic time series prediction via artificial neural square fuzzy inference system

The present article investigates the application of second order TSK (Takagi Sugeno Kang) fuzzy systems in predicting chaotic time series. A method has been introduced for training second order TSK fuzzy systems using ANFIS (Artificial Neural Fuzzy Inference System) training method. In a second order TSK system existence of nonlinear terms in the rules’ consequence prohibits use of current available ANFIS codes as is but the proposed method makes it possible to use ANFIS for a class of simplified second order TSK systems. The main impact of this method on the expert and intelligent systems is to provide a new way for modeling and predicting the future situation of more complex phenomena with a smaller decision rule base. The most significance of the proposed method is the simplicity and available code reuse property. As a case study the proposed method is used for the prediction of chaotic time series. Error comparison shows that the proposed method trains the second order TSK system more effectively.     

Time series prediction using Lyapunov exponents in embedding phase space

This paper describes a novel method of chaotic time series prediction, which is based on the fundamental characteristic of chaotic behavior that sensitive dependence upon initial conditions (SDUIC), and Lyapunov exponents (LEs) is a measure of the SDUIC in chaotic systems. Because LEs of chaotic time series data provide a quantitative analysis of system dynamics in different embedding dimension after embedding a chaotic time series in different embedding dimension phase spaces, a novel multi-dimension chaotic time series prediction method using LEs is proposed in this paper. This is done by first reconstructing a phase space using chaotic time series, then using LEs as a quantitative parameter to predict an unknown phase space point, after transferring the phase space point to time domain, the predicted chaotic time series data can be obtained. The computer simulation result of simulation showed that the proposed method is simple, practical and effective.     

遗传算法优化BP神经网络的混沌时间序列预测

为提高BP神经网络预测模型对混沌时间序列的预测精度,将改进的遗传算法和BP神经网络结合,提出了一种基于改进遗传算法优化BP神经网络的混沌时间序列预测方法。利用改进的遗传算法优化BP神经网络的权值和阈值,训练BP神经网络预测模型求得最优解。将该模型应用到几个典型的非线性系统进行预测仿真,验证了该算法的有效性,与BP神经网络预测模型的预测结果进行了比较,仿真结果表明该方法对混沌时间序列具有更好的非线性拟合能力和更高的预测精度。