Multi-step Learning Rule for Recurrent Neural Models: An Application to Time Series Forecasting

Multi-step prediction is a difficult task that has attracted increasing interest in recent years. It tries to achieve predictions several steps ahead into the future starting from current information. The interest in this work is the development of nonlinear neural models for the purpose of building multi-step time series prediction schemes. In that context, the most popular neural models are based on the traditional feedforward neural networks. However, this kind of model may present some disadvantages when a long-term prediction problem is formulated because they are trained to predict only the next sampling time. In this paper, a neural model based on a partially recurrent neural network is proposed as a better alternative. For the recurrent model, a learning phase with the purpose of long-term prediction is imposed, which allows to obtain better predictions of time series in the future. In order to validate the performance of the recurrent neural model to predict the dynamic behaviour of the series in the future, three different data time series have been used as study cases. An artificial data time series, the logistic map, and two real time series, sunspots and laser data. Models based on feedforward neural networks have also been used and compared against the proposed model. The results suggest than the recurrent model can help in improving the prediction accuracy.     

Performance evaluation of multilayer perceptron,radial basis function,fuzzy inference system,and an adaptively tuned fuzzy wavelet neural network in parameter prediction of multiphase flow measurement instrumentation

Application of predictive models in industrial multiphase flow metering has attracted an increasing attention recently. Void fraction (VF), water–liquid ratio (WLR), and flow regime are key parameters, measured by oil/water/gas multiphase flow metres (MPFM) in petroleum industry. Artificial neural networks and fuzzy inference systems (FIS) are reliable and efficient computational models, which can be simply implemented on microprocessors of MPFMs, having the advantages of trainability, adaptability, and capability to model non‐linear functions. In this paper, a wavelet‐based adaptive neuro‐FIS (WANFIS) is introduced and validated by the prediction of multiphase flow measurement critical parameters including flow regime, VF, and WLR. The performance of the proposed WANFIS model is then compared with multilayer perceptron (MLP), radial basis function (RBF) network, and an FIS trained by fuzzy c‐means and a subtractive clustering method in the prediction of flow parameters in a customized designed structure of oil/water/gas MPFM. Structural parameters of all predictive models are first optimized to yield the most efficient structure for the available dataset. Comparison is then made between the optimized predictive models, in terms of mean squared error of parameter prediction, computation time, and repeatability of the prediction process. According to the obtained results, MLP model using Levenberg–Marquardt training algorithm and WANFIS model using gradient‐based back propagation dynamical iterative learning algorithm are the most efficient models, which give the best performance compared with other used models. All predictive models can predict the flow regime with 100% accuracy, whereas the highest inaccuracy is related to the prediction of WLR. The results of this study can be used to select and develop the most appropriate predictive model applicable in predicting and identifying flow measurement parameters in industrial MPFMs.     

Continuous similarity-based queries on streaming time series

In many applications, local or remote sensors send in streams of data, and the system needs to monitor the streams to discover relevant events/patterns and deliver instant reaction correspondingly. An important scenario is that the incoming stream is a continually appended time series, and the patterns are time series in a database. At each time when a new value arrives (called a time position), the system needs to find, from the database, the nearest or near neighbors of the incoming time series up to the time position. This paper attacks the problem by using fast Fourier transform (FFT) to efficiently find the cross correlations of time series, which yields, in a batch mode, the nearest and near neighbors of the incoming time series at many time positions. To take advantage of this batch processing in achieving fast response time, this paper uses prediction methods to predict future values. When the prediction length is long, FFT is used to compute the cross correlations of the predicted series (with the values that have already arrived) and the database patterns, and to obtain predicted distances between the incoming time series at many future time positions and the database patterns. If the prediction length is short, the direct computation method is used to obtain these predicted distances to avoid the overhead of using FFT. When the actual data value arrives, the prediction error together with the predicted distances is used to filter out patterns that are not possible to be the nearest or near neighbors, which provides fast responses. Experiments show that with reasonable prediction errors, the performance gain is significant. Especially, when the long term predictions are available, the proposed method can handle incoming data at a very fast streaming rate.     

A prediction scheme using perceptually important points and dynamic time warping

An algorithmic method for assessing statistically the efficient market hypothesis (EMH) is developed based on two data mining tools, perceptually important points (PIPs) used to dynamically segment price series into subsequences, and dynamic time warping (DTW) used to find similar historical subsequences. Then predictions are made from the mappings of the most similar subsequences, and the prediction error statistic is used for the EMH assessment. The predictions are assessed on simulated price paths composed of stochastic trend and chaotic deterministic time series, and real financial data of 18 world equity markets and the GBP/USD exchange rate. The main results establish that the proposed algorithm can capture the deterministic structure in simulated series, confirm the validity of EMH on the examined equity indices, and indicate that prediction of the exchange rates using PIPs and DTW could beat at cases the prediction of last available price.     

Recurrent type-1 fuzzy functions approach for time series forecasting

Forecasting the future values of a time series is a common research topic and is studied using probabilistic and non-probabilistic methods. For probabilistic methods, the autoregressive integrated moving average and exponential smoothing methods are commonly used, whereas for non-probabilistic methods, artificial neural networks and fuzzy inference systems (FIS) are commonly used. There are numerous FIS methods. While most of these methods are rule-based, there are a few methods that do not require rules, such as the type-1 fuzzy function (T1FF) approach. While it is possible to encounter a method such as an autoregressive (AR) model integrated with a T1FF, no method that includes T1FF and the moving average (MA) model in one algorithm has yet been proposed. The aim of this study is to improve forecasting by taking the disturbance terms into account. The input dataset is organized using the following variables. First, the lagged values of the time series are used for the AR model. Second, a fuzzy c-means clustering algorithm is used to cluster the inputs. Third, for the MA, the residuals of fuzzy functions are used. Hence, AR, MA, and the degree of memberships of the objects are included in the input dataset. Because the objective function is not derivative, particle swarm optimization is preferable for solving it. The results on several datasets show that the proposed method outperforms most of the methods in literature.     

Analyzing movement predictability using human attributes and behavioral patterns

The ability to predict human mobility, i.e., transitions between a user's significant locations (the home, workplace, etc.) can be helpful in a wide range of applications, including targeted advertising, personalized mobile services, and transportation planning. Most studies on human mobility prediction have focused on the algorithmic perspective rather than on investigating human predictability. Human predictability has great significance, because it enables the creation of more robust mobility prediction models and the assignment of more accurate confidence scores to location predictions. In this study, we propose a novel method for detecting a user's stay points from millions of GPS samples. Then, after detecting these stay points, a long short-term memory (LSTM) neural network is used to predict future stay points. We explore the use of two types of stay point prediction models (a general model that is trained in advance and a personal model that is trained over time) and analyze the number of previous locations needed for accurate prediction. Our evaluation on two real-world datasets shows that by using our preprocessing approach, we can detect stay points from routine trajectories with higher accuracy than the methods commonly used in this domain, and that by utilizing various LSTM architectures instead of the traditional Markov models and advanced deep learning models, our method can predict human movement with high accuracy of more than 40% when using the Acc@1 measure and more than 59% when using the Acc@3 measure. We also demonstrate that the movement prediction accuracy varies for different user populations based on their trajectory characteristics and demographic attributes.     

Deep Learning and Time Series-to-Image Encoding for Financial Forecasting

In the last decade,market financial forecasting has attracted high interests amongst the researchers in pattern recognition.Usually,the data used for analysing the market,and then gamble on its future trend,are provided as time series;this aspect,along with the high fluctuation of this kind of data,cuts out the use of very efficient classification tools,very popular in the state of the art,like the well known convolutional neural networks(CNNs)models such as Inception,Res Net,Alex Net,and so on.This forces the researchers to train new tools from scratch.Such operations could be very time consuming.This paper exploits an ensemble of CNNs,trained over Gramian angular fields(GAF)images,generated from time series related to the Standard&Poor's 500 index future;the aim is the prediction of the future trend of the U.S.market.A multi-resolution imaging approach is used to feed each CNN,enabling the analysis of different time intervals for a single observation.A simple trading system based on the ensemble forecaster is used to evaluate the quality of the proposed approach.Our method outperforms the buyand-hold(B&H)strategy in a time frame where the latter provides excellent returns.Both quantitative and qualitative results are provided.     

A neural-network-based nonlinear metamodeling approach to financial time series forecasting

In financial time series forecasting, the problem that we often encounter is how to increase the prediction accuracy as possible using the financial data with noise. In this study, we discuss the use of supervised neural networks as a meta-learning technique to design a financial time series forecasting system to solve this problem. In this system, some data sampling techniques are first used to generate different training subsets from the original datasets. In terms of these different training subsets, different neural networks with different initial conditions or training algorithms are then trained to formulate different prediction models, i.e., base models. Subsequently, to improve the efficiency of predictions of metamodeling, the principal component analysis (PCA) technique is used as a pruning tool to generate an optimal set of base models. Finally, a neural-network-based nonlinear metamodel can be produced by learning from the selected base models, so as to improve the prediction accuracy. For illustration and verification purposes, the proposed metamodel is conducted on four typical financial time series. Empirical results obtained reveal that the proposed neural-network-based nonlinear metamodeling technique is a very promising approach to financial time series forecasting.     

遗传算法优化BP 神经网络的短时交通流混沌预测

为了提高BP神经网络预测模型对混沌时间序列的预测准确性,提出了一种基于遗传算法优化BP神经网络的改进混沌时间序列预测方法.利用遗传算法优化BP神经网络的权值和阈值,然后训练BP神经网络预测模型以求得最优解,并将该预测方法应用到几个典型混沌时间序列和实测短时交通流时间序列进行有效性验证.仿真结果表明,该方法对典型混沌时间序列和短时交通流具有较好的非线性拟合能力和更高的预测准确性.     

Fuzzy association rule mining approaches for enhancing prediction performance

This paper presents an investigation into two fuzzy association rule mining models for enhancing prediction performance. The first model (the FCM–Apriori model) integrates Fuzzy C-Means (FCM) and the Apriori approach for road traffic performance prediction. FCM is used to define the membership functions of fuzzy sets and the Apriori approach is employed to identify the Fuzzy Association Rules (FARs). The proposed model extracts knowledge from a database for a Fuzzy Inference System (FIS) that can be used in prediction of a future value. The knowledge extraction process and the performance of the model are demonstrated through two case studies of road traffic data sets with different sizes. The experimental results show the merits and capability of the proposed KD model in FARs based knowledge extraction. The second model (the FCM–MSapriori model) integrates FCM and a Multiple Support Apriori (MSapriori) approach to extract the FARs. These FARs provide the knowledge base to be utilized within the FIS for prediction evaluation. Experimental results have shown that the FCM–MSapriori model predicted the future values effectively and outperformed the FCM–Apriori model and other models reported in the literature.     

周相似特性下交通流组合预测方法研究

根据交通流量具有周相似的特性,构造了周相似序列。用霍特指数平滑法对周相似序列进行预测,用人工神经网络对残差部分进行预测。将指数平滑法与神经网络法相结合,以便发挥每种方法的优势,获得比单个方法更好的预测结果。实例分析表明,比单独使用ARIMA或单独使用神经网络方法,使用组合方法的预测误差最小,适合于实时的交通流预测。     

基于小波非线性自回归网络的水文预测模型

不同时间尺度上的水文序列预测在水资源调配和防洪减灾决策中起着重要的作用。提出了一种基于小波分解和非线性自回归神经网络相结合的水文时间序列预测模型(WNARN)。运用Daubechies 5(db5)离散小波将水文序列数据分解为低频和高频子序列,作为非线性自回归神经网络模型(NARN)的输入变量,贝叶斯正则化优化算法用来泛化网络,训练模型对各子序列进行模拟预测,预测值经db5小波重构后得到原序列预测值。利用渭河流域三个水文站40多年的月径流量序列对所提出的WNARN模型进行验证和向前48步的预测能力测试,并与单一NARN模型的验证和预测结果进行对比。结果显示在相同的网络结构下所提出的方法能够显著提高水文序列的预测精度、预测周期及对重大水文事件的预测性,具有较高的泛化能力。     

主成分分析与神经网络的结合在多变量序列预测中的应用

目前预测方法的研究主要集中在单变量时间序列上,本文建立起一种针对多元变量非线性时间序列建模和预测的方法框架.首先,同时考虑序列状态间的线性相关性和非线性相关性,建立初始延迟窗以包含充分的预测信息;然后,利用主成分分析(PCA)方法寻找不同变量在数据空间中的最大方差方向,扩展PCA应用于提取多个变量的综合信息,重构多元变量输入状态相空间;最后,利用神经网络逼近不同变量之间以及当前状态和将来状态之间的函数映射关系,实现多元变量预测.对Ro¨ssler混沌方程和大连降雨、气温序列的预测仿真说明了本文方法的有效性,为多元变量时间序列分析提供了一条新的途径.     

一种基于模糊神经系统的图像去噪方法

提出一种对含有高斯噪声的数字图像的去噪方法,这种方法能够增强高斯噪声滤波器的性能,减少去噪对图像造成的模糊和失真。设计了一个模糊推理系统（FIS）,并利用ANFIS训练这个FIS。通过训练可以调整、优化FIS的内部参数值。训练图像数据由计算机程序自动生成。优化后的FIS即可处理输入的图像数据,产生增强的图像。从结果图像的视觉效果和量化标准两方面的实验和分析,可以看出这种方法可基本消除高斯噪声滤波器产生的模糊和失真,提高滤波器性能。实验表明模糊神经系统可以应用于图像去噪问题。在合理地选择隶属度函数、规则和训练数据的前提下,会产生明显的图像增强效果。     

Extracting the main patterns of natural time series for long-term neurofuzzy prediction

A combination of singular spectrum analysis and locally linear neurofuzzy modeling technique is proposed to make accurate long-term prediction of natural phenomena. The principal components (PCs) obtained from spectral analysis have narrow band frequency spectra and definite linear or nonlinear trends and periodic patterns; hence they are predictable in large prediction horizon. The incremental learning algorithm initiates a model for each of the components as an optimal linear least squares estimation, and adds the nonlinear neurons if they help to reduce error indices over training and validation sets. Therefore, the algorithm automatically constructs the best linear or nonlinear model for each of the PCs to achieve maximum generalization, and the long-term prediction of the original time series is obtained by recombining the predicted components. The proposed method has been primarily tested in long-term prediction of some well-known nonlinear time series obtained from Mackey–Glass, Lorenz, and Ikeda map chaotic systems, and the results have been compared to the predictions made by multi-layered perceptron (MLP) and radial basis functions (RBF) networks. As a real world case study, the method has been applied to the long-term prediction of solar activity where the results have been compared to the long-term predictions of physical precursor and solar dynamo methods.     

Fuzzy-differential evolution algorithm for planning time-optimal trajectories of a unicycle mobile robot on a predefined path

An evolutionary technique with a Fuzzy Inference System (FIS) is offered for planning time-optimal trajectories on a predefined Visibility Graph Method Dijkstra (VGM-D) path of a Nomad 200 mobile robot (MR). First of all, the segmented trajectory is generated by the VGM-D algorithm. Line and curve segments are the components of the trajectory. The number of intersections of the segmented VGM-D path determines the curve segments number. It is assumed that, at each curve segment, translation velocity v _t is taken as constant. The Differential Evolution (DE) algorithm finds v _t values of all the curve segments, which minimize the trajectory tracking time. Line segments lengths are used to calculate the constraints of the problem according to the Nomad 200's limitations on the translation velocity and acceleration/deceleration. The structures of the curve segments are modeled by FIS to decrease the DE's execution time. Another FIS model is used to define the upper bound of the translation velocities on the curve segments for the same purpose. Both FIS models are trained by the adapted-network-based fuzzy inference system (ANFIS). Experiments are successfully implemented on the Nomad 200 MR.     

A unified approach for modeling longitudinal and failure time data,with application in medical monitoring

This paper considers biomedical problems in which a sample of subjects, for example clinical patients, is monitored through time for purposes of individual prediction. Emphasis is on situations in which the monitoring generates data both in the form of a time series and in the form of events (development of a disease, death, etc.) observed on each subject over specified intervals of time. A Bayesian approach to the combined modeling of both types of data for purposes of prediction is presented. The proposed method merges ideas of Bayesian hierarchical modeling, nonparametric smoothing of time series data, survival analysis, and forecasting into a unified framework. Emphasis is on flexible modeling of the time series data based on stochastic process theory. The use of Markov chain Monte Carlo simulation to calculate the predictions of interest is discussed. Conditional independence graphs are used throughout for a clear presentation of the models. An application in the monitoring of transplant patients is presented     

多任务LS-SVM在时间序列预测中的应用

针对单任务时间序列中存在的信息挖掘不充分、预测精度低等问题,提出了一种基于多任务最小二乘支持向量机（MTLS-SVM）的时间序列预测方法。该方法将多个时间序列任务同时进行学习,使得在训练过程中任务之间能够相互牵制起到归纳偏置作用,最终有效提高模型的预测精度。首先,利用相邻时间点之间的密切相关性,构造多个相邻时间点的学习任务,然后将每个任务对应的数据集同时训练MTLS-SVM模型并将其用于预测。将该方法用于几个时间序列数据集并与单任务LS-SVM方法相比,实验结果表明该方法具有较高的预测精度,验证了方法的可行性和有效性。