面向季节性时空数据的预测式循环网络及其在城市计算中的应用

实际生活中有很多带有季节特征的时空数据,在城市计算领域分布尤广,例如交通流量数据便具有较为明显的以天或周为周期的统计学特征.如何有效利用这种季节特征,如何捕捉历史观测与待预测数据之间的相关性,成为了预测此类时空数据未来变化趋势的关键.传统时序建模方法将时序数据分解为多个信号分量,并使用线性模型来进行预测.此类方法具有较强的理论基础,但对于数据的平稳性要求过于严格,难以预测趋势信息复杂的数据,更不适用于高维的时空数据.然而在真实场景下,季节性时空数据的周期长短可变,且不同周期的对应关系往往并不固定,存在时间、空间上的模式变化与偏移,很难作为理想的周期信号以传统时序方法建模.相比之下,深度神经网络建模能力更强,可拟合更为复杂的数据.近几年有许多工作研究了如何利用卷积神经网络和循环神经网络来处理时空数据,也有一些工作讨论了如何有效利用周期性信息提升预测的准确性.但深度神经网络受困于梯度消失和误差累积,难以捕捉时序数据中的长时间依赖,且少有方法讨论如何在深度神经网络中有效建模上述具有弹性周期对应关系的时空信号.本文针对真实场景下季节性时空数据的上述问题,给出具有弹性周期对应关系的时空数据预测问题的形式化定义,并提出了一种新的季节性时空数据预测模型.该模型包含季节网络、趋势网络、时空注意力模块三个部分,可捕捉短期数据中的临近变化趋势和长期数据中隐含的季节性趋势,并广泛考虑历史周期中的每个时空元素对未来预测值的影响.为了解决深度循环网络难以捕捉时序数据中的长时间依赖的问题,本文提出一种新的循环卷积记忆单元,该单元将上述模块融合于一个可端到端训练的神经网络中,一方面实现了时间和空间信息统一建模,另一方面实现了短期趋势特征与历史周期特征的统一建模.进一步地,为了解决季节性数据中的各周期时空元素对应关系不固定的问题,本文探讨了多种基于注意力模块的时空数据融合方式,创新性地提出一种级联式的时空注意力模块,并将其嵌入于上述循环卷积记忆单元内.该模块建模记忆单元的隐藏状态在不同周期内的弹性时空对应关系,自适应地选取相关度高的季节性特征辅助预测.实验部分,我们选取了两个时空数据预测在城市计算中最为典型的应用:交通流量预测和气象数据预报.本文所提出的时空周期性循环神经网络在北京、纽约的交通流量数据集、美国气象数据集上均取得了目前最高的预测准确性.     

基于神经网络集成学习算法的金融时间序列预测

本文在传统神经网络(NN)、循环神经网络(RNN)、长短时记忆网络(LSTM)与门控循环单元(GRU)等神经网络时间预测模型基础上, 进一步构建集成学习(EL)时间序列预测模型, 研究神经网络类模型、集成学习模型和传统时间序列模型在股票指数预测上的表现. 本文以16只A股和国际股票市场指数为样本, 比较模型在不同预测期间和不同国家和地区股票市场上的表现.本文主要结论如下: 第一, 神经网络类时间序列预测模型和神经网络集成学习时间序列预测模型在表现上显著稳健优于传统金融时间序列预测模型, 预测性能提高大约35%; 第二, 神经网络类模型和神经网络集成学习模型在中国和美国股票市场上的表现优于其他发达国家和地区的股票市场.     

Predicting time series using neural networks with wavelet-based denoising layers

To avoid the need to pre-process noisy data, two special denoising layers based on wavelet multiresolution analysis have been integrated into layered neural networks. A gradient-based learning algorithm has been developed that uses the same cost function to set both the neural network weights and the free parameters of the denoising layers. The denoising layers, when integrated into feedforward and recurrent neural networks, were validated on three time series prediction problems: the logistic map, a rubber hardness time series, and annual average sunspot numbers. Use of the denoising layers improved the prediction accuracy in both cases.     

Recurrent neural networks and robust time series prediction

We propose a robust learning algorithm and apply it to recurrent neural networks. This algorithm is based on filtering outliers from the data and then estimating parameters from the filtered data. The filtering removes outliers from both the target function and the inputs of the neural network. The filtering is soft in that some outliers are neither completely rejected nor accepted. To show the need for robust recurrent networks, we compare the predictive ability of least squares estimated recurrent networks on synthetic data and on the Puget Power Electric Demand time series. These investigations result in a class of recurrent neural networks, NARMA(p,q), which show advantages over feedforward neural networks for time series with a moving average component. Conventional least squares methods of fitting NARMA(p,q) neural network models are shown to suffer a lack of robustness towards outliers. This sensitivity to outliers is demonstrated on both the synthetic and real data sets. Filtering the Puget Power Electric Demand time series is shown to automatically remove the outliers due to holidays. Neural networks trained on filtered data are then shown to give better predictions than neural networks trained on unfiltered time series.     

基于长短期神经网络的遥测参数趋势预测

时间序列分析的主要目的是根据已有的历史数据对未来进行预测。传统的时间序列预测主要依靠基于模型的方法，比如季节性差分整合移动平均自回归模型（SARIMA）和指数平滑法（EXP）等。此类方法的参数选择严重依赖于专家经验，适用性并不广泛。针对周期性遥测参数，采用长短期记忆网络（LSTM），学习长时序依赖关系并给出多步预测值。试验通过将预测问题转化为监督学习问题建立半实时仿真环境，并重点研究了观测窗口、预测窗口、网络结构等对性能指标的影响。对比LSTM、SARIMA、EXP，结果表明LSTM具备优异的线性拟合能力和良好的非线性关系映射能力。LSTM预测方法摆脱了传统方法受制于专家经验和模型精度低等问题，为开展实时遥测参数预测奠定了基础。     

基于时间点过程的时间序列预测模型

对城市中发生的事件进行有效预测,可以为政府避免、控制或减轻相关的社会风险提供决策支撑。首先,提出基于积分求导法的条件强度函数式,提高序列预测精度;其次,构建基于递归神经网络和累积危险函数的时间点过程模型,通过递归神经网络捕获历史事件的非线性依赖关系,利用全连接网络获得累积危险函数;最后,选择具有代表性的合成数据集和真实数据集对几种模型的性能进行对比分析。实验结果表明,所提模型可以更好地进行城市事件的时间序列预测,在平均绝对误差、平均负对数似然值等方面均优于传统的时间点过程模型,说明了模型的优越性。     

基于时序分解的用电负荷分析与预测

在智能电网普及的大数据背景下,对电力数据进行精准的分析和预测对电网规划和经济部门的管理决策具有重要的指导意义,但大多数模型都只是在单一的时间尺度上进行研究。针对这一问题提出一种基于时序分解的后向传播算法的循环神经网络预测模型。通过对真实的居民用电消费数据以及外部因素数据统计处理,深入地分析了居民用电特点以及行为规律,并根据其数据的特征以及天气、节假日等外部因素对用户用电行为的影响建立预测模型,对用户未来时段的用电量进行预测。此外,考虑到居民用电消费数据的时序特征在不同时间尺度呈现不同的变化规律,通过时序分解建立预测模型来对用户用电行为的周期性和趋势性进行建模,并通过加权融合达到一起训练的效果,具有一定的协同性,提升预测精度。     

按龄生育率预测的神经网络方法

本文提出了利用人工神经网络非线性时间序列预测方法，对人口按龄生育率问题进行建模和预测，其结果表明这种方法在预测精度和降低模型复杂性上优于传统方法。     

Multiple linear regression and artificial neural networks based on principal components to predict ozone concentrations

The prediction of tropospheric ozone concentrations is very important due to the negative impacts of ozone on human health, climate and vegetation. The development of models to predict ozone concentrations is thus very useful because it can provide early warnings to the population and also reduce the number of measuring sites. The aim of this study was to predict next day hourly ozone concentrations through a new methodology based on feedforward artificial neural networks using principal components as inputs. The developed model was compared with multiple linear regression, feedforward artificial neural networks based on the original data and also with principal component regression. Results showed that the use of principal components as inputs improved both models prediction by reducing their complexity and eliminating data collinearity.     

结合模糊控制的时序数据预测循环神经网络

时序数据处理任务中,循环神经网络模型以及相关衍生模型有较好的性能,如长短期记忆模型(LSTM),门限循环单元(GRU)等.模型的记忆层能够保存每个时间步的信息,但是无法高效处理某些领域的时序数据中的非等时间间隔和不规律的数据波动,如金融数据.本文提出了一种基于模糊控制的新型门限循环单元(GRU-Fuzzy)来解决这些问题.本文在GRU的基础上对记忆层增加了一个子空间分解,由模糊控制模块和一个启发式的失效函数组成,根据数据波动和时间间隔决定记忆层保留的信息量,从而提升模型性能.实验表明,相比于其他的循环神经网络模型,在标普500和上证50中选出股票的股价预测任务中,本文提出的模型有较好的表现.     

A combined Independent Component Analysis–Neural Network model for forecasting exchange rate variation

The currency market is one of the most efficient markets, making it very difficult to predict future prices. Several studies have sought to develop more accurate models to predict the future exchange rate by analyzing econometric models, developing artificial intelligence models and combining both through the creation of hybrid models. This paper proposes a hybrid model for forecasting the variations of five exchange rates related to the US Dollar: Euro, British Pound, Japanese Yen, Swiss Franc and Canadian Dollar. The proposed model uses Independent Component Analysis (ICA) to deconstruct the series into independent components as well as neural networks (NN) to predict each component. This method differentiates this study from previous works where ICA has been used to extract the noise of time series or used to obtain explanatory variables that are then used in forecasting. The proposed model is then compared to random walk, autoregressive and conditional variance models, neural networks, recurrent neural networks and long–short term memory neural networks. The hypothesis of this study supposes that first deconstructing the exchange rate series and then predicting it separately would produce better forecasts than traditional models. By using the mean squared error and mean absolute percentage error as a measures of performance and Model Confidence Sets to statistically test the superiority of the proposed model, our results showed that this model outperformed the other models examined and significantly improved the accuracy of forecasts. These findings support this model’s use in future research and in decision-making related to investments.     

主成分分析与神经网络的结合在多变量序列预测中的应用

目前预测方法的研究主要集中在单变量时间序列上,本文建立起一种针对多元变量非线性时间序列建模和预测的方法框架.首先,同时考虑序列状态间的线性相关性和非线性相关性,建立初始延迟窗以包含充分的预测信息;然后,利用主成分分析(PCA)方法寻找不同变量在数据空间中的最大方差方向,扩展PCA应用于提取多个变量的综合信息,重构多元变量输入状态相空间;最后,利用神经网络逼近不同变量之间以及当前状态和将来状态之间的函数映射关系,实现多元变量预测.对Ro¨ssler混沌方程和大连降雨、气温序列的预测仿真说明了本文方法的有效性,为多元变量时间序列分析提供了一条新的途径.     

Neural learning of chaotic dynamics

In recent years, considerable progress has been made in modeling chaotic time series with neural networks. Most of the work concentrates on the development of architectures and learning paradigms that minimize the prediction error. A more detailed analysis of modeling chaotic systems involves the calculation of the dynamical invariants which characterize a chaotic attractor. The features of the chaotic attractor are captured during learning only if the neural network learns the dynamical invariants. The two most important of these are the largest Lyapunov exponent which contains information on how far in the future predictions are possible, and the Correlation or Fractal Dimension which indicates how complex the dynamical system is. An additional useful quantity is the power spectrum of a time series which characterizes the dynamics of the system as well, and this in a more thorough form than the prediction error does. In this paper, we introduce recurrent networks that are able to learn chaotic maps, and investigate whether the neural models also capture the dynamical invariants of chaotic time series. We show that the dynamical invariants can be learned already by feedforward neural networks, but that recurrent learning improves the dynamical modeling of the time series. We discover a novel type of overtraining which corresponds to the forgetting of the largest Lyapunov exponent during learning and call this phenomenondynamical overtraining. Furthermore, we introduce a penalty term that involves a dynamical invariant of the network and avoids dynamical overtraining. As examples we use the Hénon map, the logistic map and a real world chaotic series that corresponds to the concentration of one of the chemicals as a function of time in experiments on the Belousov-Zhabotinskii reaction in a well-stirred flow reactor.     

A distance-based fuzzy time series model for exchange rates forecasting

Fuzzy time series model has been successfully employed in predicting stock prices and foreign exchange rates. In this paper, we propose a new fuzzy time series model termed as distance-based fuzzy time series (DBFTS) to predict the exchange rate. Unlike the existing fuzzy time series models which require exact match of the fuzzy logic relationships (FLRs), the distance-based fuzzy time series model uses the distance between two FLRs in selecting prediction rules. To predict the exchange rate, a two factors distance-based fuzzy time series model is constructed. The first factor of the model is the exchange rate itself and the second factor comprises many candidate variables affecting the fluctuation of exchange rates. Using the exchange rate data released by the Central Bank of Taiwan, we conducted several experiments on exchange rate forecasting. The experiment results showed that the distance-based fuzzy time series outperformed the random walk model and the artificial neural network model in terms of mean square error.     

基于递归神经网络的风暴潮增水预测

风暴潮增水的准确预测能极大地减少人员伤害和经济损失,具有重要的实用价值。传统的风暴潮预报方法主要包括经验和数值预报,很难建立起相对准确的模型。现有的基于机器学习风暴潮预报方法大都只提取出静态数据间的关系,并没有充分挖掘出风暴潮数据背后的时序关联特性。文中提出了一种基于递归神经网络的风暴潮增水预测方法。本文对风暴潮时序数据进行特定的处理,并设计合适结构的递归神经网络,从而完成时序数据的预测。相较于传统的BP神经网络,递归神经网络能更好地应对时序数据的预测问题。将该方法用于潍坊水站的增水预测中,结果表明,相对于BP神经网络,递归神经网络能得到更好的预测结果,误差更小。     

Time series forecasting by recurrent product unit neural networks

Time series forecasting (TSF) consists on estimating models to predict future values based on previously observed values of time series, and it can be applied to solve many real-world problems. TSF has been traditionally tackled by considering autoregressive neural networks (ARNNs) or recurrent neural networks (RNNs), where hidden nodes are usually configured using additive activation functions, such as sigmoidal functions. ARNNs are based on a short-term memory of the time series in the form of lagged time series values used as inputs, while RNNs include a long-term memory structure. The objective of this paper is twofold. First, it explores the potential of multiplicative nodes for ARNNs, by considering product unit (PU) activation functions, motivated by the fact that PUs are specially useful for modelling highly correlated features, such as the lagged time series values used as inputs for ARNNs. Second, it proposes a new hybrid RNN model based on PUs, by estimating the PU outputs from the combination of a long-term reservoir and the short-term lagged time series values. A complete set of experiments with 29 data sets shows competitive performance for both model proposals, and a set of statistical tests confirms that they achieve the state of the art in TSF, with specially promising results for the proposed hybrid RNN. The experiments in this paper show that the recurrent model is very competitive for relatively large time series, where longer forecast horizons are required, while the autoregressive model is a good selection if the data set is small or if a low computational cost is needed.

     

基于神经网络的时序预测模型研究

古人云“以史为鉴”,说的是吸取历史的经验教训,对未来的情况做出预判或者改变。生活中,亦是存在相似的利用历史数据对未来变化趋势进行预测分析的时间序列问题。本文就时间序列一类的问题进行研究,探讨如何更好地根据历史统计数据,对未来的变化趋势进行预测分析。本文基于神经网络,以气象观测历史数据作为研究的对象,建立了气温变化时序预测模型。本模型利用大数据相关技术对数据进行特征处理,通过深度神经网络,学习特征数据和标签数据之间复杂的非线性关系,从而实现对气温变化的趋势预测。实验结果表明,相较其他模型,本文的模型能够更好地进行时序预测,同时也证明了神经网络用于气象预测的可行性。     

Prediction of a Lorenz chaotic attractor using two-layer perceptron neural network

This paper investigates the prediction of a Lorenz chaotic attractor having relatively high values of Lypunov's exponents. The characteristic of this time series is its rich chaotic behavior. For such dynamic reconstruction problem, regularized radial basis function (RBF) neural network (NN) models have been widely employed in the literature. However, author recommends using a two-layer multi-layer perceptron (MLP) NN-based recurrent model. When none of the available linear models have been able to learn the dynamics of this attractor, it is shown that the proposed NN-based auto regressive (AR) and auto regressive moving average (ARMA) models with regularization have not only learned the true trajectory of this attractor, but also performed much better in multi-step-ahead predictions. However, equivalent linear models seem to fail miserably in learning the dynamics of the time series, despite the low values of Akaike's final prediction error (FPE) estimate. Author proposes to employ the recurrent NN-based ARMA model with regularization which clearly outperforms all other models and thus, it is possible to obtain good results for prediction and reconstruction of the dynamics of the chaotic time series with NN-based models.     

一种基于深度学习的时间序列预测方法

时间序列是一种广泛存在于现实各领域之中的海量高维数据,时间序列预测是该领域的一个研究重点.传统的时间序列预测方法仅仅从时间的维度对时间序列进行分析,忽略了外界影响因素对时间序列可能产生的影响.针对传统时间序列预测方法存在的问题,提出一种基于深度学习的时间序列预测模型DAFDCRNN (dual-stage attention and full dimension convolution based recurrent neural network).该模型引入目标注意力机制来学习输入特征与被预测特征之间的相关性,引入全维度卷积机制来学习输入特征之间的相关性,并引入时间注意力(temporal attention)机制来学习时间序列的长期时间依赖性.在实验部分首先确定模型的超参数,然后对模型部件的有效性进行验证,最后通过对比实验验证了所提出的DAFDC-RNN模型在大特征量数据集上具有最佳的预测效果.     

RF-BiLSTM神经网络在海浪预测中的应用

由于风浪数据的随机性, 复杂性, 影响因素多, 多为时间序列的特点, 造成了传统预测模型预测难度大, 精确率低, 构建了基于随机森林的注意力机制与双向长短期记忆神经网络相结合的海浪预测模型. 该模型对输入进行优化, 可以使用过去和未来的数据信息进行预测, 提高了海浪波高的预测精度. 该模型利用随机森林对输入变量筛选优化, 降低网络复杂度, 然后将注意力机制与双向长短期记忆神经网络相结合建立预测模型, 并利用实际数据进行验证. 结果显示, 和BP, LSTM, BiLSTM模型比较, RF-BiLSTM模型的预测精度更高, 拟合程度更好, 在海浪数值的预测预报中有重要意义.