An artificial neural network (p, d, q) model for timeseries forecasting

Artificial neural networks (ANNs) are flexible computing frameworks and universal approximators that can be applied to a wide range of time series forecasting problems with a high degree of accuracy. However, despite all advantages cited for artificial neural networks, their performance for some real time series is not satisfactory. Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing forecasters. Both theoretical and empirical findings have indicated that integration of different models can be an effective way of improving upon their predictive performance, especially when the models in the ensemble are quite different. In this paper, a novel hybrid model of artificial neural networks is proposed using auto-regressive integrated moving average (ARIMA) models in order to yield a more accurate forecasting model than artificial neural networks. The empirical results with three well-known real data sets indicate that the proposed model can be an effective way to improve forecasting accuracy achieved by artificial neural networks. Therefore, it can be used as an appropriate alternative model for forecasting task, especially when higher forecasting accuracy is needed.     

A new class of hybrid models for time series forecasting

Applying quantitative models for forecasting and assisting investment decision making has become more indispensable in business practices than ever before. Improving forecasting especially time series forecasting accuracy is an important yet often difficult task facing forecasters. Both theoretical and empirical findings have indicated that integration of different models can be an effective way of improving upon their predictive performance, especially when the models in the ensemble are quite different. In the literature, several hybrid techniques have been proposed by combining different time series models together, in order to overcome the deficiencies of single models and yield hybrid models that are more accurate. In this paper, in contrast of the traditional hybrid models, a new methodology is proposed in order to construct a new class of hybrid models using a time series model as basis model and a classifier. As classifiers cannot be lonely applied as forecasting model for continuous problems, in the first stage of the proposed model, a forecasting model is used as basis model. Then, the estimated values of the basis model are modified in the second stage, based on the distinguished trend of the residuals of the basis model and the optimum step length, which are respectively calculated by a classifier model and a mathematical programming model. Empirical results with three well-known real data sets indicate that the proposed model can be an effective way in order to construct a more accurate hybrid model than its basis time series model. Therefore, it can be used as an appropriate alternative model for forecasting tasks, especially when higher forecasting accuracy is needed.     

Prediction method of autoregressive moving average models for uncertain time series

ABSTRACT

Time series analysis is based on the continuous regularity of the development of objective things to predict the next value depending on observed values. Based on time series analysis, we present autoregressive moving average models to predict the next future value for an uncertain time series. In this paper, imprecise observations and disturbance terms are regarded as uncertain variables and assume that the latter are satisfied uncertain normal distribution. The prediction models of uncertain time series are established combining the knowledge of autoregressive model and uncertainty theory. Therefore, the interval range of the next future value is predicted based on the reliability constraint. As an illustration to compare with the numerical examples of the existing prediction method, the innovations and effectiveness of the work are further demonstrated by the computational results.     

Application of neural networks in forecasting engine systems reliability

This paper presents a comparative study of the predictive performances of neural network time series models for forecasting failures and reliability in engine systems. Traditionally, failure data analysis requires specifications of parametric failure distributions and justifications of certain assumptions, which are at times difficult to validate. On the other hand, the time series modeling technique using neural networks provides a promising alternative. Neural network modeling via feed-forward multilayer perceptron (MLP) suffers from local minima problems and long computation time. The radial basis function (RBF) neural network architecture is found to be a viable alternative due to its shorter training time. Illustrative examples using reliability testing and field data showed that the proposed model results in comparable or better predictive performance than traditional MLP model and the linear benchmark based on Box–Jenkins autoregressive-integrated-moving average (ARIMA) models. The effects of input window size and hidden layer nodes are further investigated. Appropriate design topologies can be determined via sensitivity analysis.     

Abnormal visual event detection based on multi-instance learning and autoregressive integrated moving average model in edge-based Smart City surveillance

The abnormal visual event detection is an important subject in Smart City surveillance where a lot of data can be processed locally in edge computing environment. Real-time and detection effectiveness are critical in such an edge environment. In this paper, we propose an abnormal event detection approach based on multi-instance learning and autoregressive integrated moving average model for video surveillance of crowded scenes in urban public places, focusing on real-time and detection effectiveness. We propose an unsupervised method for abnormal event detection by combining multi-instance visual feature selection and the autoregressive integrated moving average model. In the proposed method, each video clip is modeled as a visual feature bag containing several subvideo clips, each of which is regarded as an instance. The time-transform characteristics of the optical flow characteristics within each subvideo clip are considered as a visual feature instance, and time-series modeling is carried out for multiple visual feature instances related to all subvideo clips in a surveillance video clip. The abnormal events in each surveillance video clip are detected using the multi-instance fusion method. This approach is verified on publically available urban surveillance video datasets and compared with state-of-the-art alternatives. Experimental results demonstrate that the proposed method has better abnormal event detection performance for crowded scene of urban public places with an edge environment.     

A Bayesian analysis of moving average processes with time-varying parameters

A new Bayesian method is proposed for estimation and forecasting with Gaussian moving average (MA) processes with time-varying parameters. The focus is placed on MA models of order one, but a general result is given for an MA process of an arbitrary known order. A multiplicative model for the evolution of the squares of the parameters is introduced following Bayesian conjugacy through beta and truncated gamma distributions and a discount factor. Two new distributions are proposed providing the prior and posterior distributions of the parameters of the model and the one-step forecast distribution of the process. Several well-known distributional results are extended by replacing the gamma distribution with the truncated gamma distribution. The proposed methodology is illustrated with two examples consisting of simulated data and of aluminium spot prices of the London metal exchange.     

基于ARIMA-LSSVM混合模型的犯罪时间序列预测

对犯罪时间序列的预测对帮助公安部门更好地掌握犯罪动态,实现智能犯罪发现具有重大意义。针对犯罪时间序列预测的计算需求,结合真实犯罪数据集,提出了ARIMA-LSSVM混合模型。该模型通过ARIMA预测出时间序列的线性部分,通过PSO优化的LSSVM模型预测非线性部分,以对序列进行充分拟合,最后通过混合算法计算最终结果。使用此混合模型达到了精准的预测效果,证明了模型的有效性。     

A hybrid approach of neural networks and grey modeling for adaptive electricity load forecasting

This paper proposes an effective fusion of neural networks and grey modeling for adaptive electricity load forecasting. The fusion employs the complementary strength of these two appealing techniques. In terms of forecasting accuracy, the proposed fusion scheme outperforms the individual ones and the statistical autoregressive methods according to the results of a substantial number of experiments. In addition to the fusion scheme, this paper also proposes a grey relational analysis to automatically assess the importance of each input variable for the forecasting task. This analysis helps the forecaster choose dominant ones among the many input variables, thus removing much burden of acquiring professional domain knowledge for problems and reducing the interference of irrelevant inputs on the forecasting. Experimental results are shown in this paper to verify the effectiveness of the grey relational analysis.     

多元指数加权移动平均主元分析的微小故障检测

主元分析(principal component analysis,PCA)是一种有效的数据分析方法,在故障诊断与状态监测方面已得到广泛应用.多元指数加权移动平均–主元分析(multivariate exponentially weighted moving average principal component analysis,MEWMA–PCA)方法用于解决PCA不能有效检出微小故障的问题.本文深入研究了MEWMA–PCA中EWMA影响主元分析进行故障检测的机制,导出了MEWMA–PCA可检出微小故障的原因.本文确定了MEWMA–PCA中遗忘因子λ、单传感器故障幅值和迟延时间三者的关系,并进行了数值仿真和火电厂磨煤机组运行状态的仿真实验.实验结果验证了MEWMA–PCA中EWMA提高PCA的监测性能的机制,并给出了根据系统实际要求来选取合适的遗忘因子值,从而在规定的时间内检出微小故障的实例.     

改善遗传神经网络收敛性的研究

针对BP神经网络中采用的梯度下降法局部搜索能力强、全局搜索能力差和遗传神经网络中采用的遗传算法全局搜索能力强、局部搜索能力差的特点,提出了一种集梯度下降法和遗传算法优点为一体的混合智能学习法(Hybrid Intelligence learning algorithm),简称HI算法,并将其应用到优化多层前馈型神经网络连接权问题。对该算法进行了设计和实现,从理论和实际两方面证明混合智能学习法神经网络与BP神经网络和基于遗传算法的神经网络相比有更好的运算性能、更快的收敛速度和更高的精度。     

Predicting time series using neural networks with wavelet-based denoising layers

To avoid the need to pre-process noisy data, two special denoising layers based on wavelet multiresolution analysis have been integrated into layered neural networks. A gradient-based learning algorithm has been developed that uses the same cost function to set both the neural network weights and the free parameters of the denoising layers. The denoising layers, when integrated into feedforward and recurrent neural networks, were validated on three time series prediction problems: the logistic map, a rubber hardness time series, and annual average sunspot numbers. Use of the denoising layers improved the prediction accuracy in both cases.     

基于时空图卷积网络的多变量时间序列预测方法

为了扩大时空图卷积网络的预测范围,将它应用在关联关系未知场景下的多变量时间序列预测问题,提出一种附加图学习层的时空图卷积网络预测方法（GLB-STGCN）。图学习层借助余弦相似度从时间序列中学习图邻接矩阵,通过图卷积网络捕捉多变量之间的相互影响,最后通过多核时间卷积网络捕捉时间序列的周期性特征,实现对多变量的精准预测。为验证GLB-STGCN的有效性,使用天文、电力、交通和经济四个领域的公共数据集和一个工业场景生产数据集进行预测实验,结果表明GLB-STGCN优于对比方法,在天文数据集上的表现尤为出色,预测误差分别降低了6.02%、8.01%、6.72%和5.31%。实验结果证明GLB-STGCN适用范围更广,预测效果更好,尤其适合自然周期明显的时间序列预测问题。     

Evolutionary design of hybrid self-organizing fuzzy polynomial neural networks with the aid of information granulation

We introduce a new architecture of information granulation-based and genetically optimized Hybrid Self-Organizing Fuzzy Polynomial Neural Networks (HSOFPNN). Such networks are based on genetically optimized multi-layer perceptrons. We develop their comprehensive design methodology involving mechanisms of genetic optimization and information granulation. The architecture of the resulting HSOFPNN combines fuzzy polynomial neurons (FPNs) that are located at the first layer of the network with polynomial neurons (PNs) forming the remaining layers of the network. The augmented version of the HSOFPNN, “IG_gHSOFPNN”, for brief, embraces the concept of information granulation and subsequently exhibits higher level of flexibility and leads to simpler architectures and rapid convergence speed to optimal structure in comparison with the HSOFPNNs and SOFPNNs.

The GA-based design procedure being applied at each layer of HSOFPNN leads to the selection of preferred nodes of the network (FPNs or PNs) whose local characteristics (such as the number of input variables, the order of the polynomial, a collection of the specific subset of input variables, the number of membership functions for each input variable, and the type of membership function) can be easily adjusted. In the sequel, two general optimization mechanisms are explored. The structural optimization is realized via GAs whereas the ensuing detailed parametric optimization is afterwards carried out in the setting of a standard least square method-based learning. The obtained results demonstrate a superiority of the proposed networks over the existing fuzzy and neural models.     

Representation of neural networks through their multi-linearization

The studies on interpretability of neural networks have been playing an important role in understanding the knowledge developed through their learning and promoting the use of neurocomputing in practical problems. The rule-based setting in which neural networks are interpreted provides a convenient way of expressing knowledge in a transparent and modular manner and at a desired level of granularity (specificity). In this study, we formulate a certain engineering-based style of interpretation in which a given neural network is represented as a collection of local linear models where such models are developed around a collection of linearization nodes. The notion of multi-linearization of neural networks captures the essence of the proposed interpretation. We formulate the problem as an optimization of (i) a collection of linearization nodes around which individual linear models are formed and (ii) aggregation of the individual linearizations, where the linearization fields are subject to optimization. Given the non-differentiable character of the problem, we consider the use of population-based optimization of Particle Swarm Optimization (PSO). Numeric experiments are provided to illustrate the main aspects of the multi-linearization of neural networks.     

基于周期性建模的时间序列预测方法及电价预测研究

时间序列数据广泛存在于人类的生产生活中, 通常具有复杂的非线性动态和一定的周期性. 与传统的时间序列分析方法相比, 基于深度学习的方法更能捕捉数据的深层特性, 对具有复杂非线性的时间序列有较好的建模效果. 为了在神经网络中显式地建模时间序列数据的周期性和趋势性, 本文在循环神经网络的基础上引入了周期损失和趋势损失, 建立了基于周期性建模和多任务学习的时间序列预测模型. 将模型应用到欧洲能源交易所法国市场的能源市场价格预测中, 结果表明周期损失和趋势损失能够提高神经网络的泛化能力, 并提高预测时间序列趋势的精度.     

An integrated framework of genetic network programming and multi-layer perceptron neural network for prediction of daily stock return: An application in Tehran stock exchange market

Evolutionary algorithms are generally used to find or generate the best individuals in a population. Whenever these algorithms are applied to agent systems, they will lead to optimal solutions. Genetic Network Programming (GNP), which contains graph networks, is one of the developed evolutionary algorithms. When the aim is to forecast the share price or return, ascending and descending trends, volatilities, recent returns, fundamental and technical factors have remarkable impacts on the prediction. This is why technical indicators are used to constitute a set of trading rules. In this paper, we apply an integrated framework consisting of GNP model along with a reinforcement learning and Multi-Layer Perceptron (MLP) neural network to classify data and also time series models to forecast the stock return. Moreover, we utilize rules of accumulation based on the GNP model’s results to forecast the return. The aim of using these models alongside one another is to estimate one-day return. The results derived from 9 stocks with regard to the Tehran Stock Exchange Market. GNP extracts a prodigious number of rules on the basis of 5 technical indicators with 3 times period. Next, MLP network classifies data and finds the similarity between future data and past data concerning a stock (5 sub-period) through classification. Subsequently, a number of conditions are established, in order to choose the best estimation between GNP-RL and ARMA. Distinct comparison with the ARMA–GARCH model, which is operated for return estimation and risk measurement in many researches, demonstrates an extended forecasting power of the proposed model, by the name of GNP–ARMA, reducing error by a mean of 16%.     

A novel hybrid classification model of artificial neural networks and multiple linear regression models

The classification problem of assigning several observations into different disjoint groups plays an important role in business decision making and many other areas. Developing more accurate and widely applicable classification models has significant implications in these areas. It is the reason that despite of the numerous classification models available, the research for improving the effectiveness of these models has never stopped. Combining several models or using hybrid models has become a common practice in order to overcome the deficiencies of single models and can be an effective way of improving upon their predictive performance, especially when the models in combination are quite different. In this paper, a novel hybridization of artificial neural networks (ANNs) is proposed using multiple linear regression models in order to yield more general and more accurate model than traditional artificial neural networks for solving classification problems. Empirical results indicate that the proposed hybrid model exhibits effectively improved classification accuracy in comparison with traditional artificial neural networks and also some other classification models such as linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), K-nearest neighbor (KNN), and support vector machines (SVMs) using benchmark and real-world application data sets. These data sets vary in the number of classes (two versus multiple) and the source of the data (synthetic versus real-world). Therefore, it can be applied as an appropriate alternate approach for solving classification problems, specifically when higher forecasting accuracy is needed.     

Impulsive stabilization of delayed neural networks with and without uncertainty

Many dynamic systems in physics, chemistry, biology, engineering, and information science have impulsive dynamical behaviours due to abrupt jumps at certain instants during the dynamical process, and these complex dynamic behaviours can be modelled by impulsive differential systems. This paper formulates and studies the impulsive stabilization of the Hopfield‐type delayed neural networks with and without uncertainty. Several criteria guaranteeing stabilization of such systems are established by employing Lyapunov‐like stability theorem, linear matrix inequality approach, and other inequality techniques. A simple approach to the design of an impulsive controller is then presented. Two numerical examples are given for illustration of the theoretical results. Copyright © 2007 John Wiley & Sons, Ltd.     

Stochastic stability of Markovian jumping Hopfield neural networks with constant and distributed delays

This paper investigates the problem of stability analysis for Markovian jumping Hopfield neural networks (MJHNNs) with constant and distributed delays. Some new delay-dependent stochastic stability criteria are derived based on a novel Lyapunov–Krasovskii functional (LKF) approach. These new criteria based on the delay partitioning idea prove to be less conservative, since the conservatism could be notably reduced by thinning the delay partitioning. Numerical examples are provided to show the effectiveness and advantage of the proposed techniques.     

A novel hybrid learning algorithm for full Bayesian approach of artificial neural networks

The Bayesian neural networks are useful tools to estimate the functional structure in the nonlinear systems. However, they suffer from some complicated problems such as controlling the model complexity, the training time, the efficient parameter estimation, the random walk, and the stuck in the local optima in the high-dimensional parameter cases. In this paper, to alleviate these mentioned problems, a novel hybrid Bayesian learning procedure is proposed. This approach is based on the full Bayesian learning, and integrates Markov chain Monte Carlo procedures with genetic algorithms and the fuzzy membership functions. In the application sections, to examine the performance of proposed approach, nonlinear time series and regression analysis are handled separately, and it is compared with the traditional training techniques in terms of their estimation and prediction abilities.