A novel hybrid learning algorithm for full Bayesian approach of artificial neural networks

The Bayesian neural networks are useful tools to estimate the functional structure in the nonlinear systems. However, they suffer from some complicated problems such as controlling the model complexity, the training time, the efficient parameter estimation, the random walk, and the stuck in the local optima in the high-dimensional parameter cases. In this paper, to alleviate these mentioned problems, a novel hybrid Bayesian learning procedure is proposed. This approach is based on the full Bayesian learning, and integrates Markov chain Monte Carlo procedures with genetic algorithms and the fuzzy membership functions. In the application sections, to examine the performance of proposed approach, nonlinear time series and regression analysis are handled separately, and it is compared with the traditional training techniques in terms of their estimation and prediction abilities.     

Forecasting nonlinear time series with neural network sieve bootstrap

A new method to construct nonparametric prediction intervals for nonlinear time series data is proposed. Within the framework of the recently developed sieve bootstrap, the new approach employs neural network models to approximate the original nonlinear process. The method is flexible and easy to implement as a standard residual bootstrap scheme while retaining the advantage of being a nonparametric technique. It is model-free within a general class of nonlinear processes and avoids the specification of a finite dimensional model for the data generating process. The results of a Monte Carlo study are reported in order to investigate the finite sample performances of the proposed procedure.     

基于集成预测的稀有时间序列检测

为了解决误判问题,从预测的角度给出了离群点的定义,并提出了预测可信度和离群度的概念;同时,提出采用置换技术来降低离群点对预测模型的影响,并提出了基于集成预测的稀有时间序列检测算法。针对真实数据集的实验表明,可信度和离群度的定义是合理的,稀有时间序列检测算法是有效的。     

Linguistic time series forecasting using fuzzy recurrent neural network

It is known that one of the most spread forecasting methods is the time series analysis. A weakness of traditional crisp time series forecasting methods is that they process only measurement based numerical information and cannot deal with the perception-based historical data represented by linguistic values. Application of a new class of time series, a fuzzy time series whose values are linguistic values, can overcome the mentioned weakness of traditional forecasting methods. In this paper we propose a fuzzy recurrent neural network (FRNN) based time series forecasting method for solving forecasting problems in which the data can be presented as perceptions and described by fuzzy numbers. The FRNN allows effectively handle fuzzy time series to apply human expertise throughout the forecasting procedure and demonstrates more adequate forecasting results. Recurrent links in FRNN also allow for simplification of the overall network structure (size) and forecasting procedure. Genetic algorithm-based procedure is used for training the FRNN. The effectiveness of the proposed fuzzy time series forecasting method is tested on the benchmark examples.     

基于动态递归神经网络模型的混沌时间序列预测

提出了一种动态递归神经网络模型进行混沌时间序列预测,以最佳延迟时间为间隔的最小嵌入维数作为递归神经网络的输入维数,并按预测相点步进动态递归的生成训练数据,利用混沌特性处理样本及优化网络结构,用递归神经网络映射混沌相空间相点演化的非线性关系,提高了预测精度和稳定性。将该模型应用于Lorenz系统数据仿真以及沪市股票综合指数预测,其结果与已有网络模型预测的结果相比较,精度有很大提高。因此,证明了该预测模型在实际混沌时间序列预测领域的有效性和实用性。     

Evolving modular networks with genetic algorithms: application to nonlinear time series

Abstract: A key problem of modular neural networks is finding the optimal aggregation of the different subtasks (or modules) of the problem at hand. Functional networks provide a partial solution to this problem, since the inter‐module topology is obtained from domain knowledge (functional relationships and symmetries). However, the learning process may be too restrictive in some situations, since the resulting modules (functional units) are assumed to be linear combinations of selected families of functions. In this paper, we present a non‐parametric learning approach for functional networks using feedforward neural networks for approximating the functional modules of the resulting architecture; we also introduce a genetic algorithm for finding the optimal intra‐module topology (the appropriate balance of neurons for the different modules according to the complexity of their respective tasks). Some benchmark examples from nonlinear time‐series prediction are used to illustrate the performance of the algorithm for finding optimal modular network architectures for specific problems.     

基于时空图卷积网络的多变量时间序列预测方法

为了扩大时空图卷积网络的预测范围,将它应用在关联关系未知场景下的多变量时间序列预测问题,提出一种附加图学习层的时空图卷积网络预测方法（GLB-STGCN）。图学习层借助余弦相似度从时间序列中学习图邻接矩阵,通过图卷积网络捕捉多变量之间的相互影响,最后通过多核时间卷积网络捕捉时间序列的周期性特征,实现对多变量的精准预测。为验证GLB-STGCN的有效性,使用天文、电力、交通和经济四个领域的公共数据集和一个工业场景生产数据集进行预测实验,结果表明GLB-STGCN优于对比方法,在天文数据集上的表现尤为出色,预测误差分别降低了6.02%、8.01%、6.72%和5.31%。实验结果证明GLB-STGCN适用范围更广,预测效果更好,尤其适合自然周期明显的时间序列预测问题。     

基于RBF神经网络的非线性时间序列在线预测

针对非线性非高斯时间序列, 提出观测噪声服从隐马尔可夫模型(HMM)的径向基函数(RBF)神经网络(RBF-HMM)预测模型, 其特点在于模型输入包含误差反馈项、RBF网络隐含层节点数的可变性和观测噪声的隐马尔可夫性; 并采用序列蒙特卡罗(SMC)方法实现基于RBF-HMM模型的时间序列在线预测. 最后采用太阳黑子数平滑月均值数据和CRU国际钢材价格指数月数据进行实证研究, 结果表明该模型的有效性.     

A high order fuzzy time series forecasting model based on adaptive expectation and artificial neural networks

Many fuzzy time series approaches have been proposed in recent years. These methods include three main phases such as fuzzification, defining fuzzy relationships and, defuzzification. Aladag et al. [2] improved the forecasting accuracy by utilizing feed forward neural networks to determine fuzzy relationships in high order fuzzy time series. Another study for increasing forecasting accuracy was made by Cheng et al. [6]. In their study, they employ adaptive expectation model to adopt forecasts obtained from first order fuzzy time series forecasting model. In this study, we propose a novel high order fuzzy time series method in order to obtain more accurate forecasts. In the proposed method, fuzzy relationships are defined by feed forward neural networks and adaptive expectation model is used for adjusting forecasted values. Unlike the papers of Cheng et al. [6] and Liu et al. [14], forecast adjusting is done by using constraint optimization for weighted parameter. The proposed method is applied to the enrollments of the University of Alabama and the obtained forecasting results compared to those obtained from other approaches are available in the literature. As a result of comparison, it is clearly seen that the proposed method significantly increases the forecasting accuracy.     

Improving artificial neural networks’ performance in seasonal time series forecasting

In this study, an artificial neural network (ANN) structure is proposed for seasonal time series forecasting. The proposed structure considers the seasonal period in time series in order to determine the number of input and output neurons. The model was tested for four real-world time series. The results found by the proposed ANN were compared with the results of traditional statistical models and other ANN architectures. This comparison shows that the proposed model comes with lower prediction error than other methods. It is shown that the proposed model is especially convenient when the seasonality in time series is strong; however, if the seasonality is weak, different network structures may be more suitable.     

Multiple neural networks for a long term time series forecast

The artificial neural network (ANN) methodology has been used in various time series prediction applications. However, the accuracy of a neural network model may be seriously compromised when it is used recursively for making long-term multi-step predictions. This study presents a method using multiple ANNs to make a long term time series prediction. A multiple neural network (MNN) model is a group of neural networks that work together to solve a problem. In the proposed MNN approach, each component neural network makes forecasts at a different length of time ahead. The MNN method was applied to the problem of forecasting an hourly customer demand for gas at a compression station in Saskatchewan, Canada. The results showed that a MNN model performed better than a single ANN model for long term prediction.

Univariate modeling and forecasting of monthly energy demand time series using abductive and neural networks

Neural networks have been widely used for short-term, and to a lesser degree medium and long-term, demand forecasting. In the majority of cases for the latter two applications, multivariate modeling was adopted, where the demand time series is related to other weather, socio-economic and demographic time series. Disadvantages of this approach include the fact that influential exogenous factors are difficult to determine, and accurate data for them may not be readily available. This paper uses univariate modeling of the monthly demand time series based only on data for 6 years to forecast the demand for the seventh year. Both neural and abductive networks were used for modeling, and their performance was compared. A simple technique is described for removing the upward growth trend prior to modeling the demand time series to avoid problems associated with extrapolating beyond the data range used for training. Two modeling approaches were investigated and compared: iteratively using a single next-month forecaster, and employing 12 dedicated models to forecast the 12 individual months directly. Results indicate better performance by the first approach, with mean percentage error (MAPE) of the order of 3% for abductive networks. Performance is superior to naı¨ve forecasts based on persistence and seasonality, and is better than results quoted in the literature for several similar applications using multivariate abductive modeling, multiple regression, and univariate ARIMA analysis. Automatic selection of only the most relevant model inputs by the abductive learning algorithm provides better insight into the modeled process and allows constructing simpler neural network models with reduced data dimensionality and improved forecasting performance.     

A new approach for automated parts recognition using time series analysisand neural networks

This paper presents a new approach for automated parts recognition. It is based on the use of the signature and autocorrelation functions for feature extraction and a neural network for the analysis of recognition. The signature represents the shapes of boundaries detected in digitized binary images of the parts. The autocorrelation coefficients computed from the signature are invariant to transformations such as scaling, translation and rotation of the parts. These unique extracted features are fed to the neural network. A multilayer perceptron with two hidden layers, along with a backpropagation learning algorithm, is used as a pattern classifier. In addition, the position information of the part for a robot with a vision system is described to permit grasping and pick-up. Experimental results indicate that the proposed approach is appropriate for the accurate and fast recognition and inspection of parts in automated manufacturing systems.     

Evolving dynamic Bayesian networks with Multi-objective genetic algorithms

A dynamic Bayesian network (DBN) is a probabilistic network that models interdependent entities that change over time. Given example sequences of multivariate data, we use a genetic algorithm to synthesize a network structure that models the causal relationships that explain the sequence. We use a multi-objective evaluation strategy with a genetic algorithm. The multi-objective criteria are a network's probabilistic score and structural complexity score. Our use of Pareto ranking is ideal for this application, because it naturally balances the effect of the likelihood and structural simplicity terms used in the BIC network evaluation heuristic. We use a basic structural scoring formula, which tries to keep the number of links in the network approximately equivalent to the number of variables. We also use a simple representation that favors sparsely connected networks similar in structure to those modeling biological phenomenon. Our experiments show promising results when evolving networks ranging from 10 to 30 variables, using a maximal connectivity of between 3 and 4 parents per node. The results from the multi-objective GA were superior to those obtained with a single objective GA. Brian J. Ross is a professor of computer science at Brock University, where he has worked since 1992. He obtained his BCSc at the University of Manitoba, Canada, in 1984, his M.Sc. at the University of British Columbia, Canada, in 1988, and his Ph.D. at the University of Edinburgh, Scotland, in 1992. His research interests include evolutionary computation, language induction, concurrency, and logic programming. He is also interested in computer applications in the fine arts. Eduardo Zuviria received a BS degree in Computer Science from Brock University in 2004 and a MS degree in Computer Science from Queen's University in 2006 where he held jobs as teacher and research assistant. Currently, he is attending a Ph.D. program at the University of Montreal. He holds a diploma in electronics from a technical college and has worked for eight years in the computer industry as a software developer and systems administrator. He has received several scholarships including the Ontario Graduate Scholarship, Queen's Graduate Scholarship and a NSERC- USRA scholarship.     

Financial time series prediction using polynomial pipelined neural networks

This paper proposes a novel type of higher-order pipelined neural network: the polynomial pipelined neural network. The proposed network is constructed from a number of higher-order neural networks concatenated with each other to predict highly nonlinear and nonstationary signals based on the engineering concept of divide and conquer. The polynomial pipelined neural network is used to predict the exchange rate between the US dollar and three other currencies. In this application, two sets of experiments are carried out. In the first set, the input data are pre-processed between 0 and 1 and passed to the neural networks as nonstationary data. In the second set of experiments, the nonstationary input signals are transformed into one step relative increase in price. The network demonstrates more accurate forecasting and an improvement in the signal to noise ratio over a number of benchmarked neural networks.     

基于VLRBP神经网络的汇率预测

为了提高汇率预测的准确性,分别使用VLRBP神经网络模型和GRNN模型及ARIMA模型对欧元汇率时间序列进行建模和预测,通过实证分析发现基于VLRBP的神经网络对于含有大量非线性成分的欧元汇率时间序列的预测比较准确.在分析了最速下降BP学习算法的缺点后,提出利用VLRBP学习算法来解决神经网络振荡和收敛速度过慢的缺陷,并取得较好的效果.同时,为了提高VLRBP网络的泛化性能,提出在训练VLRBP神经网络时应用浴盆曲线方法选取隐层神经元个数和滑动窗口尺寸,试验结果表明该方法适合神经网络模型.     

A FCM-based deterministic forecasting model for fuzzy time series

The study of fuzzy time series has increasingly attracted much attention due to its salient capabilities of tackling uncertainty and vagueness inherent in the data collected. A variety of forecasting models including high-order models have been devoted to improving forecasting accuracy. However, the high-order forecasting approach is accompanied by the crucial problem of determining an appropriate order number. Consequently, such a deficiency was recently solved by Li and Cheng [S.-T. Li, Y.-C. Cheng, Deterministic Fuzzy time series model for forecasting enrollments, Computers and Mathematics with Applications 53 (2007) 1904–1920] using a deterministic forecasting method. In this paper, we propose a novel forecasting model to enhance forecasting functionality and allow processing of two-factor forecasting problems. In addition, this model applies fuzzy c-means (FCM) clustering to deal with interval partitioning, which takes the nature of data points into account and produces unequal-sized intervals. Furthermore, in order to cope with the randomness of initially assigned membership degrees of FCM clustering, Monte Carlo simulations are used to justify the reliability of the proposed model. The superior accuracy of the proposed model is demonstrated by experiments comparing it to other existing models using real-world empirical data.     

连续非线性动态系统建模的模糊神经网络方法

提出一种适合于一般连续非线性动态系统建模的新的Runge-Kutta模糊神经网络(RKFNN),证明了RKFNN的存在性。采用传统的Runge-Kutta求积公式构造,实现了对系统的状态变化特性进行学习,解决了直接映射方式对系统的动态轨迹进行学习时存在的精度低等问题,并提出了RKFNN的在线递推学习算法。对连续非线性动态系统进行楚模的仿真结果表明,RKFNN方法是一种较好的方法。     

Predicting time series using neural networks with wavelet-based denoising layers

To avoid the need to pre-process noisy data, two special denoising layers based on wavelet multiresolution analysis have been integrated into layered neural networks. A gradient-based learning algorithm has been developed that uses the same cost function to set both the neural network weights and the free parameters of the denoising layers. The denoising layers, when integrated into feedforward and recurrent neural networks, were validated on three time series prediction problems: the logistic map, a rubber hardness time series, and annual average sunspot numbers. Use of the denoising layers improved the prediction accuracy in both cases.     

Bayesian MLP neural networks for image analysis

We demonstrate the advantages of using Bayesian multi-layer perceptron (MLP) neural networks for image analysis. The Bayesian approach provides consistent way to do inference by combining the evidence from the data to prior knowledge from the problem. A practical problem with MLPs is to select the correct complexity for the model, i.e., the right number of hidden units or correct regularization parameters. The Bayesian approach offers efficient tools for avoiding overfitting even with very complex models, and facilitates estimation of the confidence intervals of the results. In this contribution we review the Bayesian methods for MLPs and present comparison results from two case studies. In the first case, MLPs were used to solve the inverse problem in electrical impedance tomography. The Bayesian MLP provided consistently better results than other methods. In the second case, the goal was to locate trunks of trees in forest scenes. With Bayesian MLP it was possible to use large number of potentially useful features and prior for determining the relevance of the features automatically.