A comparative study of neural network and Box-Jenkins ARIMA modeling in time series prediction

This paper aims to investigate suitable time series models for repairable system failure analysis. A comparative study of the Box-Jenkins autoregressive integrated moving average (ARIMA) models and the artificial neural network models in predicting failures are carried out. The neural network architectures evaluated are the multi-layer feed-forward network and the recurrent network. Simulation results on a set of compressor failures showed that in modeling the stochastic nature of reliability data, both the ARIMA and the recurrent neural network (RNN) models outperform the feed-forward model; in terms of lower predictive errors and higher percentage of correct reversal detection. However, both models perform better with short term forecasting. The effect of varying the damped feedback weights in the recurrent net is also investigated and it was found that RNN at the optimal weighting factor gives satisfactory performances compared to the ARIMA model.     

结合深度学习和分解算法的股票价格预测研究

针对股票价格预测问题,实现对非平稳、非线性股票价格序列的预测,提出一种结合深度学习和分解算法的股票价格预测模型.该模型引入自适应噪声的完整集成经验模态分解(CEEMDAN)算法提取股票价格时间序列在时间尺度上的特征,利用注意力机制捕获输入特征参数的权重并结合门控循环单元(GRU)网络进行股票价格预测.实验对苹果、贵州茅...     

A novel model by evolving partially connected neural network for stock price trend forecasting

This paper proposes a novel model by evolving partially connected neural networks (EPCNNs) to predict the stock price trend using technical indicators as inputs. The proposed architecture has provided some new features different from the features of artificial neural networks: (1) connection between neurons is random; (2) there can be more than one hidden layer; (3) evolutionary algorithm is employed to improve the learning algorithm and training weights. In order to improve the expressive ability of neural networks, EPCNN utilizes random connection between neurons and more hidden layers to learn the knowledge stored within the historic time series data. The genetically evolved weights mitigate the well-known limitations of gradient descent algorithm. In addition, the activation function is defined using sin(x) function instead of sigmoid function. Three experiments were conducted which are explained as follows. In the first experiment, we compared the predicted value of the trained EPCNN model with the actual value to evaluate the prediction accuracy of the model. Second experiment studied the over fitting problem which occurred in neural network training by taking different number of neurons and layers. The third experiment compared the performance of the proposed EPCNN model with other models like BPN, TSK fuzzy system, multiple regression analysis and showed that EPCNN can provide a very accurate prediction of the stock price index for most of the data. Therefore, it is a very promising tool in forecasting of the financial time series data.     

An ensemble of neural networks for weather forecasting

This study presents the applicability of an ensemble of artificial neural networks (ANNs) and learning paradigms for weather forecasting in southern Saskatchewan, Canada. The proposed ensemble method for weather forecasting has advantages over other techniques like linear combination. Generally, the output of an ensemble is a weighted sum, which are weight-fixed, with the weights being determined from the training or validation data. In the proposed approach, weights are determined dynamically from the respective certainties of the network outputs. The more certain a network seems to be of its decision, the higher the weight. The proposed ensemble model performance is contrasted with multi-layered perceptron network (MLPN), Elman recurrent neural network (ERNN), radial basis function network (RBFN), Hopfield model (HFM) predictive models and regression techniques. The data of temperature, wind speed and relative humidity are used to train and test the different models. With each model, 24-h-ahead forecasts are made for the winter, spring, summer and fall seasons. Moreover, the performance and reliability of the seven models are then evaluated by a number of statistical measures. Among the direct approaches employed, empirical results indicate that HFM is relatively less accurate and RBFN is relatively more reliable for the weather forecasting problem. In comparison, the ensemble of neural networks produced the most accurate forecasts.     

使用GNN与RNN实现用户行为分析

随着国家高性能计算环境(CNGrid)各个节点产生日志数量不断增加,采用传统的人工方式进行用户行为分析已不能满足日常的分析需求.近年来,深度学习在入侵检测、图像识别、自然语言处理和恶意软件检测等与计算机科学相关的关键任务中取得了良好的效果.演示了如何将深度学习模型应用于用户行为分析.为此,在CNGrid中对用户行为进行...     

基于卷积循环神经网络的关系抽取

关系抽取是信息抽取领域一项十分具有挑战性的任务,用于将非结构化文本转化为结构化数据。近年来,卷积神经网络和循环神经网络等深度学习模型,被广泛应用于关系抽取的任务中,且取得了不错的效果。卷积网络和循环网络在该任务上各有优势,且存在一定的差异性。其中,卷积网络擅长局部特征提取,循环网络能够捕获序列整体信息。针对该现象,该文综合卷积网络抽取局部特征的优势和循环网络在时序依赖中的建模能力,提出了卷积循环神经网络(convolutional recurrent neural network,CRNN)。该模型分为三层: 首先针对关系实例抽取多粒度局部特征,然后通过聚合层融合不同粒度的特征,最后利用循环网络提取特征序列的整体信息。此外,该文还探究多种聚合策略对信息融合的增益,发现注意力机制对多粒度特征的融合能力最为突出。实验结果显示,CRNN优于主流的卷积神经网络和循环神经网络,在SemEval 2010 Task 8数据集上取得了86.52%的F₁值。     

模糊神经网络与SARIMA结合的时间序列预测模型

模糊神经网络和SARIMA模型分别对非线性和线性时间序列有很好的预测能力,但在实际应用中大多数序列并非稳定、单纯线性或非线性的。为了提高预测精度,提出了一种基于T-S模糊神经网络与SARIMA结合的时间序列预测模型。针对悉尼航班乘客收入数据给出了三种混合模型,并与模糊神经网络、支持向量机、SARIMA和BP神经网络四种单独模型进行比较。实验结果表明,从预测精度和参数选择方面来看,所给模型是有效的。     

A regime-switching recurrent neural network model applied to wind time series

This paper proposes a regime-switching recurrent network model (RS-RNN) for non-stationary time series. The RS-RNN model emits a mixture density with dynamic nonlinear regimes that fit flexibly data distributions with non-Gaussian shapes. The key novelties are: development of an original representation of the means of the component distributions by dynamic nonlinear recurrent networks, and derivation of a corresponding expectation maximization (EM) training algorithm for finding the model parameters. The results show that the RS-RNN applied to a real-world wind speed time series achieves standardized residuals similar to popular previous models, but it is more accurate distribution forecasting than other linear switching (MS-AR) and nonlinear neural network (MLP and RNN) models.     

Application of soft computing models to hourly weather analysis in southern Saskatchewan,Canada

Accurate weather forecasts are necessary for planning our day-to-day activities. However, dynamic behavior of weather makes the forecasting a formidable challenge. This study presents a soft computing model based on a radial basis function network (RBFN) for 24-h weather forecasting of southern Saskatchewan, Canada. The model is trained and tested using hourly weather data of temperature, wind speed and relative humidity in 2001. The performance of the RBFN is compared with those of multi-layered perceptron (MLP) network, Elman recurrent neural network (ERNN) and Hopfield model (HFM) to examine their applicability for weather analysis. Reliabilities of the models are then evaluated by a number of statistical measures. The results indicate that the RBFN produces the most accurate forecasts compared to the MLP, ERNN and HFM.     

Neuro-fuzzy position control of demining tele-operation system based on RNN modeling

The paper considers the neuro-fuzzy position control of multi-finger robot hand in tele-operation system—an active master–slave hand system (MSHS) for demining. Recently, fuzzy control systems utilizing artificial intelligent techniques are also being actively investigated in robotic area. Neural network with their powerful learning capability are being sought as the basis for many adaptive control systems where on-line adaptation can be implemented. Fuzzy logic on the other hand has been proved to be rather popular in many control system applications providing a rule-base like structure. In this paper, the design and optimization process of fuzzy position controller is supported by learning techniques derived from neural network where a radial basis function (RBF) neural network is implemented to learn fuzzy rules and membership functions with predictor of recurrent neural network (RNN) model. The results of experiment show that based on the predictive capability of RNN model neuro-fuzzy controller with good adaptation and robustness capability can be designed.     

A HYBRID DEMAND FORECASTING MODEL BASED ON EMPIRICAL MODE DECOMPOSITION AND NEURAL NETWORK IN TFT-LCD INDUSTRY

Demand forecasting plays an important role in the thin-film transistor liquid crystal display (TFT-LCD) industry. A hybrid approach is proposed for demand forecasting by combining empirical mode decomposition (EMD) and neural networks. From the signal analysis point of view, demand can be considered as a nonlinear and nonstationary combination of different frequencies. Every demand can be represented by one or several frequencies. The process of the proposed approach first decomposes the historical demand data into a finite set of intrinsic mode functions (IMFs) and a residual through EMD. Then, these IMFs are input into a back-propagation neural network (BPN) and the corresponding demand is used to predict these IMFs. Finally, the demand is forecasted by summing the predicted IMFs. The results show that the proposed model outperforms the single BPN model without EMD preprocessing and the traditional autoregressive integrated moving average (ARIMA) models.     

Neural network models for group behavior prediction: a case of soccer match attendance

Soccer match attendance is an example of group behavior with noisy context that can only be approximated by a limited set of quantifiable factors. However, match attendance is representative of a wider spectrum of context-based behaviors for which only the aggregate effect of otherwise individual decisions is observable. Modeling of such behaviors is desirable from the perspective of economics, psychology, and other social studies with prospective use in simulators, games, product planning, and advertising. In this paper, we evaluate the efficiency of different neural network architectures as models of context in attendance behavior by comparing the achieved prediction accuracy of a multilayer perceptron (MLP), an Elman recurrent neural network (RNN), a time-lagged feedforward neural network (TLFN), and a radial basis function network (RBFN) against a multiple linear regression model, an autoregressive moving average model with exogenous inputs, and a naive cumulative mean model. We show that the MLP, TLFN, and RNN are superior to the RBFN and achieve comparable prediction accuracy on datasets of three teams from the English Football League Championship, which indicates weak importance of context transition modeled by the TLFN and the RNN. The experiments demonstrate that all neural network models outperform linear predictors by a significant margin. We show that neural models built on individual datasets achieve better performance than a generalized neural model constructed from pooled data. We analyze the input parameter influences extracted from trained networks and show that there is an agreement between nonlinear and linear measures about the most significant attributes.

     

基于CNN-BIGRU的中文文本情感分类模型

在当今商业领域,对网络评论的情感分类一直是一个比较热门的研究方向,而为了克服传统机器学习方法所构建分类器会产生较大计算开销,精度表现较差的缺点,提出一种基于深度学习模型中卷积神经网络(CNN)与循环神经网络(RNN)模型的情感分类方法。在以往的研究中,卷积神经网络往往被用来提取文本的局部特征信息,但却容易忽视文本的长距离特征,而RNN则往往被用来提取句子的长距离依赖信息,但容易陷入梯度爆炸问题。因此,结合卷积神经网络对于局部特征信息的良好提取能力与循环神经网络对于长距离依赖信息的记忆能力,构建了一个CNN-BIGRU混合模型,用以提取文本的局部特征以及文本的长距离特征。其中循环神经网络模型使用了双向GRU模型,以避免RNN模型的梯度爆炸与梯度消失问题。在谭松波的酒店评论数据集上的实验结果表明,利用该模型,实验分类的准确率比单独使用卷积神经网络模型最高提升了26.3%,比单独使用循环神经网络模型最高提升了7.9%,从而提高了对中文文本情感分类的精度,并减少了计算开销。     

基于神经网络集成学习算法的金融时间序列预测

本文在传统神经网络(NN)、循环神经网络(RNN)、长短时记忆网络(LSTM)与门控循环单元(GRU)等神经网络时间预测模型基础上, 进一步构建集成学习(EL)时间序列预测模型, 研究神经网络类模型、集成学习模型和传统时间序列模型在股票指数预测上的表现. 本文以16只A股和国际股票市场指数为样本, 比较模型在不同预测期间和不同国家和地区股票市场上的表现.本文主要结论如下: 第一, 神经网络类时间序列预测模型和神经网络集成学习时间序列预测模型在表现上显著稳健优于传统金融时间序列预测模型, 预测性能提高大约35%; 第二, 神经网络类模型和神经网络集成学习模型在中国和美国股票市场上的表现优于其他发达国家和地区的股票市场.     

Neural network-based adaptive multiuser detection schemes in SDMA–OFDM system for wireless application

Neural network applications in adaptive multiuser detection (MUD) schemes are suggested here in the context of space division multiple access–orthogonal frequency division multiplexing system. In this paper, various neural network (NN) models like feed forward network (FFN), recurrent neural network (RNN) and radial basis function network (RBFN) are adopted for MUD. MUD using NN models outperforms other existing schemes like genetic algorithm--assisted minimum bit error rate (MBER) and minimum mean square error MUDs in terms of BER performance and convergence speed. Among these NN models, the FNN MUD performs efficiently as RNN in full load scenario, where the number of users is equal to number of receiving antennas. In overload scenario, where the number of users is more than the number of receiving antennas, the FNN MUD performs better than RNN MUD. Further, the RBFN MUD provides a significant enhancement in performance over FNN and RNN MUDs under both overload and full load scenarios because of its better classification ability due to Gaussian nonlinearity. Extensive simulation analysis considering Stanford University Interim channel models applied for fixed wireless applications shows improvement in convergence speed and BER performance of the proposed NN-based MUD algorithms.     

Immunological Algorithm-based Neural Network Learning for Sales Forecasting

This study proposes a variation immunological system (VIS) algorithm with radial basis function neural network (RBFN) learning for function approximation and the exercise of industrial computer (IC) sales forecasting. The proposed VIS algorithm was applied to the RBFN to execute the learning process for adjusting the network parameters involved. To compare the performance of relevant algorithms, three benchmark problems were used to justify the results of the experiment. With better accuracy in forecasting, the trained RBFN can be practically utilized in the IC sales forecasting exercise to make predictions and could enhance business profit.     

联合自注意力和循环网络的图像标题生成

目前大多数图像标题生成模型都是由一个基于卷积神经网络(Convolutional Neural Network,CNN)的图像编码器和一个基于循环神经网络(Recurrent Neural Network,RNN)的标题解码器组成。其中图像编码器用于提取图像的视觉特征,标题解码器基于视觉特征通过注意力机制来生成标题。然而,使用基于注意力机制的RNN的问题在于,解码端虽然可以对图像特征和标题交互的部分进行注意力建模,但是却忽略了标题内部交互作用的自我注意。因此,针对图像标题生成任务,文中提出了一种能同时结合循环网络和自注意力网络优点的模型。该模型一方面能够通过自注意力模型在统一的注意力区域内同时捕获模态内和模态间的相互作用,另一方面又保持了循环网络固有的优点。在MSCOCO数据集上的实验结果表明,CIDEr值从1.135提高到了1.166,所提方法能够有效提升图像标题生成的性能。     

Robust feedforward and recurrent neural network based dynamic weighted combination models for software reliability prediction

Traditional parametric software reliability growth models (SRGMs) are based on some assumptions or distributions and none such single model can produce accurate prediction results in all circumstances. Non-parametric models like the artificial neural network (ANN) based models can predict software reliability based on only fault history data without any assumptions. In this paper, initially we propose a robust feedforward neural network (FFNN) based dynamic weighted combination model (PFFNNDWCM) for software reliability prediction. Four well-known traditional SRGMs are combined based on the dynamically evaluated weights determined by the learning algorithm of the proposed FFNN. Based on this proposed FFNN architecture, we also propose a robust recurrent neural network (RNN) based dynamic weighted combination model (PRNNDWCM) to predict the software reliability more justifiably. A real-coded genetic algorithm (GA) is proposed to train the ANNs. Predictability of the proposed models are compared with the existing ANN based software reliability models through three real software failure data sets. We also compare the performances of the proposed models with the models that can be developed by combining three or two of the four SRGMs. Comparative studies demonstrate that the PFFNNDWCM and PRNNDWCM present fairly accurate fitting and predictive capability than the other existing ANN based models. Numerical and graphical explanations show that PRNNDWCM is promising for software reliability prediction since its fitting and prediction error is much less relative to the PFFNNDWCM.     

SDAE-LSTM模型在金融时间序列预测中的应用

针对金融时间序列预测的复杂性和长期依赖性，提出了一种基于深度学习的LSTM神经网络预测模型。利用堆叠去噪自编码从金融时间序列的基本行情数据和技术指标中提取特征，将其作为LSTM神经网络的输入对金融时间序列进行预测；通过LSTM神经网络的长期依赖特性来提高金融时间序列的预测精度。利用股价指数数据，与传统的神经网络的预测结果进行比较，结果表明基于深度学习的LSTM神经网络具有比较高的预测精度。     

随机神经网络发展现状综述

随机神经网络 (RNN)在人工神经网络中是一类比较独特、出现较晚的神经网络 ,它的网络结构、学习算法、状态更新规则以及应用等方面都因此具有自身的特点 .作为仿生神经元数学模型 ,随机神经网络在联想记忆、图像处理、组合优化问题上都显示出较强的优势 .在阐述随机神经网络发展现状、网络特性以及广泛应用的同时 ,专门将RNN分别与Hopfield网络、模拟退火算法和Boltzmann机在组合优化问题上的应用进行了分析对比 ,指出RNN是解决旅行商 (TSP)等问题的有效途径