基于并行神经网络集成的多步预测方法

神经网络集成通过训练多个神经网络并将其结论进行合成,可以显著地提高学习系统的泛化能力。该文提出了一种基于特征提取的并行神经网络集成多步预测模型ECPNN(ExtractionofCharacteristicsParallelNeuralNetwork)。从单因素时间序列中提取出代表内在机制的特征,采取并行TDNN(Time-delayNeuralNetwork)集成的方式实现时间序列多步预测。实验结果表明了该模型在多步预测方面的可行性和有效性。     

Normal and hypoacoustic infant cry signal classification using time-frequency analysis and general regression neural network

Crying is the most noticeable behavior of infancy. Infant cry signals can be used to identify physical or psychological status of an infant. Recently, acoustic analysis of infant cry signal has shown promising results and it has been proven to be an excellent tool to investigate the pathological status of an infant. This paper proposes short-time Fourier transform (STFT) based time-frequency analysis of infant cry signals. Few statistical features are derived from the time-frequency plot of infant cry signals and used as features to quantify infant cry signals. General Regression Neural Network (GRNN) is employed as a classifier for discriminating infant cry signals. Two classes of infant cry signals are considered such as normal cry signals and pathological cry signals from deaf infants. To prove the reliability of the proposed features, two neural network models such as Multilayer Perceptron (MLP) and Time-Delay Neural Network (TDNN) trained by scaled conjugate gradient algorithm are also used as classifiers. The experimental results show that the GRNN classifier gives very promising classification accuracy compared to MLP and TDNN and the proposed method can effectively classify normal and pathological infant cries.     

A hybrid approach based on neural networks and genetic algorithms for detecting temporal patterns in stock markets

This study investigates the effectiveness of a hybrid approach based on the artificial neural networks (ANNs) for time series properties, such as the adaptive time delay neural networks (ATNNs) and the time delay neural networks (TDNNs), with the genetic algorithms (GAs) in detecting temporal patterns for stock market prediction tasks. Since ATNN and TDNN use time-delayed links of the network into a multi-layer feed-forward network, the topology of which grows by on layer at every time step, it has one more estimate of the number of time delays in addition to several control variables of the ANN design. To estimate these many aspects of the ATNN and TDNN design, a general method based on trial and error along with various heuristics or statistical techniques is proposed. However, for the reason that determining the number of time delays or network architectural factors in a stand-alone mode does not guarantee the illuminating improvement of the performance for building the ATNN and TDNN model, we apply GAs to support optimization of the number of time delays and network architectural factors simultaneously for the ATNN and TDNN model. The results show that the accuracy of the integrated approach proposed for this study is higher than that of the standard ATNN, TDNN and the recurrent neural network (RNN).     

A new approach to predict normalized difference vegetation index using time-delay neural network in the arid and semi-arid grassland

ABSTRACT

Normalized difference vegetation index (NDVI) has been used to conduct important research on plant growth and vegetation productivity. In this paper, a new approach to predict NDVI based on precipitation in the grass-growing season for the arid and semi-arid grassland is proposed using time-delay neural network (TDNN). To intuitively know the ability of TDNN to learn the relationship between NDVI and precipitation and to predict NDVI, the performance of the TDNN model is compared with back propagation neural network (BPNN) trained with the same data. The results indicate that TDNN works well to predict precipitation. Moreover, the relationship between precipitation and NDVI can be predicted accurately by the proposed TDNN model. The results show the goodness-of-fit between the observed NDVI and predicted NDVI measured by the determination coefficient of R² being 0.999 from the TDNN model, with the mean absolute percentage error, mean absolute error, and root-mean-square error to be 0.23%, 0.20, and 0.19, respectively. The analysis shows that the proposed method can result in an accurate NDVI prediction. Thus, the algorithm is applied to predict the NDVI during the grass-growing season for the validation of the approach. This validation results suggest the potential application of this approach for reduction of overgrazing pressure and vegetation restoration in the arid and semi-arid grassland.     

Hybrid multilayered perceptron networks

This paper introduces a modified multilayered perception network (MLP) called the Hybrid Multilayered Perceptron (HMLP) network to improve the performance of a MLP network. The convergence rate of the proposed network is further improved by proposing a modified version of the recursive prediction error algorithm as the training algorithm. The capability of the proposed network architecture trained using the modified recursive prediction error algorithm was demonstrated using simulated and real data sets. The results indicated that the proposed network provides a significant improvement over a standard MLP network. These additional linear input connections do not significantly increase the complexity of the MLP network since the connections are linear. In fact, by using the linear input connections, the number of hidden nodes required by the standard MLP network model can be reduced, which will also reduce computational load. The performance of the HMLP network was also compared with Radial Basis Function (RBF) and Hybrid Radial Basis Function (HRBF) networks. It was found that the proposed HMLP network was much more efficient than both RBF and HRBF networks.     

Time-delay neural networks for time series prediction: an application to the monthly wholesale price of oilseeds in India

Agricultural price forecasting is one of the challenging areas of time series forecasting. The feed-forward time-delay neural network (TDNN) is one of the promising and potential methods for time series prediction. However, empirical evaluations of TDNN with autoregressive integrated moving average (ARIMA) model often yield mixed results in terms of the superiority in forecasting performance. In this paper, the price forecasting capabilities of TDNN model, which can model nonlinear relationship, are compared with ARIMA model using monthly wholesale price series of oilseed crops traded in different markets in India. Most earlier studies of forecast accuracy for TDNN versus ARIMA do not consider pretesting for nonlinearity. This study shows that the nonlinearity test of price series provides reliable guide to post-sample forecast accuracy for neural network model. The TDNN model in general provides better forecast accuracy in terms of conventional root mean square error values as compared to ARIMA model for nonlinear patterns. The study also reveals that the neural network models have clear advantage over linear models for predicting the direction of monthly price change for different series. Such direction of change forecasts is particularly important in economics for capturing the business cycle movements relating to the turning points.     

基于MLP改进型深度神经网络学习资源推荐算法

针对在线学习过程中出现的知识过载及传统推荐算法中存在的数据稀疏和冷启动问题,提出了一种基于多层感知机（MLP）的改进型深度神经网络学习资源推荐算法。该算法利用多层感知机对非线性数据处理的优势,将学习者特征和学习资源特征进行向量相乘的预测方式转换为输入多层感知机的方式,改进了DN-CBR神经网络推荐模型。为验证模型的有效性,以爱课程在线学习平台数据为样本构建数据集,通过对比实验表明,在该数据集上,改进后模型相较于DN-CBR模型在归一化折损累积增益和命中率指标上分别提升了1.2%和3%,有效地提高了模型的推荐性能。     

Short-term Maharashtra state electrical power load prediction with special emphasis on seasonal changes using a novel focused time lagged recurrent neural network based on time delay neural network model

In this paper, the parameter-wise optimization training process is implemented to achieve an optimal configuration of focused time lagged recurrent neural network (FTLRNN) models by embedding the gamma, laguarre, and multi-channel tapped delay line memory structure. The aim is to examine the prediction ability of the proposed models in order to predict one-day-ahead electric power load simultaneously as usual to oppose 1–24 h forecast in sequel with a special emphasis on seasonal changes over a year. An improved delta-bar-delta algorithm is used to accelerate the training of neural networks and to improve the stability of the convergence.Experimental results indicate that the FTLRNN with time delay neural network (TDNN) clearly outperformed the gamma and laguarre based short-term memory structure in various performance metrics such as mean square error (MSE), normalized MSE, correlation coefficient (r) and mean absolute percentage error (MAPE) during evaluation process. Empirical results show that the proposed dynamic NN model consistently performs well on daily, weekly, and monthly average basis in terms of prediction accuracy. It is noticed from the literature review that an optimally configured FTLRNN with multi-channel tapped delay line memory structure is not currently available to solve short-term electrical power load prediction. The proposed method gives acceptable errors in all seasons, months and on daily basis. The average prediction error on three weeks is obtained as low as 1.67%.     

Achieving a More Robust Neural Network Model for Control of a MR Damper by Signal Sensitivity Analysis

Most neural network models can work accurately on their trained samples, but when encountering noise, there could be significant errors if the trained neural network is not robust enough to resist the noise. Sensitivity to perturbation in the control signal due to noise is very important for the prediction of an output signal. The goal of this paper is to provide a methodology of signal sensitivity analysis in order to enable the selection of an ideal Multi-Layer Perception (MLP) neural network model from a group of MLP models with different parameters, i.e. to get a highly accurate and robust model for control problems. This paper proposes a signal sensitivity which depends upon the variance of the output error due to noise in the input signals of a single output MLP with differentiable activation functions. On the assumption that noise arises from additive/multiplicative perturbations, the signal sensitivity of the MLP model can be easily calculated, and a method of lowering the sensitivity of the MLP model is proposed. A control system of a magnetorheological (MR) fluid damper, which is a relatively new type of device that shows the future promise for the control of vibration, is modelled by MLP. A large number of simulations on the MR damper’s MLP model show that a much better model is selected using the proposed method.     

基于MLP神经网络的分组密码算法能量分析研究

随着嵌入式密码设备的广泛应用,侧信道分析(side channel analysis,SCA)成为其安全威胁之一。通过对密码算法物理实现过程中的泄露信息进行分析实现密钥恢复,进而对密码算法实现的安全性进行评估。为了精简用于能量分析的多层感知器(multi-layer perceptron,MLP)网络结构,减少模型的训练参数和训练时间,针对基于汉明重量(HW)和基于比特的MLP神经网络的模型进行了研究,输出类别由256分类分别减少为9分类和2分类;通过采集AES密码算法运行过程中的能量曲线对所提出的MLP神经网络进行训练和测试。实验结果表明,该模型在确保预测精度的前提下能减少MLP神经网络84%的训练参数和28%的训练时间,并减少了密钥恢复阶段需要的能量曲线数量,最少只需要一条能量曲线即可完成AES算法完整密钥的恢复。实验验证了模型的有效性,使用该模型可以对分组密码算法实现的安全性进行分析和评估。     

Prediction of parameters characterizing the state of a pollution removal biologic process

This work is devoted to the prediction, based on neural networks, of physicochemical parameters impossible to measure on-line. These parameters—the chemical oxygen demand (COD) and the ammonia NH₄—characterize the organic matter and nitrogen removal biological process carried out at the Saint Cyprien WWTP (France). Their knowledge make it possible to estimate the process quality and efficiency. First, the data are treated by K-Means clustering then by principal components analysis (PCA) in order to optimize the multi-level perceptron (MLP) learning phase. K-Means clustering makes it possible to highlight different operations within the Saint Cyprien treatment plant. The PCA is used to eliminate redundancies and synthesizes the information expressed by a data set. With respect to the neural network used, these techniques facilitate the pollution removal process understanding and the identification of existing relations between the predictive variables and the variables to be predicted.     

Prediction of soil compression coefficient for urban housing project using novel integration machine learning approach of swarm intelligence and Multi-layer Perceptron Neural Network

In many engineering projects, the soil compression coefficient is an important parameter used for estimating the settlement of soil layers. The common practice of determining the soil compression coefficient via the oedometer test is time-consuming and expensive. This study proposes a machine learning solution to replace the conventional tests used for obtaining the coefficient of soil compression. The new approach is an integration of the Multi-Layer Perceptron Neural Network (MLP Neural Nets) and Particle Swarm Optimization (PSO). These two computational intelligence methods work synergistically to establish a prediction model of soil compression coefficient. The PSO metaheuristic is employed to optimize the MLP Neural Nets model structure. To train and validate the proposed method, named as PSO-MLP Neural Nets, a dataset of 154 soil samples featuring 12 influencing factors has been collected from the geotechnical investigation process of a high-rise building project. Experimental results show that the proposed PSO-MLP Neural Nets has attained the most accurate prediction of the soil compression coefficient performance with RMSE = 0.0267, MAE = 0.0145, and R² = 0.884. The result of the proposed model is significantly better than those obtained from other benchmark methods including the backpropagation neural network, the radial basis function neural network, the support vector regression, the random forest, and the Gaussian process. Based on the experimental results, the newly constructed PSO-MLP Neural Nets is very potential to be a new alternative to assist geotechnical engineers in design phase of civil engineering projects.     

MLP neural network models of CMM measuring processes

The objective of this study is to show how a multi-layer perceptron (MLP) neural network can be used to model a CMM measuring process. To date, most MLP-based process models have been established for process mean only. An innovative approach is proposed to model simultaneously the mean and the variation of a CMM process using one integrated MLP architecture. Therefore, the MLP-based model obtained captures not only the process mean but also the process variation information. Selected issues related to neural network training are also discussed. Specifically, the guideline that was proposed by Mirchandani and Cao (1989) for selecting a number of hidden neurons is tested to determine the effects of the number of hidden neurons. The performances of two different learning algorithms - back-propagation with momentum factor (BPM) and the Fletcher-Reeves (FR) algorithm - are studied in terms of CPU time, training error, and generalization error.     

基于混合模型CHMM和MLP的数码语音识别系统

针对传统的CHMM应用于语音识别系统存在的缺点,提出了一种由CHMM和MLP网构成的混合模型。该混合模型将MLP网引入到CHMM中来计算每个状态的输出概率,通过MLP网的非线性预测能力代替CHMM中的似然估计值对输出概率进行分析、分类,从而加强和提高CHMM的语音识别能力。实验结果表明,将该混合模型应用到语音识别系统中,其识别效果明显优于基于传统的CHMM的识别系统。     

Early prediction of cooling load in energy-efficient buildings through novel optimizer of shuffled complex evolution

The crucial significance of proper management of heating, ventilating, and air conditioning systems in energy-efficient buildings were the main reason for dedicating this study to test a novel approach for this task. Shuffled complex evolution (SCE) is an efficient metaheuristic technique that is used to optimize the performance of a multi-layer perceptron neural network (MLP) for accurate prediction of cooling load (CL). The CL information of 768 residential buildings, obtained from a vast computer simulation in the published literature, is used to train and validate the performance of the proposed model. The results showed that the SCE could properly surmount the computational drawbacks of the MLP, as its learning and prediction accuracies are enhanced by 19.52 and 22.84%, respectively. Also, the SCE outperformed two benchmark optimizers of moth–flame optimization and optics inspired optimization in both training and testing phases. Another advantage of the tested SCE-MLP was the considerably simpler structure, and consequently, shorter computation time (722 vs. 1050 and 46,192 s). Therefore, the proposed model can be promisingly used in practice for the early prediction of CL in energy-efficient buildings.

     

基于OWO-HWO算法的MLP网络在织物染色配色中的应用

讨论了具有非线性、不确定特性的织物染色配色过程建模与仿真问题。针对传统的织物染色配色方法效果差、精确度不高和难以达到期望结果的问题,结合MLP神经网络的特点,提出了基于OWO-HWO算法训练的MLP神经网络,同时分别优化网络输入层到隐层和隐层到输出层的权值,并利用基于OWO-HWO算法的MLP神经网络建立织物染色配色模型。针对此种模型,利用NuMap神经网络软件进行仿真实验。仿真结果表明,该配色模型收敛速度快,精确度高,在解决织物染色配色问题上取得了令人满意的配色效果。     

Time-delay recurrent neural network for temporal correlations and prediction

This paper presents a time-delay recurrent neural network (TDRNN) for temporal correlations and prediction. The TDRNN employs adaptive time delays and recurrences where the adaptive time delays make the network choose the optimal values of time delays for the temporal location of the important information in the input sequence and the recurrences enable the network to encode and integrate temporal context information of sequences. The TDRNN and multiple recurrent neural network(MRNN) described in this paper, adaptive time-delay neural network (ATNN) proposed by Lin, and time-delay neural network (TDNN) introduced by Waibel were simulated and applied to the chaotic time series prediction of Mackey–Glass delay-differential equation and the Korean stock market index prediction. The simulation results suggest that employing time delayed recurrences in the layered network is more effective for temporal correlations and prediction than putting multiple time delays into the neurons or their connections. The best performance is attained by the TDRNN. The TDRNN will be well applicable for temporal signal recognition, prediction and identification.     

Enhancing MLP networks using a distributed data representation

Multilayer perceptron (MLP) networks trained using backpropagation can be slow to converge in many instances. The primary reason for slow learning is the global nature of backpropagation. Another reason is the fact that a neuron in an MLP network functions as a hyperplane separator and is therefore inefficient when applied to classification problems in which decision boundaries are nonlinear. This paper presents a data representational approach that addresses these problems while operating within the framework of the familiar backpropagation model. We examine the use of receptors with overlapping receptive fields as a preprocessing technique for encoding inputs to MLP networks. The proposed data representation scheme, termed ensemble encoding, is shown to promote local learning and to provide enhanced nonlinear separability. Simulation results for well known problems in classification and time-series prediction indicate that the use of ensemble encoding can significantly reduce the time required to train MLP networks. Since the choice of representation for input data is independent of the learning algorithm and the functional form employed in the MLP model, nonlinear preprocessing of network inputs may be an attractive alternative for many MLP network applications.     

Performance evaluation of multilayer perceptron,radial basis function,fuzzy inference system,and an adaptively tuned fuzzy wavelet neural network in parameter prediction of multiphase flow measurement instrumentation

Application of predictive models in industrial multiphase flow metering has attracted an increasing attention recently. Void fraction (VF), water–liquid ratio (WLR), and flow regime are key parameters, measured by oil/water/gas multiphase flow metres (MPFM) in petroleum industry. Artificial neural networks and fuzzy inference systems (FIS) are reliable and efficient computational models, which can be simply implemented on microprocessors of MPFMs, having the advantages of trainability, adaptability, and capability to model non‐linear functions. In this paper, a wavelet‐based adaptive neuro‐FIS (WANFIS) is introduced and validated by the prediction of multiphase flow measurement critical parameters including flow regime, VF, and WLR. The performance of the proposed WANFIS model is then compared with multilayer perceptron (MLP), radial basis function (RBF) network, and an FIS trained by fuzzy c‐means and a subtractive clustering method in the prediction of flow parameters in a customized designed structure of oil/water/gas MPFM. Structural parameters of all predictive models are first optimized to yield the most efficient structure for the available dataset. Comparison is then made between the optimized predictive models, in terms of mean squared error of parameter prediction, computation time, and repeatability of the prediction process. According to the obtained results, MLP model using Levenberg–Marquardt training algorithm and WANFIS model using gradient‐based back propagation dynamical iterative learning algorithm are the most efficient models, which give the best performance compared with other used models. All predictive models can predict the flow regime with 100% accuracy, whereas the highest inaccuracy is related to the prediction of WLR. The results of this study can be used to select and develop the most appropriate predictive model applicable in predicting and identifying flow measurement parameters in industrial MPFMs.     

基于PPI网络与机器学习的蛋白质功能预测方法

针对现有的基于蛋白质相互作用（PPI）网络的蛋白质功能预测方法预测精度不高、易受数据噪声影响的问题,提出一种基于机器学习（层次聚类、主成分分析和多层感知器）的蛋白质功能预测方法HPMM。该方法综合考虑蛋白质宏观和微观层面的信息,将蛋白质家族、结构域和重要位点信息作为顶点属性整合到PPI网络中以减轻网络中数据噪声的影响。首先,基于层次聚类和主成分分析进行特征提取,得到功能模块和属性主成分特征,然后训练多层感知器模型,建立多特征与多功能之间的映射关系以用于功能预测。在三个分别被分子功能（MF）、生物过程（BP）和细胞组件（CC）注释的人类PPI网络上进行测试,对HPMM、余弦迭代算法（CIA）和有向PPI网络基因本体术语传播（GoDIN）算法的功能预测效果进行比较分析。实验结果表明,相比CIA和GoDIN这两种完全基于PPI网络的方法,HPMM的精确度与F值更高。