Neural network committee-based sensitivity analysis strategy for geotechnical engineering problems

Neural network usually acts as a “black box” in diverse fields to perform prediction, classification, and regression. Different from the conventional usages, neural network is herein attempted to handle factor sensitivity analysis in a geotechnical engineering system. After systematically investigating instability of employing single neural network in factor sensitivity analysis, a neural network committee (NNC)-based sensitivity analysis strategy is first algorithmically presented based on the particular mathematical ideas of weak law of large numbers in probability and optimization. Significantly, this study especially emphasizes the practical application of the NNC-based sensitivity analysis strategy to highlight the mechanism underlying in strata movement. The principal goal is to reveal the relationships among influential factors on strata movement through estimating the relative contribution of each explicative (input) variable on dependent (output) variables of strata movement. It is demonstrated that the NNC-based sensitivity analysis strategy rationally not only reveals the relative contribution of each explicative variable on dependent variables but also indicates the predictability of each dependent variable. In addition, an improved prediction model is resulted from integrating the sensitivity analysis results into neural network modeling, and it is capable of facilitating the convergence training of neural network model and advancing its prediction precision on strata movement angles. The above outcomes indicate that the NNC-based sensitivity analysis strategy provides a new paradigm of applying neural networks to deal with complex geotechnical engineering problems.     

Prediction of seismic slope stability through combination of particle swarm optimization and neural network

One of the main concerns in geotechnical engineering is slope stability prediction during the earthquake. In this study, two intelligent systems namely artificial neural network (ANN) and particle swarm optimization (PSO)–ANN models were developed to predict factor of safety (FOS) of homogeneous slopes. Geostudio program based on limit equilibrium method was utilized to obtain 699 FOS values with different conditions. The most influential factors on FOS such as slope height, gradient, cohesion, friction angle and peak ground acceleration were considered as model inputs in the present study. A series of sensitivity analyses were performed in modeling procedures of both intelligent systems. All 699 datasets were randomly selected to 5 different datasets based on training and testing. Considering some model performance indices, i.e., root mean square error, coefficient of determination (R ²) and value account for (VAF) and using simple ranking method, the best ANN and PSO–ANN models were selected. It was found that the PSO–ANN technique can predict FOS with higher performance capacities compared to ANN. R ² values of testing datasets equal to 0.915 and 0.986 for ANN and PSO–ANN techniques, respectively, suggest the superiority of the PSO–ANN technique.     

Forecasting of Software Reliability Using Neighborhood Fuzzy Particle Swarm Optimization Based Novel Neural Network

This paper proposes an artificial neural network (ANN) based software reliability model trained by novel particle swarm optimization (PSO) algorithm for enhanced forecasting of the reliability of software. The proposed ANN is developed considering the fault generation phenomenon during software testing with the fault complexity of different levels. We demonstrate the proposed model considering three types of faults residing in the software. We propose a neighborhood based fuzzy PSO algorithm for competent learning of the proposed ANN using software failure data. Fitting and prediction performances of the neighborhood fuzzy PSO based proposed neural network model are compared with the standard PSO based proposed neural network model and existing ANN based software reliability models in the literature through three real software failure data sets. We also compare the performance of the proposed PSO algorithm with the standard PSO algorithm through learning of the proposed ANN. Statistical analysis shows that the neighborhood fuzzy PSO based proposed neural network model has comparatively better fitting and predictive ability than the standard PSO based proposed neural network model and other ANN based software reliability models. Faster release of software is achievable by applying the proposed PSO based neural network model during the testing period.      

A systematic review and meta-analysis of artificial neural network application in geotechnical engineering: theory and applications

Artificial neural network (ANN) aimed to simulate the behavior of the nervous system as well as the human brain. Neural network models are mathematical computing systems inspired by the biological neural network in which try to constitute animal brains. ANNs recently extended, presented, and applied by many research scholars in the area of geotechnical engineering. After a comprehensive review of the published studies, there is a shortage of classification of study and research regarding systematic literature review about these approaches. A review of the literature reveals that artificial neural networks is well established in modeling retaining walls deflection, excavation, soil behavior, earth retaining structures, site characterization, pile bearing capacity (both skin friction and end-bearing) prediction, settlement of structures, liquefaction assessment, slope stability, landslide susceptibility mapping, and classification of soils. Therefore, the present study aimed to provide a systematic review of methodologies and applications with recent ANN developments in the subject of geotechnical engineering. Regarding this, a major database of the web of science has been selected. Furthermore, meta-analysis and systematic method which called PRISMA has been used. In this regard, the selected papers were classified according to the technique and method used, the year of publication, the authors, journals and conference names, research objectives, results and findings, and lastly solution and modeling. The outcome of the presented review will contribute to the knowledge of civil and/or geotechnical designers/practitioners in managing information in order to solve most types of geotechnical engineering problems. The methods discussed here help the geotechnical practitioner to be familiar with the limitations and strengths of ANN compared with alternative conventional mathematical modeling methods.

     

基于粒子群优化的灰色神经网络组合预测模型研究

灰色神经网络在人工智能预测领域已经得到广泛的应用,但由于其自身存在局部最小化和收敛速度慢等问题,使其预测精度受到一定的限制。针对其不足,本文提出一种利用粒子群算法优化BP神经网络的学习算法,在此基础上,利用灰色预测方法对股指期货历史数据进行初步预测,并且把初步预测的结果作为优化BP神经网络的输入进行训练和预测,构建了基于粒子群优化的灰色神经网络组合预测模型(PSO-GMNN)。仿真实验结果表明,新预测模型的预测精度高于BP神经网络、灰色神经网络和灰色预测模型,同时也表明了该方法的有效性和可行性。     

Predictive accuracy comparison between neural networks and statistical regression for development effort of software projects

To get a better prediction of costs, schedule, and the risks of a software project, it is necessary to have a more accurate prediction of its development effort. Among the main prediction techniques are those based on mathematical models, such as statistical regressions or machine learning (ML). The ML models applied to predicting the development effort have mainly based their conclusions on the following weaknesses: (1) using an accuracy criterion which leads to asymmetry, (2) applying a validation method that causes a conclusion instability by randomly selecting the samples for training and testing the models, (3) omitting the explanation of how the parameters for the neural networks were determined, (4) generating conclusions from models that were not trained and tested from mutually exclusive data sets, (5) omitting an analysis of the dependence, variance and normality of data for selecting the suitable statistical test for comparing the accuracies among models, and (6) reporting results without showing a statistically significant difference. In this study, these six issues are addressed when comparing the prediction accuracy of a radial Basis Function Neural Network (RBFNN) with that of a regression statistical (the model most frequently compared with ML models), to feedforward multilayer perceptron (MLP, the most commonly used in the effort prediction of software projects), and to general regression neural network (GRNN, a RBFNN variant). The hypothesis tested is the following: the accuracy of effort prediction for RBFNN is statistically better than the accuracy obtained from a simple linear regression (SLR), MLP and GRNN when adjusted function points data, obtained from software projects, is used as the independent variable. Samples obtained from the International Software Benchmarking Standards Group (ISBSG) Release 11 related to new and enhanced projects were used. The models were trained and tested from a leave-one-out cross-validation method. The criteria for evaluating the models were based on Absolute Residuals and by a Friedman statistical test. The results showed that there was a statistically significant difference in the accuracy among the four models for new projects, but not for enhanced projects. Regarding new projects, the accuracy for RBFNN was better than for a SLR at the 99% confidence level, whereas the MLP and GRNN were better than for a SLR at the 90% confidence level.     

神经网络在软件失效预测中的性能比较和研究

在软件开发的早期预测有失效倾向的软件模块,能够极大地提高软件的质量.软件失效预测中的一个普遍问题是数据中噪声的存在.神经网络具有鲁棒性而且对噪声有很强的抑制能力.不同结构的神经网络在训练算法和应用领域都有差异.该文主要就软件失效预测这个应用领域叙述几种适用的网络,并比较这几种网络在训练结果和性能上的差异.上述方法在SDH通信软件的失效预测中得到了成功的应用.试验结果显示虽然MLP、PNN、LVQ网络都能解决这类模式分类问题,但是只有MLP网络训练结果比较稳定,在不同的数据集上训练出的网络都有很好的预测效果.     

QPSO算法优化BP网络的网络流量预测

网络流量预测对于大规模网络的规划设计和网络资源管理等方面都具有积极的意义,是网络流量工程重要组成部分。结合QPSO算法和BP神经网络的优势,采用QPSO算法对BP神经网络的权值和阈值进行优化,并利用历史记录训练BP网络。仿真实验表明,与PSO训练的BP网络以及直接用BP网络进行预测的模型相比,基于QPSO训练的BP网络流量预测模型具有更好的预测能力。     

Extraction of voltage harmonics using multi-layer perceptron neural network

This paper presents a harmonic extraction algorithm using artificial neural networks for Dynamic Voltage Restorers (DVRs). The suggested algorithm employs a feed forward Multi Layer Perceptron (MLP) Neural Network with error back propagation learning to effectively track and extract the 3rd and 5th voltage harmonics. For this purpose, two different MLP neural network structures are constructed and their performances compared. The effects of hidden layer, supervisors and learning rate are also presented. The proposed MLP Neural Network algorithm is trained and tested in MATLAB program environment. The results show that MLP neural network enable to extract each harmonic effectively.     

基于PSO-DE-BP的光伏发电功率短期预测

提高光伏发电功率预测精度对保障智能电网安全稳定运行有重要意义;针对传统BP神经网络存在预测精度不高且收敛速度慢的弊端,提出一种基于粒子群(PSO)差分进化(DE)并行计算优化BP神经网络的光伏发电短期预测方法;首先分析影响因素重要程度,通过带权重的欧式距离筛选相似的训练样本集;其次,对粒子群分组,通过粒子群和差分进化混合算法对粒子组内和组间优化,以保证种群多样性、提高预测稳定和精度、避免局部最优;然后,建立预测模型,通过基于spark的内存计算平台,将PSO-DE-BP算法并行优化以提高算法运行效率;最后,根据不同天气类型的预测结果对模型进行分析验证,此方法比PSO-BP、BP算法模型具有更高的稳定性和预测精度。     

基于改进的粒子群算法优化反向传播神经网络的热舒适度预测模型

针对热舒适度预测是一个复杂的非线性过程,不便于空调的实时控制应用的问题,提出一种基于改进的粒子群优化(PSO)算法优化反向传播(BP)神经网络的热舒适度预测模型。这一预测模型通过采用PSO算法优化BP神经网络的初始权值和阈值,改善了传统BP算法收敛速度慢及对网络初始值敏感的问题。同时,针对标准PSO算法易出现早熟收敛、局部寻优能力弱等缺点,提出了相应改进策略,进一步提高了PSO优化BP神经网络的能力。实验结果表明：与传统BP模型和标准PSO-BP模型相比,基于改进的PSO-BP算法的热舒适度预测模型具有更高的预测精度和更快的收敛速度。     

An on-line wastewater quality predication system based on a time-delay neural network

The biological treatment process in a wastewater treatment system is a very complex process. The efficiency of the treatment is usually measured by laboratory tests, which typically take five days. In this paper, a time-delay neural network (TDNN) modeling method is proposed for predicting the treatment results. As the first step, a sensitivity analysis performed on a multi-layer perceptron (MLP) network model is used to reduce the input dimensions of the model. Then a TDNN model is further used to improve the performance of the original MLP network model. Subsequently, an on-line prediction and model-updating strategy is proposed and implemented. Simulations using industrial process data show that the prediction accuracy can be improved by the on-line model updating.     

Achieving a More Robust Neural Network Model for Control of a MR Damper by Signal Sensitivity Analysis

Most neural network models can work accurately on their trained samples, but when encountering noise, there could be significant errors if the trained neural network is not robust enough to resist the noise. Sensitivity to perturbation in the control signal due to noise is very important for the prediction of an output signal. The goal of this paper is to provide a methodology of signal sensitivity analysis in order to enable the selection of an ideal Multi-Layer Perception (MLP) neural network model from a group of MLP models with different parameters, i.e. to get a highly accurate and robust model for control problems. This paper proposes a signal sensitivity which depends upon the variance of the output error due to noise in the input signals of a single output MLP with differentiable activation functions. On the assumption that noise arises from additive/multiplicative perturbations, the signal sensitivity of the MLP model can be easily calculated, and a method of lowering the sensitivity of the MLP model is proposed. A control system of a magnetorheological (MR) fluid damper, which is a relatively new type of device that shows the future promise for the control of vibration, is modelled by MLP. A large number of simulations on the MR damper’s MLP model show that a much better model is selected using the proposed method.     

粒子群算法优化BP神经网络的粉尘浓度预测

对综采工作面粉尘浓度预测的方法是建立BP神经网络预测模型.为了提高算法的拟合能力及预测的准确度,使用粒子群算法对目标函数进行改进,即将粒子群算法寻到的最优权值和阈值应用于神经网络预测模型求综采工作面粉尘浓度.比较分析新的预测模型与常用的灰色模型以及标准的BP神经网络算法,结果表明粒子群优化的神经网络算法的拟合能力和预测的准确率显著提高.     

A non-symbolic implementation of abdominal pain estimation in childhood

The abdominal pain is a very common disease in childhood, which lurks complications. Pediatric surgeons have to estimate at least 15 clinical and laboratory factors in order to make a diagnosis and decide about performing a surgical operation of the abdomen. Artificial Neural Networks (ANNs) are particular implementations of Artificial Intelligence (AI) systems and they are used in a wide area of application fields. This study examines the implementation of ANN architectures, using Multi-Layer Perceptron (MLP) neural networks and Probabilistic Neural Networks (PNN) architectures, in order to specify the appropriate ANN structure for abdominal pain estimation in childhood. The architecture with the best performance is a fully interconnected MLP neural network with an input layer of 15 nodes, one hidden layer of 5 neurons and an output layer, with error back-propagation algorithm being used as the learning scheme. In the output layer, the estimation of appendicitis’ stage is reached automatically. The proposed ANN achieved a percentage of 88.5% of correct classification on testing set cases. Further analysis of obtained results, exhibited the ability of ANN for distinguishing the necessity of a case for operative treatment of abdominal pain based on diagnostic features, attaining a percentage of 100% of successful prognosis over the cases of testing set. The aim of proposed MLP neural network is to assist surgeons in appendicitis prediction, avoiding an unnecessary operative treatment.     

基于扩展T-S模型的PSO神经网络在故障诊断中的应用

针对现实故障现象具有模糊性和非线性的特点,提出了一种利用自适应扩展T-S(Takagi-Sugeno)模糊模型的PSO(Particle Swarm Optimization)算法和神经网络相结合的新型智能结构化算法来进行故障诊断的新方法.首先通过自适应的高斯函数来更改基本T-S模糊模型中的隶属度函数,进而使用扩展的T-S模糊模型来调整PSO算法的参数.然后使用该PSO算法作为神经网络的学习训练算法来进行训练.最后将此算法用于齿轮箱实测故障诊断.诊断结果显示均方误差提高了0.1981%.通过不同模型的诊断结果比较,表明本方法便捷、高效,为解决故障诊断问题提供了一条新途径.     

基于蚁群粒子群的模糊神经网络交通流量预测

实时准确的交通流量预测是智能交通诱导和交通控制实现的前提和关键。针对城市交通流的特点,建立了模糊神经网络预测模型,并将全局优化的蚁群算法和粒子群算法组成递阶结构优化模糊神经网络的参数。算法中,主级为蚁群算法,进行全局搜索;从级为粒子群算法,进行局部搜索。仿真结果表明该模型能够取得比梯度下降法更高的预测精度。     

结合信息融合和BP神经网络的决策算法

针对网络输入信息复杂多变,固定的 BP(Back-Propagation)网络结构难以发挥其优势的情况,提出了结合信息融合和BP神经网络的决策算法。即根据输入的变化情况,利用D-S证据理论(Dempster-Shafer,D-S)对BP神经网络的结构进行优选。同时使用粒子群(PSO, Particle Swarm Optimization)算法来确定BP神经网络的初值,以改善其收敛速度慢和容易陷入局部极小值的问题。仿真结果显示,结合信息融合和 BP 神经网络的决策算法和BP神经网络相比,有效提高了BP神经网络训练的时间及预测的准确率,在适应复杂多变的输入信息时具有一定的优势。     

Time series prediction using PSO-optimized neural network and hybrid feature selection algorithm for IEEE load data

Artificial neural networks have been widely used in time series prediction. In this paper, multi-layer feedforward neural networks with optimized structures, using particle swarm optimization (PSO) algorithm, are used for hourly load prediction based on load time series of IEEE Reliability Test System. To have a small and appropriate feature subset, a hybrid method is used for feature selection in this paper. This hybrid method uses the combination of genetic algorithm (GA) and ant colony optimization (ACO) algorithm. The season, day of the week, time of the day and history load are considered as load influencing factors in this study based on the mentioned standard load dataset. The optimized number of neurons in the hidden layers of multi-layer perceptron (MLP) is determined using PSO algorithm. Experimental results show that the proposed hybrid model offers superior performance, in terms of mean absolute percentage error (MAPE), in time series prediction as compared to some recent researches in this field.     

基于MLP神经网络的分组密码算法能量分析研究

随着嵌入式密码设备的广泛应用,侧信道分析(side channel analysis,SCA)成为其安全威胁之一。通过对密码算法物理实现过程中的泄露信息进行分析实现密钥恢复,进而对密码算法实现的安全性进行评估。为了精简用于能量分析的多层感知器(multi-layer perceptron,MLP)网络结构,减少模型的训练参数和训练时间,针对基于汉明重量(HW)和基于比特的MLP神经网络的模型进行了研究,输出类别由256分类分别减少为9分类和2分类;通过采集AES密码算法运行过程中的能量曲线对所提出的MLP神经网络进行训练和测试。实验结果表明,该模型在确保预测精度的前提下能减少MLP神经网络84%的训练参数和28%的训练时间,并减少了密钥恢复阶段需要的能量曲线数量,最少只需要一条能量曲线即可完成AES算法完整密钥的恢复。实验验证了模型的有效性,使用该模型可以对分组密码算法实现的安全性进行分析和评估。