Range estimation of construction costs using neural networks with bootstrap prediction intervals

Modeling of construction costs is a challenging task, as it requires representation of complex relations between factors and project costs with sparse and noisy data. In this paper, neural networks with bootstrap prediction intervals are presented for range estimation of construction costs. In the integrated approach, neural networks are used for modeling the mapping function between the factors and costs, and bootstrap method is used to quantify the level of variability included in the estimated costs. The integrated method is applied to range estimation of building projects. Two techniques; elimination of the input variables, and Bayesian regularization were implemented to improve generalization capabilities of the neural network models. The proposed modeling approach enables identification of parsimonious mapping function between the factors and cost and, provides a tool to quantify the prediction variability of the neural network models. Hence, the integrated approach presents a robust and pragmatic alternative for conceptual estimation of costs.     

基于小波非线性自回归网络的水文预测模型

不同时间尺度上的水文序列预测在水资源调配和防洪减灾决策中起着重要的作用。提出了一种基于小波分解和非线性自回归神经网络相结合的水文时间序列预测模型(WNARN)。运用Daubechies 5(db5)离散小波将水文序列数据分解为低频和高频子序列,作为非线性自回归神经网络模型(NARN)的输入变量,贝叶斯正则化优化算法用来泛化网络,训练模型对各子序列进行模拟预测,预测值经db5小波重构后得到原序列预测值。利用渭河流域三个水文站40多年的月径流量序列对所提出的WNARN模型进行验证和向前48步的预测能力测试,并与单一NARN模型的验证和预测结果进行对比。结果显示在相同的网络结构下所提出的方法能够显著提高水文序列的预测精度、预测周期及对重大水文事件的预测性,具有较高的泛化能力。     

Regularization networks for inverse problems: A state-space approach

Linear inverse problems with discrete data are equivalent to the estimation of the continuous-time input of a linear dynamical system from samples of its output. The solution obtained by means of regularization theory has the structure of a neural network similar to classical RBF networks. However, the basis functions depend in a nontrivial way on the specific linear operator to be inverted and the adopted regularization strategy. By resorting to the Bayesian interpretation of regularization, we show that such networks can be implemented rigorously and efficiently whenever the linear operator admits a state-space representation. An analytic expression is provided for the basis functions as well as for the entries of the matrix of the linear system used to compute the weights. The results are illustrated through a deconvolution problem where the spontaneous secretory rate of luteinizing hormone (LH) of the hypophisis is reconstructed from measurements of plasma LH concentrations.     

Knowledge distilling based model compression and feature learning in fault diagnosis

Recently, there has been interest in developing diagnosis methods that combine model-based and data-driven diagnosis. In both approaches, selecting the relevant measurements or extracting important features from historical data is a key determiner of the success of the algorithm. Recently, deep learning methods have been effective in automating the feature selection process. Autoencoders have been shown to be an effective neural network configuration for extracting features from complex data, however, they may also learn irrelevant features. In addition, end-to-end classification neural networks have also been used for diagnosis, but like autoencoders, this method may also learn unimportant features thus making the diagnostic inference scheme inefficient. To rapidly extract significant fault features, this paper employs end-to-end networks and develops a new feature extraction method based on importance analysis and knowledge distilling. First, a set of cumbersome neural network models are trained to predict faults and some of their internal values are defined as features. Then an occlusion-based importance analysis method is developed to select the most relevant input variables and learned features. Finally, a simple student neural network model is designed based on the previous analysis results and an improved knowledge distilling method is proposed to train the student model. Because of the way the cumbersome networks are trained, only fault features are learned, with the importance analysis further pruning the relevant feature set. These features can be rapidly generated by the student model. We discuss the algorithms, and then apply our method to two typical dynamic systems, a communication system and a 10-tank system employed to demonstrate the proposed approach.     

基于贝叶斯方法的神经网络非线性模型辨识

研究了基于贝叶斯推理的多层前向神经网络训练算法,以提高网络的泛化性能。在网络目标函数中引入表示网络结构复杂性的惩罚项,以便能够在训练优化过程中降低网络结构的复杂性,达到避免网络过拟合的目的。训练过程中使用显式的概率分布假设对模型进行分析和推断,根据融入先验分布的假设和依据,获取网络参数和正则化参数的后验条件概率,并基于后验分布的贝叶斯推理得出最优化参数。利用上述算法训练前向网络,对一个微型锅炉对象进行了模型辨识,通过测试,证明所辨识出的对象模型能够较好地表现出对象的动态行为,且具有较好的泛化性能。     

Computer-aided prediction of the Al2O3 nanoparticles’ effects on tensile strength and percentage of water absorption of concrete specimens

In the present paper, two models based on artificial neural networks and genetic programming for predicting split tensile strength and percentage of water absorption of concretes containing Al₂O₃ nanoparticles have been developed at different ages of curing. For purpose of building these models, training and testing using experimental results for 144 specimens produced with 16 different mixture proportions were conducted. The data used in the multilayer feed-forward neural networks models and input variables of genetic programming models are arranged in a format of eight input parameters that cover the cement content, nanoparticle content, aggregate type, water content, the amount of superplasticizer, the type of curing medium, Age of curing and number of testing try. According to these input parameters, in the neural networks and genetic programming models, the split tensile strength and percentage of water absorption values of concretes containing Al₂O₃ nanoparticles were predicted. The training and testing results in the neural network and genetic programming models have shown that every two models have strong potential for predicting the split tensile strength and percentage of water absorption values of concretes containing Al₂O₃ nanoparticles. It has been found that NN and GEP models will be valid within the ranges of variables. In neural networks model, as the training and testing ended when minimum error norm of network gained, the best results were obtained, and in genetic programming model, when 4 gens was selected to construct the model, the best results were acquired. Although neural network have predicted better results, genetic programming is able to predict reasonable values with a simpler method rather than neural network.     

RTS游戏中用户行为的神经网络预测模型

即时战略游戏(简称RTS游戏)中,用户的行为由于游戏自身庞大的决策空间而难以预测.针对这个问题,提出了通过对RTS游戏的对战记录进行分析,建立5种结构的神经网络模型来预测用户行为的方法.模型考虑了不同时间片的状态对于决策行为的影响,设计了单时间片输入和双时间片输入的神经网络,并与基于动态贝叶斯网络的模型进行了比较.实验结果表明,基于单时间片输入的神经网络模型能够更加快速地完成训练过程并达到满意的预测准确度.     

Regularized variational Bayesian learning of echo state networks with delay&sum readout

In this work, a variational Bayesian framework for efficient training of echo state networks (ESNs) with automatic regularization and delay&sum (D&S) readout adaptation is proposed. The algorithm uses a classical batch learning of ESNs. By treating the network echo states as fixed basis functions parameterized with delay parameters, we propose a variational Bayesian ESN training scheme. The variational approach allows for a seamless combination of sparse Bayesian learning ideas and a variational Bayesian space-alternating generalized expectation-maximization (VB-SAGE) algorithm for estimating parameters of superimposed signals. While the former method realizes automatic regularization of ESNs, which also determines which echo states and input signals are relevant for "explaining" the desired signal, the latter method provides a basis for joint estimation of D&S readout parameters. The proposed training algorithm can naturally be extended to ESNs with fixed filter neurons. It also generalizes the recently proposed expectation-maximization-based D&S readout adaptation method. The proposed algorithm was tested on synthetic data prediction tasks as well as on dynamic handwritten character recognition.     

Tree density estimation in a tropical woodland ecosystem with multiangular MISR and MODIS data

In this paper we evaluate the potential of spectral, temporal and angular aspect of remotely sensed data for quantitative extraction of forest structure information in tropical woodlands. Moderate resolution imaging spectroradiometer (MODIS) multispectral data at 500-meter spatial resolution from different dates, multiangle imaging spectroradiometer (MISR) bidirectional reflectance factors (BRF) and normalized difference angular index (NDAI) derived from MISR data at 275-meter spatial resolution were used as input data. The number of trees per hectare bigger than 20cm in diameter at breast height was taken as variable of interest. Simple and multiple ordinary least square regressions and artificial neural networks (ANN) were tested to understand the relationships between the various sources of remotely sensed data and the output variable. An experimental design technique, followed by a classification of the input variables and a factor analysis were implemented in order to understand the structure, reduce the dimensionality of the data and avoid the overfitting of the neural network. The results show that there is a significant amount of independent information in the angular dimension, and this information is highly relevant to the estimation of tree densities in the study area. The MISR NDAI indexes improved the performance of the MISR BRF. The non-linear ANN outperformed the linear regressions. The best results were obtained with the ANN after selecting the input variables according to the results of the experimental design, the classification and the factor analysis, with a 0.71 correlation coefficient against the 0.58 of the best linear regression model.     

基于神经网络和灰色模型的非线性预估

以某己内酰胺厂磷酸羟胺(HPO)的制备的现场数据为基础,利用贝叶斯正则化神经网络和灰色模型建立了磷酸羟胺中的H+浓度的预测模型;比较了神经网络和灰色模型的差异,并把两者结合起来,建立模型进行预测。最后验证了用神经网络和灰色模型相结合建立起来的磷酸羟胺模型可以迅速有效的预测信息,从而为实现质量指标的实时预估和获取专家系统知识奠定了基础。     

Transformation radial basis neural network for relevant feature selection

Radial basis neural networks are excellent candidates for selecting relevant features in pattern recognition problems. By a slight change in the traditional three-layer architecture of a radial basis neural network, we can obtain a quantitative method, which allows us to get a ranking within the features. We present a new neural network concept, combining at the same time two different skills: classification and detection of relevant features in the input vector.     

Bayesian Networks for Data Mining

A Bayesian network is a graphical model that encodesprobabilistic relationships among variables of interest. When used inconjunction with statistical techniques, the graphical model hasseveral advantages for data modeling. One, because the model encodesdependencies among all variables, it readily handles situations wheresome data entries are missing. Two, a Bayesian network can be used tolearn causal relationships, and hence can be used to gain understanding about a problem domain and to predict the consequencesof intervention. Three, because the model has both a causal andprobabilistic semantics, it is an ideal representation for combiningprior knowledge (which often comes in causal form) and data. Four,Bayesian statistical methods in conjunction with Bayesian networksoffer an efficient and principled approach for avoiding theoverfitting of data. In this paper, we discuss methods for constructing Bayesian networks from prior knowledge and summarizeBayesian statistical methods for using data to improve these models.With regard to the latter task, we describe methods for learning boththe parameters and structure of a Bayesian network, includingtechniques for learning with incomplete data. In addition, we relateBayesian-network methods for learning to techniques for supervised andunsupervised learning. We illustrate the graphical-modeling approachusing a real-world case study.     

Prediction of service life of large centrifugal compressor remanufactured impeller based on clustering rough set and fuzzy Bandelet neural network

In order to predict the service life of large centrifugal compressor impeller correctly, the rough set and fuzzy Bandelet neural network are combined to construct the novel prediction model which can give full play to theirs advantages. The attribute reduction algorithm based rough set and clustering method is firstly designed to optimize the inputting variables of fuzzy Bandelet neural network. And then the prediction model based on fuzzy Bandelet neural network is proposed, the Bandelet function is used as the excitation function of hidden layer and is combined with fuzzy theory to improve the prediction effectiveness of the prediction model. The training algorithm of fuzzy Bandelet neural network is designed based on improved genetic algorithm, the improved genetic algorithm introduces the adaptive differential evolution method into the traditional genetic algorithm, which can effectively optimize the parameters of fuzzy Bandelet neural network. Finally, the original 30 input variables of fuzzy Bandelet neural network are reduced to 9 input nodes based on rough set using 500 remanufacturing impellers as research objects. The service life of remanufacturing impeller is predicted based on three prediction models, and simulation results show that the fuzzy Bandelet neural network optimized by improved genetic algorithm has highest prediction precision and efficiency, which can correctly predict the service life of remanufacturing impeller.     

Influence of the optimization methods on neural state estimation quality of the drive system with elasticity

The paper deals with the implementation of optimized neural networks (NNs) for state variable estimation of the drive system with an elastic joint. The signals estimated by NNs are used in the control structure with a state-space controller and additional feedbacks from the shaft torque and the load speed. High estimation quality is very important for the correct operation of a closed-loop system. The precision of state variables estimation depends on the generalization properties of NNs. A short review of optimization methods of the NN is presented. Two techniques typical for regularization and pruning methods are described and tested in detail: the Bayesian regularization and the Optimal Brain Damage methods. Simulation results show good precision of both optimized neural estimators for a wide range of changes of the load speed and the load torque, not only for nominal but also changed parameters of the drive system. The simulation results are verified in a laboratory setup.     

Neural network‐based state prediction for strategy planning of an air hockey robot

We analyze a neural network implementation for puck state prediction in robotic air hockey. Unlike previous prediction schemes which used simple dynamic models and continuously updated an intercept state estimate, the neural network predictor uses a complex function, computed with data acquired from various puck trajectories, and makes a single, timely estimate of the final intercept state. Theoretically, the network can account for the complete dynamics of the table if all important state parameters are included as inputs, an accurate data training set of trajectories is used, and the network has an adequate number of internal nodes. To develop our neural networks, we acquired data from 1500 no‐bounce and 1500 one‐bounce puck trajectories, noting only translational state information. Analysis showed that performance of neural networks designed to predict the results of no‐bounce trajectories was better than the performance of neural networks designed for one‐bounce trajectories. Since our neural network input parameters did not include rotational puck estimates and recent work shows the importance of spin in impact analysis, we infer that adding a spin input to the neural network will increase the effectiveness of state estimates for the one‐bounce case. © 2001 John Wiley & Sons, Inc.     

Comparative analysis of artificial neural networks and dynamic models as virtual sensors

This paper presents a comparison of predictive models for the estimation of engine power and tailpipe emissions for a 4 kW gasoline scooter. This study forms a benchmark toward establishing an online emissions control and monitoring system to bring the emissions to within specific limits. Three emissions predictive models were investigated in this study; direct and series artificial neural network (ANN) models and a MATLAB dynamic model. The direct models takes variables lambda, throttle position, engine and vehicle speed to predict the engine power and the emissions CO, CO₂ and HC. The series model first takes the mentioned input to predict the engine power and consequently using the engine power as the fifth input to predict the emissions. For the ANN models, two multilayered networks were compared and analyzed; the backpropagation (BP) and optimization layer-by-layer (OLL) algorithms. The predictive accuracy for each algorithm were compared and it was found that the OLL network is the most accurate with a maximum mean relative error (MRE) of 1.78% and 1.38% for the direct and series predictive model respectively. Comparative results showed that the series neural network model gives the most accurate predictions, with MRE of 0.63% and 0.47% for the engine power and emissions respectively. The series neural network model can be seen as generic virtual power and emissions sensors, substituting costly and cumbersome hardware. Simple obtainable process parameters together with the series neural network will contribute immensely in control and tuning of emissions for real-time vehicular applications.     

Restricted Bayesian classification networks

Bayesian networks are graphical models that describe dependency relationships between variables, and are powerful tools for studying probability classifiers. At present, the causal Bayesian network learning method is used in constructing Bayesian network classifiers while the contribution of attribute to class is over-looked. In this paper, a Bayesian network specifically for classification-restricted Bayesian classification networks is proposed. Combining dependency analysis between variables, classification accuracy evaluation criteria and a search algorithm, a learning method for restricted Bayesian classification networks is presented. Experiments and analysis are done using data sets from UCI machine learning repository. The results show that the restricted Bayesian classification network is more accurate than other well-known classifiers.     

Neural network-based identification of SMB chromatographic processes

In this contribution, the identification problem for the control of nonlinear simulated moving bed (SMB) chromatographic processes is addressed. For process control the flow rates of extract, desorbent, and recycle of the SMB process, and the switching time are the manipulated variables. But these variables influence the process in a strongly coupled manner. Therefore, a new set of input variables is introduced by a nonlinear transformation of the physical inputs, such that the couplings are reduced considerably. The front positions of the axial concentration profile are taken as model outputs. Multilayer feedforward neural networks (NN) are utilized as approximating models of the nonlinear input–output behavior. The gradient distribution of the model outputs with respect to the inputs is used to determine their structural parameters and the network size is chosen by the SVD method. To illustrate the effectiveness of the identification method, a laboratory scale SMB process is used as an example. The simulation results of the identified model confirm a very good approximation of the first principles models and exhibit a satisfactory long-range prediction performance.     

Damage detection in Timoshenko beam structures by multilayer perceptron and radial basis function networks

This study investigates the efficiency of artificial neural networks (ANNs) in health monitoring of pristine and damaged beam-like structures. Beam modeling is based on Timoshenko theory. Two commonly used network models, multilayer perceptron (MLP) and radial basis neural network (RBNN), are used. Beam material and geometrical properties, beam end conditions and dynamically obtained data are used as input to the neural networks. The combinations of these parameters yield umpteenth input data. Therefore, to examine the effectiveness of ANNs, the frequency of intact beams is first tried to be determined by the network models, given the material and geometrical characteristics of beam elements and support conditions. The methodology to compute the vibrational data utilized in training the networks is provided. Showing the robustness of network models, the second stage of the study is carried out. At this stage, the crack parameters (e.g. the location and severity of crack) are estimated by the ANNs using the beam properties, beam end conditions and vibrational data, which consist of natural frequencies and mode shape rotation values. Despite the multiplexed input data, no data reduction schemes or multistage computations are executed in training and validation of neural network models. As a result of analysis runs, the optimal MLP and RBNN models are determined. Comparison of these models shows that the optimal RBNN algorithm performs better. The effectiveness of optimal ANN models in the presence of noise is also presented. As a conclusion, the trained network can be used as a diagnosis method in structural health monitoring of beam-like structures.