An inferential modeling method using enumerative PLS based nonnegative garrote regression

In this paper a robust linear regression method with variable selection is proposed for predicting desirable end-of-line quality variables in complex industrial processes. The development of such prediction models is challenging because there is usually a large pool of candidate explanatory variables, limited sample data, and multicollinearity among explanatory variables. The proposed method is named as the enumerative partial least square based nonnegative garrote regression. It employs partial least square regression in enumerative manner to generate initial model coefficients and then uses a nonnegative garrote method to shrink original coefficients so that irrelevant variables can be eliminated implicitly. Analysis about the advantages of the proposed method is provided compared to existing state-of-art model construction methods. Two simulation examples as well as an industrial application in a local semiconductor factory unit are used to validate the proposed method. These examples witness substantial improvement in terms of accuracy and robustness in variable selection compared to existing methods. Specifically, for the industrial case the percentages of improvement in terms of root mean squared error is up to 24.3% compared with the previous work.     

基于非负绞杀与长短期记忆神经网络的动态软测量算法

现代工业过程建模中,生产过程的多变量、非线性及动态性会导致模型复杂度增高且建模精度降低.针对这一问题,将非负绞杀算法(NNG)嵌入长短期记忆(LSTM)神经网络,提出一种基于LSTM神经网络及其输入变量选择的动态软测量算法.首先,通过参数优化生成训练好的LSTM神经网络,利用其出色的历史信息记忆能力处理工业过程中的动态、时滞等问题;其次,采用NNG算法对LSTM网络输入权重进行压缩,剔除冗余变量,提高模型精度,并采用网格搜索法与分块交叉验证对其超参数寻优;最后,将算法应用于某火电厂脱硫过程排放烟气SO2浓度软测量建模,并与其它先进算法进行性能比较.实验结果表明所提算法能有效剔除冗余变量,降低模型复杂度并提高其预测性能.     

一种结构自适应神经网络及其训练方法

针对神经网络建模效果对网络结构、训练方法过于敏感的缺陷,提出一种结构自适应神经网络模型及其训练方法.模型具有双网结构并以"提前终止法"训练,一定程度上降低了建模效果对网络结构的敏感性;模型结构根据建模数据的噪声方差、模型当前误差等信息自适应调整,进一步提高了模型的建模效果,同时具有较高的时间效率.仿真结果表明,该方法弥补了提前终止等传统方法的部分不足,具有较好的效果.     

Artificial neural network models for lot-sizing problem: a case study

Lot-sizing is one of the most difficult problems in production planning. The main purpose of this study is to propose a new lot-sizing based on artificial neural network (ANN), which may lead to a better performance than commonly used lot-sizing heuristics (SM, EOQ, PPB, LUC, and LTC). The data obtained are the results of years 2004 thru 2009 for 186 different types of stock items from the 2nd Air Supply and Maintenance Centre Command, a state-funded factory in Kayseri, Turkey. Factual data were applied under the coverage of the study, and the system from which the data have been obtained is still in live and active status. In the study, the purchasing costs, holding costs, and set-up costs were taken into consideration. These data were obtained from the administration data system of the enterprise. The solutions of this lot-sizing heuristics were found by WinQSB software accordingly. The ANN was constituted by using the NeuroSolutions software. The criterion of deviation from the optimum solution and the criterion of percentage of times obtaining the optimum order pattern were taken into account for the comparison purposes. The performance values of 400 ANNs were compared to lot-sizing methods. MS Excel and Visual Basic Macro were utilized for all calculations applied after this stage. The results showed that the proposed ANN-based method outperformed all lot-sizing methods taken into account in this study.     

Modeling glass-forming ability of bulk metallic glasses using computational intelligent techniques

Modeling the glass-forming ability (GFA) of bulk metallic glasses (BMGs) is one of the hot issues ever since bulk metallic glasses (BMGs) are discovered. It is very useful for the development of new BMGs for various engineering applications, if GFA criterion modeled precisely. In this paper, we have proposed support vector regression (SVR), artificial neural network (ANN), general regression neural network (GRNN), and multiple linear regression (MLR) based computational intelligent (CI) techniques that model the maximum section thickness (D_max) parameter for glass forming alloys. For this study, a reasonable large number of BMGs alloys are collected from the current literature of material science. CI models are developed using three thermal characteristics of glass forming alloys i.e., glass transition temperature (T_g), the onset crystallization temperature (T_x), and liquidus temperature (T_l). The R²-values of GRNN, SVR, ANN, and MLR models are computed to be 0.5779, 0.5606, 0.4879, and 0.2611 for 349 BMGs alloys, respectively. We have investigated that GRNN model is performing better than SVR, ANN, and MLR models. The performance of proposed models is compared to the existing physical modeling and statistical modeling based techniques. In this study, we have investigated that proposed CI approaches are more accurate in modeling the experimental D_max than the conventional GFA criteria of BMGs alloys.     

Kernelized LARS–LASSO for constructing radial basis function neural networks

Model structure selection is of crucial importance in radial basis function (RBF) neural networks. Existing model structure selection algorithms are essentially forward selection or backward elimination methods that may lead to sub-optimal models. This paper proposes an alternative selection procedure based on the kernelized least angle regression (LARS)–least absolute shrinkage and selection operator (LASSO) method. By formulating the RBF neural network as a linear-in-the-parameters model, we derive a l ₁-constrained objective function for training the network. The proposed algorithm makes it possible to dynamically drop a previously selected regressor term that is insignificant. Furthermore, inspired by the idea of LARS, the computing of output weights in our algorithm is greatly simplified. Since our proposed algorithm can simultaneously conduct model structure selection and parameter optimization, a network with better generalization performance is built. Computational experiments with artificial and real world data confirm the efficacy of the proposed algorithm.     

A practical ANN model for predicting the PSS of two-way reinforced concrete slabs

This paper aims to develop a practical artificial neural network (ANN) model for predicting the punching shear strength (PSS) of two-way reinforced concrete slabs. In this regard, a total of 218 test results collected from the literature were used to develop the ANN models. Accordingly, the slab thickness, the width of the column section, the effective depth of the slab, the reinforcement ratio, the compressive strength of concrete, and the yield strength of reinforcement were considered as input variables. Meanwhile, the PSS was considered as the output variable. Several ANN models were developed, but the best model with the highest coefficient of determination (R²) and the smallest root mean square errors was retained. The performance of the best ANN model was compared with multiple linear regression and existing design code equations. The comparative results showed that the proposed ANN model was provided the most accurate prediction of PSS of two-way reinforced concrete slabs. The parametric study was carried out using the proposed ANN model to assess the effect of each input parameter on the PSS of two-way reinforced concrete slabs. Finally, a graphical user interface was developed to apply for practical design of PSS of two-way reinforced concrete slabs.

     

A multilayer-perceptron based method for variable selection in soft sensor design

The paper proposes a new method for variable selection for prediction settings and soft sensors applications. The new variable selection method is based on the multi-layer perceptron (MLP) neural network model, where the network is trained a single time, maintaining low computational cost. The proposed method was successfully applied, and compared with four state-of-the-art methods in one artificial dataset and three real-world datasets, two publicly available datasets (Box–Jenkins gas furnace and gas mileage), and a dataset of a problem where the objective is to estimate the fluoride concentration in the effluent of a real urban water treatment plant (WTP). The proposed method presents similar or better approximation performance when compared to the other four methods. In the experiments, among all the five methods, the proposed method selects the lowest number of variables and variables-delays pairs to achieve the best solution. In soft sensors applications having a lower number of variables is a positive factor for decreasing implementation costs, or even making the soft sensor feasible at all.     

An optimized ANN model based on genetic algorithm for predicting ripping production

Due to the environmental constraints and the limitations on blasting, ripping as a ground loosening and breaking method has become more popular in both mining and civil engineering applications. As a result, a more applicable rippability model is required to predict ripping production (Q) before conducting such tests. In this research, a hybrid genetic algorithm (GA) optimized by artificial neural network (ANN) was developed to predict ripping production results obtained from three sites in Johor state, Malaysia. It should be noted that the mentioned hybrid model was first time applied in this field. In this regard, 74 ripping tests were investigated in the studied areas and the relevant parameters were also measured. A series of GA–ANN models were conducted in order to propose a hybrid model with a higher accuracy level. To demonstrate the performance capacity of the hybrid GA–ANN model, a pre-developed ANN model was also proposed and results of predictive models were compared using several performance indices. The results revealed higher accuracy of the proposed hybrid GA–ANN model in estimating Q compared to ANN technique. As an example, root-mean-square error values of 0.092 and 0.131 for testing datasets of GA–ANN and ANN techniques, respectively, express the superiority of the newly developed model in predicting ripping production.

     

High performance concrete compressive strength forecasting using ensemble models based on discrete wavelet transform

This paper investigates the use of wavelet ensemble models for high performance concrete (HPC) compressive strength forecasting. More specifically, we incorporate bagging and gradient boosting methods in building artificial neural networks (ANN) ensembles (bagged artificial neural networks (BANN) and gradient boosted artificial neural networks (GBANN)), first. Coefficient of determination (R²), mean absolute error (MAE) and the root mean squared error (RMSE) statics are used for performance evaluation of proposed predictive models. Empirical results show that ensemble models (R²_BANN=0.9278, R²_GBANN=0.9270) are superior to a conventional ANN model (R²_ANN=0.9088). Then, we use the coupling of discrete wavelet transform (DWT) and ANN ensembles for enhancing the prediction accuracy. The study concludes that DWT is an effective tool for increasing the accuracy of the ANN ensembles (R²_WBANN=0.9397, R²_WGBANN=0.9528).     

Variable Selection for Artificial Neural Networks with Applications for Stock Price Prediction

We propose a new Artificial neural network (ANN) method where we select a set of variables as input variables to the ANN. The selection is made so that the input variables may be informative for a target variable as much as possible. The proposed method compared favorably with the existing ANN methods when their performances were evaluated based on 488 stocks in S&P500 in terms of prediction accuracy.     

Evaluation and prediction of flyrock resulting from blasting operations using empirical and computational methods

Mines, quarries and construction sites face environmental impacts, such as flyrock, due to blasting operations. Flyrock may cause damage to structures and injury to human. Therefore, flyrock prediction is required to determine safe blasting zone. In this regard, 232 blasting operations were investigated in five granite quarries, Malaysia. Blasting parameters comprising maximum charge per delay and powder factor were prepared to predict flyrock using empirical and intelligent methods. An empirical graph was proposed to predict flyrock distance for different powder factor values. In addition, using the same datasets, two intelligent systems, namely artificial neural network (ANN) and adaptive neuro-fuzzy inference system (ANFIS) were used to predict flyrock. Considering some model performance indices including coefficient of determination (R ²), value account for and root mean squared error and also using simple ranking procedure, the best flyrock prediction models were selected. It was found that the ANFIS model can predict flyrock with higher performance capacity compared to ANN predictive model. R ² values of testing datasets are 0.925 and 0.964 for ANN and ANFIS techniques, respectively, suggesting the superiority of the ANFIS technique in predicting flyrock.     

Artificial neural network–genetic algorithm for estimation of crop evapotranspiration in a semi-arid region of Iran

This study compares the daily potato crop evapotranspiration (ET_C) estimated by artificial neural network (ANN), neural network–genetic algorithm (NNGA) and multivariate nonlinear regression (MNLR) methods. Using a 6-year (2000–2005) daily meteorological data recorded at Tabriz synoptic station and the Penman–Monteith FAO 56 standard approach (PMF-56), the daily ET_C was determined during the growing season (April–September). Air temperature, wind speed at 2 m height, net solar radiation, air pressure, relative humidity and crop coefficient for every day of the growing season were selected as the input of ANN models. In this study, the genetic algorithm was applied for optimization of the parameters used in ANN approach. It was found that the optimization of the ANN parameters did not improve the performance of ANN method. The results indicated that MNLR, ANN and NNGA methods were able to predict potato ET_C at desirable level of accuracy. However, the MNLR method with highest coefficient of determination (R ² > 0.96, P value < 0.05) and minimum errors provided superior performance among the other methods.     

Variable selection using Gaussian process regression-based metrics for high-dimensional model approximation with limited data

In recent years, the importance of computationally efficient surrogate models has been emphasized as the use of high-fidelity simulation models increases. However, high-dimensional models require a lot of samples for surrogate modeling. To reduce the computational burden in the surrogate modeling, we propose an integrated algorithm that incorporates accurate variable selection and surrogate modeling. One of the main strengths of the proposed method is that it requires less number of samples compared with conventional surrogate modeling methods by excluding dispensable variables while maintaining model accuracy. In the proposed method, the importance of selected variables is evaluated using the quality of the model approximated with the selected variables only. Nonparametric probabilistic regression is adopted as the modeling method to deal with inaccuracy caused by using selected variables during modeling. In particular, Gaussian process regression (GPR) is utilized for the modeling because it is suitable for exploiting its model performance indices in the variable selection criterion. Outstanding variables that result in distinctly superior model performance are finally selected as essential variables. The proposed algorithm utilizes a conservative selection criterion and appropriate sequential sampling to prevent incorrect variable selection and sample overuse. Performance of the proposed algorithm is verified with two test problems with challenging properties such as high dimension, nonlinearity, and the existence of interaction terms. A numerical study shows that the proposed algorithm is more effective as the fraction of dispensable variables is high.

     

Prediction of first lactation 305-day milk yield in Karan Fries dairy cattle using ANN modeling

In this paper, an artificial neural network (ANN) model is proposed to predict the first lactation 305-day milk yield (FLMY305) using partial lactation records pertaining to the Karan Fries (KF) crossbred dairy cattle. A scientifically determined optimum dataset of representative breeding traits of the cattle is used to develop the model.Several training algorithms, viz., (i) gradient descent algorithm with adaptive learning rate; (ii) Fletcher–Reeves conjugate gradient algorithm; (iii) Polak–Ribiére conjugate gradient algorithm; (iv) Powell–Beale conjugate gradient algorithm; (v) Quasi-Newton algorithm with Broyden, Fletcher, Goldfarb, and Shanno (BFGS) update; and (vi) Levenberg–Marquardt algorithm with Bayesian regularization; along with various network architectural parameters, i.e., data partitioning strategy, initial synaptic weights, number of hidden layers, number of neurons in each hidden layer, activation functions, regularization factor, etc., are experimentally investigated to arrive at the best model for predicting the FLMY305.Also, a multiple linear regression (MLR) model is developed for the milk-yield prediction. The performances of ANN and MLR models are compared to assess the relative prediction capability of the former model.It emerges from this study that the performance of ANN model seems to be slightly superior to that of the conventional regression model. Hence, it is recommended that the ANNs can potentially be used as an alternative technique to predict FLMY305 in the KF cattle.     

Prediction of seismic slope stability through combination of particle swarm optimization and neural network

One of the main concerns in geotechnical engineering is slope stability prediction during the earthquake. In this study, two intelligent systems namely artificial neural network (ANN) and particle swarm optimization (PSO)–ANN models were developed to predict factor of safety (FOS) of homogeneous slopes. Geostudio program based on limit equilibrium method was utilized to obtain 699 FOS values with different conditions. The most influential factors on FOS such as slope height, gradient, cohesion, friction angle and peak ground acceleration were considered as model inputs in the present study. A series of sensitivity analyses were performed in modeling procedures of both intelligent systems. All 699 datasets were randomly selected to 5 different datasets based on training and testing. Considering some model performance indices, i.e., root mean square error, coefficient of determination (R ²) and value account for (VAF) and using simple ranking method, the best ANN and PSO–ANN models were selected. It was found that the PSO–ANN technique can predict FOS with higher performance capacities compared to ANN. R ² values of testing datasets equal to 0.915 and 0.986 for ANN and PSO–ANN techniques, respectively, suggest the superiority of the PSO–ANN technique.     

A comparative study of artificial neural network and adaptive neurofuzzy inference system for prediction of compressional wave velocity

In this study, two solutions for prediction of compressional wave velocity (p wave) are presented and compared: artificial neural network (ANN) and adaptive neurofuzzy inference system (ANFIS). Series of analyses were performed to determine the optimum architecture of utilized methods using the trial and error process. Several ANNs and ANFISs are constructed, trained and validated to predict p wave in the investigated carbonate reservoir. A comparative study on prediction of p wave by ANN and ANFIS is addressed, and the quality of the target prediction was quantified in terms of the mean-squared errors (MSEs), correlation coefficient (R ²) and prediction efficiency error. ANFIS with MSE of 0.0552 and R ² of 0.9647, and ANN with MSE of 0.042 and R ² of 0.976, showed better performance in comparison with MLR methods. ANN and ANFIS systems have performed comparably well and accurate for prediction of p wave.     

一种基于互信息变量选择的极端学习机算法

针对回归问题中存在的变量选择和网络结构设计问题, 提出一种基于互信息的极端学习机(ELM) 训练算法, 同时实现输入变量的选择和隐含层的结构优化. 该算法将互信息输入变量选择嵌入到ELM网络的学习过程之中, 以网络的学习性能作为衡量输入变量与输出变量相关与否的指标, 并以增量式的方法确定隐含层节点的规模.在Lorenz、Gas Furnace 和10 组标杆数据上的仿真结果表明了所提出算法的有效性. 该算法不仅可以简化网络结构, 还可以提高网络的泛化性能.

     

Forecasting of Software Reliability Using Neighborhood Fuzzy Particle Swarm Optimization Based Novel Neural Network

This paper proposes an artificial neural network (ANN) based software reliability model trained by novel particle swarm optimization (PSO) algorithm for enhanced forecasting of the reliability of software. The proposed ANN is developed considering the fault generation phenomenon during software testing with the fault complexity of different levels. We demonstrate the proposed model considering three types of faults residing in the software. We propose a neighborhood based fuzzy PSO algorithm for competent learning of the proposed ANN using software failure data. Fitting and prediction performances of the neighborhood fuzzy PSO based proposed neural network model are compared with the standard PSO based proposed neural network model and existing ANN based software reliability models in the literature through three real software failure data sets. We also compare the performance of the proposed PSO algorithm with the standard PSO algorithm through learning of the proposed ANN. Statistical analysis shows that the neighborhood fuzzy PSO based proposed neural network model has comparatively better fitting and predictive ability than the standard PSO based proposed neural network model and other ANN based software reliability models. Faster release of software is achievable by applying the proposed PSO based neural network model during the testing period.      

Effective recognition of MCCs in mammograms using an improved neural classifier

Computer-aided diagnosis is one of the most important engineering applications of artificial intelligence. In this paper, early detection of breast cancer through classification of microcalcification clusters from mammograms is emphasized. Although artificial neural network (ANN) has been widely applied in this area, the average accuracy achieved is only around 80% in terms of the area under the receiver operating characteristic curve A_z. This performance may become much worse when the training samples are imbalanced. As a result, an improved neural classifier is proposed, in which balanced learning with optimized decision making are introduced to enable effective learning from imbalanced samples. When the proposed learning strategy is applied to individual classifiers, the results on the DDSM database have demonstrated that the performance from has been significantly improved. An average improvement of more than 10% in the measurements of F₁ score and A_z has fully validated the effectiveness of our proposed method for the successful classification of clustered microcalcifications.