Validation and data splitting in predictive regression modeling of honing surface roughness data

Model validation is critical in predicting the performance of manufacturing processes. In predictive regression, proper selection of variables helps minimize the model mismatch error, proper selection of models helps reduce the model estimation error, and proper validation of models helps minimize the model prediction error. In this paper, the literature is briefly reviewed and a rigorous procedure is proposed for evaluating the validation and data splitting methods in predictive regression modeling. Experimental data from a honing surface roughness study will be used to illustrate the methodology. In particular, the individual versus average data splitting methods as well as the fivefold versus threefold cross-validation methods are compared. This paper shows that statistical tests and prediction errors evaluation are important in subset selection and cross-validation of predictive regression models. No statistical differences were found between the fivefold and the threefold cross-validation methods, and between use of the individual and average data splitting methods in predictive regression modeling.     

Performance Evaluation of Deep Dense Layer Neural Network for Diabetes Prediction

Diabetes is one of the fastest-growing human diseases worldwide and poses a significant threat to the population’s longer lives. Early prediction of diabetes is crucial to taking precautionary steps to avoid or delay its onset. In this study, we proposed a Deep Dense Layer Neural Network (DDLNN) for diabetes prediction using a dataset with 768 instances and nine variables. We also applied a combination of classical machine learning (ML) algorithms and ensemble learning algorithms for the effective prediction of the disease. The classical ML algorithms used were Support Vector Machine (SVM), Logistic Regression (LR), Decision Tree (DT), K-Nearest Neighbor (KNN), and Naïve Bayes (NB). We also constructed ensemble models such as bagging (Random Forest) and boosting like AdaBoost and Extreme Gradient Boosting (XGBoost) to evaluate the performance of prediction models. The proposed DDLNN model and ensemble learning models were trained and tested using hyperparameter tuning and K-Fold cross-validation to determine the best parameters for predicting the disease. The combined ML models used majority voting to select the best outcomes among the models. The efficacy of the proposed and other models was evaluated for effective diabetes prediction. The investigation concluded that the proposed model, after hyperparameter tuning, outperformed other learning models with an accuracy of 84.42%, a precision of 85.12%, a recall rate of 65.40%, and a specificity of 94.11%.     

基于机器学习训练金属离子吸附能预测模型的研究

本研究通过密度泛函理论对氧化石墨烯和金属离子的吸附行为进行理论模拟。基于机器学习方法训练预测模型的过程中,缺失值采用推荐系统中广泛使用的奇异值分解方法处理,并用梯度提升机解释了影响吸附能的重要因素。结果发现吸附体系中存在九种特征可为吸附能提供90%的累积重要性,分别为离子半径、零点振动能量、密立根电荷、沸点、偶极矩、原子量、摩尔定容热容、自旋多重度和键长。定量评估了六种回归方法的预测精度,包括支持向量回归、岭回归、随机森林、极端随机森林、极端梯度提升和轻梯度提升机。结果表明,机器学习方法可提供足够的吸附能预测准确性,其中极端随机森林方法表现出最优的预测性能,均方误差仅为0.075。该模型用于香兰素吸附金属离子的测试,验证了基于机器学习训练金属离子吸附能预测模型的可行性,但仍需进一步提高其泛化能力。本研究基于机器学习预测吸附能,简化预测过程、节省计算时间,可为吸附去除金属离子的理论和实验研究提供参考。     

Prediction of COVID-19 Confirmed Cases Using Gradient Boosting Regression Method

The fast spread of coronavirus disease (COVID-19) caused by SARSCoV-2 has become a pandemic and a serious threat to the world. As of May 30, 2020, this disease had infected more than 6 million people globally, with hundreds of thousands of deaths. Therefore, there is an urgent need to predict confirmed cases so as to analyze the impact of COVID-19 and practice readiness in healthcare systems. This study uses gradient boosting regression (GBR) to build a trained model to predict the daily total confirmed cases of COVID-19. The GBR method can minimize the loss function of the training process and create a single strong learner from weak learners. Experiments are conducted on a dataset of daily confirmed COVID-19 cases from January 22, 2020, to May 30, 2020. The results are evaluated on a set of evaluation performance measures using 10-fold cross-validation to demonstrate the effectiveness of the GBR method. The results reveal that the GBR model achieves 0.00686 root mean square error, the lowest among several comparative models.     

Deep learning and machine learning-based early survival predictions of glioblastoma patients using pre-operative three-dimensional brain magnetic resonance imaging modalities

To propose and implement an automated machine learning (ML) based methodology to predict the overall survival of glioblastoma multiforme (GBM) patients. In the proposed methodology, we used deep learning (DL) based 3D U-shaped Convolutional Neural Network inspired encoder-decoder architecture to segment the brain tumor. Further, feature extraction was performed on these segmented and raw magnetic resonance imaging (MRI) scans using a pre-trained 2D residual neural network. The dimension-reduced principal components were integrated with clinical data and the handcrafted features of tumor subregions to compare the performance of regression-based automated ML techniques. Through the proposed methodology, we achieved the mean squared error (MSE) of 87 067.328, median squared error of 30 915.66, and a SpearmanR correlation of 0.326 for survival prediction (SP) with the validation set of Multimodal Brain Tumor Segmentation 2020 dataset. These results made the MSE far better than the existing automated techniques for the same patients. Automated SP of GBM patients is a crucial topic with its relevance in clinical use. The results proved that DL-based feature extraction using 2D pre-trained networks is better than many heavily trained 3D and 2D prediction models from scratch. The ensembled approach has produced better results than single models. The most crucial feature affecting GBM patients' survival is the patient's age, as per the feature importance plots presented in this work. The most critical MRI modality for SP of GBM patients is the T2 fluid attenuated inversion recovery, as evident from the feature importance plots.     

Predicting glass furnace output using statistical and neural computing methods

This paper describes the development of predictive models for glass production at a regional manufacturing company. The objectives of the models are to predict the actual batch tonnage produced per week from the glass furnace based on the planned production schedule. Four modelling methods were explored: (i) linear regression; (ii) nonlinear regression; (iii) artificial neural network using back-propagation; and (iv) radial basis function neural network. Using 175 cases of production schedule data and subsequent furnace output, the two neural network-based prediction models resulted in lower average absolute error and lower maximum absolute error than the linear or nonlinear regression models. Accurate neural network-based prediction models of furnace output will subsequently be used in the overall production planning system by utilizing estimates of furnace output to determine the necessary energy, raw material, repair and personnel requirements of the glass manufacturing facility.     

Wind Power Prediction Based on Machine Learning and Deep Learning Models

Wind power is one of the sustainable ways to generate renewable energy. In recent years, some countries have set renewables to meet future energy needs, with the primary goal of reducing emissions and promoting sustainable growth, primarily the use of wind and solar power. To achieve the prediction of wind power generation, several deep and machine learning models are constructed in this article as base models. These regression models are Deep neural network (DNN), k-nearest neighbor (KNN) regressor, long short-term memory (LSTM), averaging model, random forest (RF) regressor, bagging regressor, and gradient boosting (GB) regressor. In addition, data cleaning and data preprocessing were performed to the data. The dataset used in this study includes 4 features and 50530 instances. To accurately predict the wind power values, we propose in this paper a new optimization technique based on stochastic fractal search and particle swarm optimization (SFS-PSO) to optimize the parameters of LSTM network. Five evaluation criteria were utilized to estimate the efficiency of the regression models, namely, mean absolute error (MAE), Nash Sutcliffe Efficiency (NSE), mean square error (MSE), coefficient of determination (R²), root mean squared error (RMSE). The experimental results illustrated that the proposed optimization of LSTM using SFS-PSO model achieved the best results with R² equals 99.99% in predicting the wind power values.     

非线性动态WPLS航煤干点软测量

针对炼油工业过程存在的多变量、非线性和数据动态性问题,提出一种自回归移动平均模型与径向基函数-加权偏最小二乘相结合的非线性动态建模方法。首先建立基于径向基函数-加权偏最小二乘方法的软测量模型,然后利用自回归移动平均模型对数据进行时序分析校正,将动态误差信息加入到模型中去,实现模型的动态装换。将该方法应用到加氢裂化航煤干点的软测量建模中,从而获得比径向基函数-加权偏最小二乘算法更高的预测精度。     

An automatic system using mobile-agent software to model the calculation process of a chemical vapor deposition film deposition simulator

We have developed an automatic modeling system for calculation processes of the simulator to reproduce experimental results of chemical vapor deposition (CVD), in order to decrease the calculation cost of the simulator. Replacing the simulator by the mathematical models proposed by the system will contribute towards decreasing the calculation costs for predicting the experimental results. The system consists of a mobile agent and two software resources in computer networks, that is, generalized modeling software and a simulator reproducing cross-sections of the deposited films on the substrates with the micrometer- or nanometer-sized trenches. The mobile agent autonomously creates appropriate models by moving to and then operating the software resources. The models are calculated by partial least squares regression (PLS), quadratic PLS (QPLS) and error back propagation (BP) methods using artificial neural networks (ANN) and expresses by mathematical formulas to reproduce the calculated results of the simulator. The models show good reproducibility and predictability both for uniformity and filling properties of the films calculated by the simulator. The models using the BP method yield the best performance. The filling property data are more suitable to modeling than film uniformity.     

Local composition modelling of the thermodynamic properties of refrigerant and oil mixtures

Six local composition models of the thermodynamic behaviour of mixtures are described. Using data from the literature and a non-linear regression analysis, a comparison of the predictive abilities of the models is undertaken for R12, R22, R134a and R125 with various oils. The Wilson and Heil equations provide the most consistent results, with the Heil equation providing a modest improvement over the Wilson model. Using a 95% confidence interval, the Heil equation predicted the behaviour of R12 with a paraffinic mineral oil to within 3.1%; its worst-case 2-σ error was 10.4% (R22 with a polyol ester oil), and its average 2-σ error for all of the mixtures was 6.2%. Using model parameters and error estimates from the regression analyses, pressure-temperature-concentration behaviour for these mixtures can be predicted for system design and simulation.     

Developing linear error models for analog devices

Techniques are presented for developing linear error models for analog and mixed-signal devices. A simulation program developed to understand the modeling process is described, and results of simulations are presented. Methods for optimizing the size of empirical error models based on simulated error analyses are included. Once established, the models can be used in a comprehensive approach for optimizing the testing of the subject devices. Models are developed using data from a group of 13-bit A/D converters and compared with the simulation results     

Comments on: Process modeling for slope and aspect with application to elevation data maps

The authors should be commended on their methodological development of stochastic processes for slope and aspect. Their development of basic distribution theory needed to study these two processes, and the sufficient conditions that ensure independence and non-informative induced priors, provide a thorough contribution to the collection of work on gradient processes. The fully model-based approach for inference for slope and aspect enables the propagation of uncertainty to environmental process models of interest that would use these variables as explanatory variables in a regression. I greatly appreciate the opportunity to comment on this exciting work and offer some additional model considerations and applications. Specifically, I reaffirm the importance of scalability of the methodology to large datasets, offering a few considerations with regard to model specification. Next, I discuss the unique challenges of using the predictive distributions of the slope and aspect processes as input variables in spatial regression models. Finally, I offer possible applications and extensions of this work that might provide innovative insights into environmental processes.     

High-Fidelity Magnetic Equivalent Circuit Model for an Axisymmetric Electromagnetic Actuator

A computationally inexpensive magnetic equivalent circuit (MEC) improves axisymmetric electromagnet design and modeling tools by accurately capturing fringing and leakage effects. Lumped parameter MEC models are typically less accurate for modeling electromagnetic devices than distributed parameter finite-element models (FEMs). However, MEC models require significantly less computational time to solve than FEMs and therefore lend themselves to applications where solution time is critical, such as in optimization routines, dynamic simulation, or preliminary design. This paper describes how fringing permeances in axisymmetric electromagnetic devices can be derived and then included in a MEC model. Including fringing field effects significantly decreases error in the MEC model, creating a more accurate, or high fidelity, magnetic equivalent circuit (HFMEC). Eighty-nine electromagnets with unique geometries, coil currents, and materials were modeled with MEC, HFMEC, and FEM methods. The axisymmetric HFMEC developed in this work had 67% less average force error and 88% less average flux error compared to traditional MEC results while still being computationally inexpensive to solve.      

Response Modeling Methodology Validating Evidence from Engineering and the Sciences

Modeling a response in terms of the factors that affect it is often required in quality applications. While the normal scenario is commonly assumed in such modeling efforts, leading to the application of linear regression analysis, there are cases when the assumptions underlying this scenario are not valid and alternative approaches need to be pursued, like the normalization of the data or generalized linear modeling. Recently, a new response modeling methodology (RMM) has been introduced, which seems to be a natural generalization of various current scientific and engineering mainstream models, where a monotone convex (concave) relationship between the response and the affecting factor (or a linear combination of factors) may be assumed. The purpose of this paper is to provide the quality practitioner with a survey of these models and demonstrate how they can be derived as special cases of the new RMM. A major implication of this survey is that RMM can be considered a valid approach for quality engineering modeling and, thus, may be conveniently applied where theory‐based models are not available or the goodness‐of‐fit of current empirically‐derived models is unsatisfactory. A numerical example demonstrates the application of the new RMM to software reliability‐growth modeling. The behavior of the new model when the systematic variation vanishes (there is only random variation) is also briefly explored. Copyright © 2003 John Wiley & Sons, Ltd.     

A two-stage regression approach for spectroscopic quantitative analysis

In this paper, we propose a two-stage regression approach, which is based on the residual correction concept. Its underlying idea is to correct any given regressor by analyzing and modeling its residual errors in the input space. We report and discuss results of experiments conducted on three different datasets in infrared spectroscopy and designed in such a way to test the proposed approach by: 1) varying the kind of adopted regression method used to approximate the chemical parameter of interest. Partial least squares regression (PLSR), support vector machines (SVM) and radial basis function neural network (RBF) methods are considered; 2) adopting or not a feature selection strategy to reduce the dimension of the space where to perform the regression task. A comparative study with another approach which exploits differently estimation errors, namely adaptive boosting for regression (AdaBoost.R), is also included. The obtained results point out that the residual-based correction approach (RBC) can improve the accuracy of the estimation process. Not all the improvements are statistically significant but, at the same time, no case of accuracy decrease has been observed.     

Application of nonlinear regression analysis for ammonium exchange by natural (Bigadiç) clinoptilolite

The experimental data of ammonium exchange by natural Bigadiç clinoptilolite was evaluated using nonlinear regression analysis. Three two-parameters isotherm models (Langmuir, Freundlich and Temkin) and three three-parameters isotherm models (Redlich–Peterson, Sips and Khan) were used to analyse the equilibrium data. Fitting of isotherm models was determined using values of standard normalization error procedure (SNE) and coefficient of determination (R²). HYBRID error function provided lowest sum of normalized error and Khan model had better performance for modeling the equilibrium data. Thermodynamic investigation indicated that ammonium removal by clinoptilolite was favorable at lower temperatures and exothermic in nature.