A neural network-based approach for dynamic quality prediction in a plastic injection molding process

This paper presents an innovative neural network-based quality prediction system for a plastic injection molding process. A self-organizing map plus a back-propagation neural network (SOM-BPNN) model is proposed for creating a dynamic quality predictor. Three SOM-based dynamic extraction parameters with six manufacturing process parameters and one level of product quality were dedicated to training and testing the proposed system. In addition, Taguchi’s parameter design method was also applied to enhance the neural network performance. For comparison, an additional back-propagation neural network (BPNN) model was constructed for which six process parameters were used for training and testing. The training and testing data for the two models respectively consisted of 120 and 40 samples. Experimental results showed that such a SOM-BPNN-based model can accurately predict the product quality (weight) and can likely be used for various practical applications.     

基于BP神经网络的产品造型设计评价

为了对产品的造型设计进行评价,在分析人工神经网络原理的基础上提出了应用BP神经网络评价产品造型设计的方法.根据BP神经网络具有自学习、自组织、自适应和非线性动态处理等优点,建立了产品造型设计BP神经网络评价模型,选择某一产品造型设计的13款方案作为样本,利用Matlab软件进行了BP网络的实例训练和验证.实验结果表明,BP神经网络模型可以较准确的对产品造型设计进行评价.     

Predictions-on-chip: model-based training and automated deployment of machine learning models at runtime

The design of gas turbines is a challenging area of cyber-physical systems where complex model-based simulations across multiple disciplines (e.g., performance, aerothermal) drive the design process. As a result, a continuously increasing amount of data is derived during system design. Finding new insights in such data by exploiting various machine learning (ML) techniques is a promising industrial trend since better predictions based on real data result in substantial product quality improvements and cost reduction. This paper presents a method that generates data from multi-paradigm simulation tools, develops and trains ML models for prediction, and deploys such prediction models into an active control system operating at runtime with limited computational power. We explore the replacement of existing traditional prediction modules with ML counterparts with different architectures. We validate the effectiveness of various ML models in the context of three (real) gas turbine bearings using over 150,000 data points for training, validation, and testing. We introduce code generation techniques for automated deployment of neural network models to industrial off-the-shelf programmable logic controllers.

     

Application of neural networks in forecasting engine systems reliability

This paper presents a comparative study of the predictive performances of neural network time series models for forecasting failures and reliability in engine systems. Traditionally, failure data analysis requires specifications of parametric failure distributions and justifications of certain assumptions, which are at times difficult to validate. On the other hand, the time series modeling technique using neural networks provides a promising alternative. Neural network modeling via feed-forward multilayer perceptron (MLP) suffers from local minima problems and long computation time. The radial basis function (RBF) neural network architecture is found to be a viable alternative due to its shorter training time. Illustrative examples using reliability testing and field data showed that the proposed model results in comparable or better predictive performance than traditional MLP model and the linear benchmark based on Box–Jenkins autoregressive-integrated-moving average (ARIMA) models. The effects of input window size and hidden layer nodes are further investigated. Appropriate design topologies can be determined via sensitivity analysis.     

Bayesian neural networks at scale: a performance analysis and pruning study

Bayesian neural networks (BNNs) are a promising method of obtaining statistical uncertainties for neural network predictions but with a higher computational overhead which can limit their practical usage. This work explores the use of high-performance computing with distributed training to address the challenges of training BNNs at scale. We present a performance and scalability comparison of training the VGG-16 and Resnet-18 models on a Cray-XC40 cluster. We demonstrate that network pruning can speed up inference without accuracy loss and provide an open-source software package, BPrune, to automate this pruning. For certain models we find that pruning up to 80% of the network results in only a 7.0% loss in accuracy. With the development of new hardware accelerators for deep learning, BNNs are of considerable interest for benchmarking performance. This analysis of training a BNN at scale outlines the limitations and benefits compared to a conventional neural network.

     

A NEURAL NETWORK APPROACH FOR EARLY COST ESTIMATION OF PACKAGING PRODUCTS

Product costs need to be identified early, i.e., during the design stage, where they can be controlled best. This implies the need to estimate the product's cost without full knowledge of the manufacturing process plans. In this paper, a feature-based cost estimation using a back-propagation neural network is proposed and a prototype system has been developed for estimating the costs of packaging products based on design information only. All the cost-related features of a product design were extracted and quantified according to their cost effects. The correlation between these cost-related features and the final cost of the product was established by training a back-propagation neural network using historical cost data. The extraction of cost-related features and the construction, training and validation of the neural network are described. The performance of the trained neural network based on a set of testing samples is also given.     

Neural networks of data inhibiting long memory pattern

We experiment with three neural network models for forecasting to better understand the performance of neural networks for the case when the data exhibits a long memory pattern. To obtain the optimum networks, the effect of network characteristics such as the training parameters, the number of hidden layers, and the testing and training percentages are simulated. The third model, which consists of a combination of individual time series forecasts, provides superior results.     

Customer churn prediction by hybrid neural networks

As churn management is a major task for companies to retain valuable customers, the ability to predict customer churn is necessary. In literature, neural networks have shown their applicability to churn prediction. On the other hand, hybrid data mining techniques by combining two or more techniques have been proved to provide better performances than many single techniques over a number of different domain problems. This paper considers two hybrid models by combining two different neural network techniques for churn prediction, which are back-propagation artificial neural networks (ANN) and self-organizing maps (SOM). The hybrid models are ANN combined with ANN (ANN + ANN) and SOM combined with ANN (SOM + ANN). In particular, the first technique of the two hybrid models performs the data reduction task by filtering out unrepresentative training data. Then, the outputs as representative data are used to create the prediction model based on the second technique. To evaluate the performance of these models, three different kinds of testing sets are considered. They are the general testing set and two fuzzy testing sets based on the filtered out data by the first technique of the two hybrid models, i.e. ANN and SOM, respectively. The experimental results show that the two hybrid models outperform the single neural network baseline model in terms of prediction accuracy and Types I and II errors over the three kinds of testing sets. In addition, the ANN + ANN hybrid model significantly performs better than the SOM + ANN hybrid model and the ANN baseline model.     

COMBINING A RADIAL BASIS NEURAL NETWORK WITH TIME SERIES ANALYSIS TECHNIQUES TO PREDICT MANUFACTURING PROCESS PARAMETERS

The accurate prediction of the values of critical quality parameters of a product during the production stage is a key factor in the success of a manufacturing operation. Neural network algorithms have been used to successfully predict process parameter values. However, techniques to further improve the predictive capability of neural network models are sought. Thus, an analysis was conducted to determine if the predictive capability of the network would he improved if the prediction from a time series model of a manufacturing process parameter were included in the training data set of a radial basis function neural network model. A manufacturing process data set was evaluated, and the use of the time series model prediction significantly improved the neural network's prediction of critical process parameters. Often in a manufacturing environment, the collection of adequate amounts of data for network training is difficult. This integrated technique offers potential for improving network performance without collecting additional data.     

Short‐term electric power load forecasting based on cosine radial basis function neural networks: An experimental evaluation

This article presents the results of a study aimed at the development of a system for short‐term electric power load forecasting. This was attempted by training feedforward neural networks (FFNNs) and cosine radial basis function (RBF) neural networks to predict future power demand based on past power load data and weather conditions. This study indicates that both neural network models exhibit comparable performance when tested on the training data but cosine RBF neural networks generalize better since they outperform considerably FFNNs when tested on the testing data. © 2005 Wiley Periodicals, Inc. Int J Int Syst 20: 591–605, 2005.     

Computer-aided prediction of the Al2O3 nanoparticles’ effects on tensile strength and percentage of water absorption of concrete specimens

In the present paper, two models based on artificial neural networks and genetic programming for predicting split tensile strength and percentage of water absorption of concretes containing Al₂O₃ nanoparticles have been developed at different ages of curing. For purpose of building these models, training and testing using experimental results for 144 specimens produced with 16 different mixture proportions were conducted. The data used in the multilayer feed-forward neural networks models and input variables of genetic programming models are arranged in a format of eight input parameters that cover the cement content, nanoparticle content, aggregate type, water content, the amount of superplasticizer, the type of curing medium, Age of curing and number of testing try. According to these input parameters, in the neural networks and genetic programming models, the split tensile strength and percentage of water absorption values of concretes containing Al₂O₃ nanoparticles were predicted. The training and testing results in the neural network and genetic programming models have shown that every two models have strong potential for predicting the split tensile strength and percentage of water absorption values of concretes containing Al₂O₃ nanoparticles. It has been found that NN and GEP models will be valid within the ranges of variables. In neural networks model, as the training and testing ended when minimum error norm of network gained, the best results were obtained, and in genetic programming model, when 4 gens was selected to construct the model, the best results were acquired. Although neural network have predicted better results, genetic programming is able to predict reasonable values with a simpler method rather than neural network.     

Development of a parsimonious GA-NN ensemble model with a case study for Charpy impact energy prediction

A parsimonious genetic algorithm guided neural network ensemble modelling strategy is presented. Each neural network candidate model to participate in the ensemble model is structurally selected using a genetic algorithm. This provides an effective route to improve the performance of the individual neural network models as compared to more traditional neural network modelling approaches, whereby the neural network structure is selected through some trial-and-error methods or heuristics. The parsimonious neural network ensemble modelling strategy developed in this paper is highly efficient and requires very little extra computation for developing the ensemble model, thus overcoming one of the major known obstacles for developing an ensemble model. The key techniques behind the implementation of the ensemble model, include the formulation of the fitness function, the generation of the qualified neural network candidate models, as well as the specific definitions of the assemble strategies. A case study is presented which exploits a complex industrial data set relating to the Charpy impact energy for heat-treated steels, which was provided by Tata Steel Europe. Modelling results show a significant performance improvement over the previously developed models for the same data set.     

PREDICTIVE MODELS OF THE SPRING-BACK IN THE BENDING PROCESS

Spring-back is one of the most sensitive features of sheet metal forming processes because the elastic recovery during unloading leads to some geometric changes in the product. Three parameters that are most influential on spring-back in the V-die bending process are sheet thickness, sheet orientation, and punch tip radius. In this research, radial basis function (RBF) neural network and back propagation (BP) neural network approaches are used to predict the spring-back using the data generated from experimental observations. The performances of the models in training and testing sets are compared with those observations. Furthermore, output obtained through regression analysis is used to compare with the two developed model outputs. The results indicate that both networks can be applied successfully for prediction of spring-back against the input parameters of the V-die bending process. Also, the BP model shows better performance for prediction of spring-back with respect to the RBF model.     

Integration of case-based forecasting, neural network, and discriminant analysis for bankruptcy prediction

Recently, it has been an issue of interest how to integrate classification models to increase the prediction performance. This paper suggests a new structured model with multiple stages. It consists of four phases (training, test, adjustment, and prediction), and three types of input data (training, testing, and generalization). The integrated model is applied for bankruptcy prediction. A statistical model, discriminant analysis and two artificial intelligence models, neural network and case-based forecasting, are used in this study. The integration approach produces higher prediction accuracy than individual models.     

Artificial neural network approach for fault detection in rotary system

The detection and diagnosis of faults in technical systems are of great practical significance and paramount importance for the safe operation of the plant. An early detection of faults may help to avoid product deterioration, performance degradation, major damage to the machinery itself and damage to human health or even loss of lives. The centrifugal pumping rotary system is considered for this research. This paper presents the development of artificial neural network-based model for the fault detection of centrifugal pumping system. The fault detection model is developed by using two different artificial neural network approaches, namely feed forward network with back propagation algorithm and binary adaptive resonance network (ART1). The training and testing data required are developed for the neural network model that were generated at different operating conditions, including fault condition of the system by real-time simulation through experimental model. The performance of the developed back propagation and ART1 model were tested for a total of seven categories of faults in the centrifugal pumping system. The results are compared and the conclusions are presented.     

A Neural Syntactic Language Model

This paper presents a study of using neural probabilistic models in a syntactic based language model. The neural probabilistic model makes use of a distributed representation of the items in the conditioning history, and is powerful in capturing long dependencies. Employing neural network based models in the syntactic based language model enables it to use efficiently the large amount of information available in a syntactic parse in estimating the next word in a string. Several scenarios of integrating neural networks in the syntactic based language model are presented, accompanied by the derivation of the training procedures involved. Experiments on the UPenn Treebank and the Wall Street Journal corpus show significant improvements in perplexity and word error rate over the baseline SLM. Furthermore, comparisons with the standard and neural net based N-gram models with arbitrarily long contexts show that the syntactic information is in fact very helpful in estimating the word string probability. Overall, our neural syntactic based model achieves the best published results in perplexity and WER for the given data sets.This work was supported by the National Science Foundation under grant No. IIS-0085940.Editors: Dan Roth and Pascale Fung     

Building surrogate models for engineering problems by integrating limited simulation data and monotonic engineering knowledge

The use of surrogate models to replace expensive computations with computer simulations has been widely studied in engineering problems. However, often only limited simulation data is available when designing complex products due to the cost of obtaining this kind of data. This presents a challenge for building surrogate models because the information contained in the limited simulation data is incomplete. Therefore, a method for building surrogate models by integrating limited simulation data and engineering knowledge with evolutionary neural networks (eDaKnow) is presented. In eDaKnow, a neural network uses an evolutionary algorithm to integrate the simulation data and the monotonic engineering knowledge to learn its weights and structure synchronously. This method involves converting both limited simulation data and engineering knowledge into the respective fitness functions. Compared with the previous work of others, we propose a method to train the surrogate model by combining data and knowledge through evolutionary neural network. We take knowledge as fitness function to train the model, and use a network structure self-learning method, which means that there is no need to adjust the network structure manually. The empirical results show that: (1) eDaKnow can be used to integrate limited simulation data and monotonic knowledge into a neural network, (2) the prediction accuracy of the newly constructed surrogate model is increased significantly, and (3) the proposed eDaKnow outperforms other methods on relatively complex benchmark functions and engineering problems.     

一种高精度微孔板浑浊度识别网络

提出一种基于卷积神经网络的高精度微孔板浑浊度分类算法。该算法主要将传统图像处理技术与卷积神经网络技术相结合,通过传统图像处理算法将圆孔从自然拍摄的微孔板图像中切割下来,并将切割下来的圆孔图像制作成圆孔数据集,用于网络模型的训练、评估和测试。同时,通过深度学习技术,设计并训练多个基于深度可分离卷积核的卷积神经网络模型,然后筛选出评估准确率最高的浑浊度分类模型,应用于圆孔识别系统,从而可提高研究人员的工作效率。     

An optimization methodology for machine learning strategies and?regression problems in ballistic impact scenarios

In domains with limited data, such as ballistic impact, prior researches have proven that the optimization of artificial neural models is an efficient tool for improving the performance of a classifier based on MultiLayer Perceptron. In addition, this research aims to explore, in the ballistic domain, the optimization of other machine learning strategies and their application in regression problems. Therefore, this paper presents an optimization methodology to use with several approaches of machine learning in regression problems, maximizing the limited dataset and locating the best network topology and input vector of each network model. This methodology is tested in real regression scenarios of ballistic impact with different artificial neural models, obtaining substantial improvement in all the experiments. Furthermore, the quality stage, based on criteria of information theory, enables the determination of when the complexity of the network design does not penalize the fit over the data and thereby the selection of the best neural network model from a series of candidates. Finally, the results obtained show the relevance of this methodology and its application improves the performance and efficiency of multiple machine learning strategies in regression scenarios.     

Visualization of Neural Network Predictions for Weather Forecasting

Recurrent neural networks are prime candidates for learning evolutions in multi‐dimensional time series data. The performance of such a network is judged by the loss function, which is aggregated into a scalar value that decreases during training. Observing only this number hides the variation that occurs within the typically large training and testing data sets. Understanding these variations is of highest importance to adjust network hyper‐parameters, such as the number of neurons, number of layers or to adjust the training set to include more representative examples. In this paper, we design a comprehensive and interactive system that allows users to study the output of recurrent neural networks on both the complete training data and testing data. We follow a coarse‐to‐fine strategy, providing overviews of annual, monthly and daily patterns in the time series and directly support a comparison of different hyper‐parameter settings. We applied our method to a recurrent convolutional neural network that was trained and tested on 25 years of climate data to forecast meteorological attributes, such as temperature, pressure and wind velocity. We further visualize the quality of the forecasting models, when applied to various locations on the Earth and we examine the combination of several forecasting models.