神经网络模型的透明化及输入变量约简

由于神经网络很容易实现从输入空间到输出空间的非线性映射,因此,神经网络应用者往往未考虑输入变量和输出变量之间的相关性,直接用神经网络来实现输入变量与输出变量之间的黑箱建模,致使模型中常存在冗余变量,并造成模型可靠性和鲁棒性差。提出一种透明化神经网络黑箱特性的方法,并用它剔除模型中的冗余变量。该方法首先利用神经网络释义图可视化网络;再利用连接权法计算神经网络输入变量的相对贡献率,判断其对输出变量的重要性;最后利用改进的随机化测验对连接权和输入变量贡献率进行显著性检验,修剪模型,并以综合贡献度和相对贡献率均不显著的输入变量的交集为依据,剔除冗余变量,实现NN模型透明化及变量选择。实验结果表明,该方法增加了模型的透明度,选择出了最佳输入变量,剔除了冗余输入变量,提高了模型的可靠性和鲁棒性。因此,该研究为神经网络模型的透明化及变量约简提供了一种新的方法。     

Weekly milk prediction on dairy goats using neural networks

Artificial neural networks (NN) have been widely used for both prediction and classification tasks in many fields of knowledge; however, few studies are available on dairy science. In this work, we use NN models to predict next week’s goat milk based on the current and previous milk production. A total of 35 Murciano-Granadina dairy goats were selected from a commercial farm according to number of lactation, litter size and body weight. Input variables taken into account were diet, milk production, stage of lactation and days between partum and first control. From the 35 goats, 22 goats were used to build the neural model and 13 goats were used to validate the model. It is important to emphasize that these 13 goats were not used to build the model in order to demonstrate the generalization capability of the network. Afterwards, the neural models that provided better prediction results were analysed in order to determine the relative importance of the input variables of the model. We found that the most important inputs are present and previous milk production, followed by days between parturition, and first milk control, and type of diet. Besides, we benchmark NN to other widely used prediction models, such as auto-regressive system modelling or naïve prediction. The results obtained with the neural models are better than with the rest of models. The best neural model in terms of accuracy provided a root mean square error equal to 0.57 kg/day and a low bias mean error equal to ? 0.05 kg/day. Dairy goat farmers could make management decisions during current lactation from one week to the next (present time), based on present and/or previous milk production and dairy goat factors, without waiting until the end of lactation.     

Neural modelling and control of a Diesel engine with pollution constraints

The paper describes a neural approach for modelling and control of a turbocharged Diesel engine. A neural model, whose structure is mainly based on some physical equations describing the engine behaviour, is built for the rotation speed and the exhaust gas opacity. The model is composed of three interconnected neural submodels, each of them constituting a nonlinear multi-input single-output error model. The structural identification and the parameter estimation from data gathered on a real engine are described. The neural direct model is then used to determine a neural controller of the engine, in a specialized training scheme minimising a multivariable criterion. Simulations show the effect of the pollution constraint weighting on a trajectory tracking of the engine speed. Neural networks, which are flexible and parsimonious nonlinear black-box models, with universal approximation capabilities, can accurately describe or control complex nonlinear systems, with little a priori theoretical knowledge. The presented work extends optimal neuro-control to the multivariable case and shows the flexibility of neural optimisers. Considering the preliminary results, it appears that neural networks can be used as embedded models for engine control, to satisfy the more and more restricting pollutant emission legislation. Particularly, they are able to model nonlinear dynamics and outperform during transients the control schemes based on static mappings.     

Municipal credit rating modelling by neural networks

The paper presents the modelling possibilities of neural networks on a complex real-world problem, i.e. municipal credit rating modelling. First, current approaches in credit rating modelling are introduced. Second, previous studies on municipal credit rating modelling are analyzed. Based on this analysis, the model is designed to classify US municipalities (located in the State of Connecticut) into rating classes. The model includes data pre-processing, the selection process of input variables, and the design of various neural networks' structures for classification. The selection of input variables is realized using genetic algorithms. The input variables are extracted from financial statements and statistical reports in line with previous studies. These variables represent the inputs of neural networks, while the rating classes from Moody's rating agency stand for the outputs. In addition to exact rating classes, data are also labelled by four basic rating classes. As a result, the classification accuracies and the contributions of input variables are studied for the different number of classes. The results show that the rating classes assigned to bond issuers can be classified with a high accuracy rate using a limited subset of input variables.     

Feedback linearizing control using hybrid neural networks identified by sensitivity approach

Globally linearizing control (GLC) is a control algorithm capable of using nonlinear process model directly. In GLC, mostly, first-principle models derived from dynamic mass, energy and momentum balances are used. When the process is not perfectly known, the unknown parts of the first principles model should be represented by black-box models, e.g. by neural networks. This paper is devoted to the identification and application of such hybrid models for GLC. It is shown that the first principles part of the model determines the dominant structure of the controller, while the black-box elements of the hybrid model are used as state and/or disturbance estimators. For the identification of the neural network elements of the hybrid model a sensitivity approach based algorithm has been developed. The underlying framework is illustrated by the temperature control of a continuous stirred tank reactor where a neural network is used to model the heat released by an exothermic chemical reaction.     

Expanding the occupational health methodology: A concatenated artificial neural network approach to model the burnout process in Chinese nurses

Artificial neural networks are sophisticated modelling and prediction tools capable of extracting complex, non-linear relationships between predictor (input) and predicted (output) variables. This study explores this capacity by modelling non-linearities in the hardiness-modulated burnout process with a neural network. Specifically, two multi-layer feed-forward artificial neural networks are concatenated in an attempt to model the composite non-linear burnout process. Sensitivity analysis, a Monte Carlo–based global simulation technique, is then utilised to examine the first-order effects of the predictor variables on the burnout sub-dimensions and consequences. Results show that (1) this concatenated artificial neural network approach is feasible to model the burnout process, (2) sensitivity analysis is a prolific method to study the relative importance of predictor variables and (3) the relationships among variables involved in the development of burnout and its consequences are to different degrees non-linear.     

Application of partial mutual information variable selection to ANN forecasting of water quality in water distribution systems

Recent trends in the management of water supply have increased the need for modelling techniques that can provide reliable, efficient, and accurate representation of the complex, non-linear dynamics of water quality within water distribution systems. Statistical models based on artificial neural networks (ANNs) have been found to be highly suited to this application, and offer distinct advantages over more conventional modelling techniques. However, many practitioners utilise somewhat heuristic or ad hoc methods for input variable selection (IVS) during ANN development.This paper describes the application of a newly proposed non-linear IVS algorithm to the development of ANN models to forecast water quality within two water distribution systems. The intention is to reduce the need for arbitrary judgement and extensive trial-and-error during model development. The algorithm utilises the concept of partial mutual information (PMI) to select inputs based on the analysis of relationship strength between inputs and outputs, and between redundant inputs. In comparison with an existing approach, the ANN models developed using the IVS algorithm are found to provide optimal prediction with significantly greater parsimony. Furthermore, the results obtained from the IVS procedure are useful for developing additional insight into the important relationships that exist between water distribution system variables.     

Selection of input variables for model identification of static nonlinear systems

System identification can be divided into structure and parameter identification. In most system-identification approaches the structure is presumed and only a parameter identification is performed to obtain the coefficients in the functional system. Yet, often there is little knowledge about the system structure. In such cases, the first step has to be the identification of the decisive input variables. In this paper a black-box input variable identification approach using feedforward neural networks is proposed.     

Modeling plasma surface modification of textile fabrics using artificial neural networks

In this paper, a neural network approach is used to understand the effects of fabric features and plasma processing parameters on fabric surface wetting properties. In this approach, fourteen features characterizing woven structures and two plasma parameters are taken as input variables, and the water contact angle cosine and the capillarity height of woven fabrics as output variables. In order to reduce the complexity of the model and effectively learn the network structure from a small number of data, a fuzzy logic based method is used for selecting the most relevant parameters which are taken as input variables of the reduced neural network models. With these relevant parameters, we can effectively control the plasma treatment by selecting the most appropriate fabric materials. Two techniques are used for improving the generalization capability of neural networks: (i) early stopping and (ii) Bayesian regularization. A methodology for optimizing such models is described. The learning abilities and prediction capabilities of the neural net models are compared in terms of different statistical performance criteria. Moreover, a connection weight method is used to determine the relative importance of each input variable in the networks. The obtained results show that neural network models could predict the process performance with reasonable accuracy. However, the neural model trained using Bayesian regularization provides the best results. Thus, it can be concluded that Bayesian network promises to be a valuable quantitative tool to evaluate, understand, and predict woven fabric surface modification by atmospheric air-plasma treatment.     

Artificial neural networks for infant mortality modelling

This work aims to investigate a simple to use and easy to interpret methodology for assessing the relative importance of input variables in artificial neural networks (ANNs) applied to epidemiological modelling. The independent variables were 43 variables of the social, economic, environmental and health sector of 59 Brazilian municipalities, and the outcomes were infant mortality rates from these municipalities. Two assays were developed for the ANN modelling. On the first, all 43 variables were taken as input; and on the second, input variables were chosen with the help of factor analysis (FA). The relative importance of the input variables was investigated by means of bootstrap replications of the ANN model on the second assay. Further, multiple linear regression models (LRMs) were developed with the same data set and compared to the ANN models. The FA analysis allowed the selection of eight variables for the second assay. The percent of explained variance R(2) on the ANNs was in the range 0.74-0.80, while linear models had R(2)=0.4-0.5. These findings were validated by the bootstrap replications, in which the ANN models remained with higher R(2) and lower mean square error than the LRMs. The analysis of the best (second) ANN model indicated the highest ranking of importance for the variables literacy, agricultural and livestock sector jobs, number of commercial establishments and telephones. The approach presented here successfully integrated a data-oriented model with expert knowledge, indicating the potentiality of ANN modelling in the prediction, planning and assessment of public health actions.     

Explicit output-feedback nonlinear predictive control based on black-box models

Nonlinear model predictive control (NMPC) algorithms are based on various nonlinear models. A number of on-line optimization approaches for output-feedback NMPC based on various black-box models can be found in the literature. However, NMPC involving on-line optimization is computationally very demanding. On the other hand, an explicit solution to the NMPC problem would allow efficient on-line computations as well as verifiability of the implementation. This paper applies an approximate multi-parametric nonlinear programming approach to explicitly solve output-feedback NMPC problems for constrained nonlinear systems described by black-box models. In particular, neural network models are used and the optimal regulation problem is considered. A dual-mode control strategy is employed in order to achieve an offset-free closed-loop response in the presence of bounded disturbances and/or model errors. The approach is applied to design an explicit NMPC for regulation of a pH maintaining system. The verification of the NMPC controller performance is based on simulation experiments.     

Assessment and prediction of tropospheric ozone concentration levels using artificial neural networks

This work deals specifically with the use of a neural network for ozone modelling in the lower atmosphere. The development of a neural network model is presented to predict the tropospheric (surface or ground) ozone concentrations as a function of meteorological conditions and various air quality parameters. The development of the model was based on the realization that the prediction of ozone from a theoretical basis (i.e. detailed atmospheric diffusion model) is difficult. In contrast, neural networks are useful for modelling because of their ability to be trained using historical data and because of their capability for modelling highly non-linear relationships. The network was trained using summer meteorological and air quality data when the ozone concentrations are the highest. The data were collected from an urban atmosphere. The site was selected to represent a typical residential area with high traffic influences. Three neural network models were developed. The main emphasis of the first model has been placed on studying the factors that control the ozone concentrations during a 24-hour period (daylight and night hours were included). The second model was developed to study the factors that regulate the ozone concentrations during daylight hours at which higher concentrations of ozone were recorded. The third model was developed to predict daily maximum ozone levels. The predictions of the models were found to be consistent with observations. A partitioning method of the connection weights of the network was used to study the relative percent contribution of each of the input variables. The contribution of meteorology on the ozone concentration variation was found to fall within the range 33.15–40.64%. It was also found that nitrogen oxide, sulfur dioxide, relative humidity, non-methane hydrocarbon and nitrogen dioxide have the most effect on the predicted ozone concentrations. In addition, temperature played an important role while solar radiation had a lower effect than expected. The results of this study indicate that the artificial neural network (ANN) is a promising method for air pollution modelling.     

保持生理特征的人脸编辑

提出了一种保持生理特征的交互式人脸编辑方法。采用控制点分层策略，即以用户直接操作的控制点对（称为主控制点对）为输入层，其他控制点对（称为次控制点对）为输出层，建立人工神经网络；然后采用误差反向传播法（Error Back Propagation）学习，从而建立主、次控制点之间的约束关系；最后通过输出层将编辑信息在模型中进行插值。该编辑结果可以应用到具有相同拓扑的任意人脸模型上。实验结果表明，采用分层控制的方法不仅保持了编辑操作的方便性、精确性，同时还保持了人脸生理特征的真实性。     

Application of neural networks and sensitivity analysis to improved prediction of trauma survival

The performance of trauma departments is widely audited by applying predictive models that assess probability of survival, and examining the rate of unexpected survivals and deaths. Although the TRISS methodology, a logistic regression modelling technique, is still the de facto standard, it is known that neural network models perform better. A key issue when applying neural network models is the selection of input variables. This paper proposes a novel form of sensitivity analysis, which is simpler to apply than existing techniques, and can be used for both numeric and nominal input variables. The technique is applied to the audit survival problem, and used to analyse the TRISS variables. The conclusions discuss the implications for the design of further improved scoring schemes and predictive models.     

Dynamics identification and control of nonlinear MIMO coupled plant using supervised neural gas and comparison with recurrent neural controller

The dynamics identification and subsequent control of a nonlinear system is not a trivial issue. The application of a neural gas network that is trained with a supervised batch version of the algorithm can produce identification models in a robust way. In this paper, the neural model identifies each local transfer function, demonstrating that the local linear approximation can be done. Moreover, other parameters are analyzed in order to obtain a correct modeling. Furthermore, the algorithm is applied to control a nonlinear multi-input multi-output system composed of tanks. In addition, this plant is a coupled system where the manipulated input variables are influencing all the output variables. The aim of the work is to demonstrate that the supervised neural gas algorithm is able to obtain linear models to be used in a state space design scenario to control nonlinear coupled systems and guarantee a robust control method. The results are compared with the common approach of using a recurrent neural controller trained with a dynamic backpropagation algorithm. Regarding the steady-state errors in disturbance rejection, reference tracking and sensitivity to simple process changes, the proposed approach shows an interesting application to control nonlinear plants.

     

基于改进PSO的发酵过程同步串联混合建模

准确可靠的过程模型是实现发酵过程优化的基础和前提. 对于反应机理复杂的发酵过程,串联混合建模是一种相对有效的建模方法, 但现有方法需要利用插值所得的数据进行中间变量黑箱模型的构建, 较大程度地影响了所建混合模型的泛化性能. 为此,提出一种可将黑箱模型构建问题转化为动态模型参数辨识问题的同步串联混合建模方法, 从而避免了现有方法需利用插值数据来构建黑箱模型的不足; 通过引入多精英学习策略和惯性权重自适应调整策略, 构造了一种改进的粒子群优化(Particle swarm optimization, PSO)算法自适应多精英学习PSO (Adaptive multi-elite learning PSO, AMLPSO)算法,并采用该算法求取黑箱模型的参数; 借鉴均匀设计思想确定黑箱模型的结构. 利用诺西肽分批发酵过程实际生产数据进行实验研究, 结果验证了所提方法的有效性.     

Neural network based model predictive control for a steel pickling process

A multi-layer feedforward neural network model based predictive control scheme is developed for a multivariable nonlinear steel pickling process in this paper. In the acid baths three variables under controlled are the hydrochloric acid concentrations. The baths exhibit the normal features of an industrial system such as nonlinear dynamics and multi-effects among variables. In the modeling, multiple input, single-output recurrent neural network subsystem models are developed using input–output data sets obtaining from mathematical model simulation. The Levenberg–Marquardt algorithm is used to train the process models. In the control (MPC) algorithm, the feedforward neural network models are used to predict the state variables over a prediction horizon within the model predictive control algorithm for searching the optimal control actions via sequential quadratic programming. The proposed algorithm is tested for control of a steel pickling process in several cases in simulation such as for set point tracking, disturbance, model mismatch and presence of noise. The results for the neural network model predictive control (NNMPC) overall show better performance in the control of the system over the conventional PI controller in all cases.     

A constrained optimization method based on BP neural network

A constrained optimization method based on back-propagation (BP) neural network is proposed in this paper. Taking the maximization of output for example, using unipolar sigmoid function as transfer function, the method presents a general mathematical expression of BP neural network constrained optimization and derives the partial derivative of output with respect to input. On this basis, the fundamental idea, algorithms and related models are given in this article. When BP neural network is on the basis of fitting, this method can adjust the input values of BP neural network to make the output values maximal or minimal. Therefore, with this method the application of BP neural network is expanded by combining BP network’s fitting with optimization. At the same time, the article also provides a new method to study the black-box problem. The experiments show that the constrained optimization method is effective.

     

Gaussian process approach for modelling of nonlinear systems

Parametric modelling principals such as neural networks, fuzzy models and multiple model techniques have been proposed for modelling of nonlinear systems. Research effort has focused on issues such as the selection of the structure, constructive learning techniques, computational issues, the curse of dimensionality, off-equilibrium behaviour, etc. To reduce these problems, the use of non-parametrical modelling approaches have been proposed. This paper introduces the Gaussian process (GP) prior approach for the modelling of nonlinear dynamic systems. The relationship between the GP model and the radial basis function neural network is explained. Issues such as selection of the dimension of the input space and the computation load are also discussed. The GP modelling technique is demonstrated on an example of the nonlinear hydraulic positioning system.     

Data-driven soft sensor of downhole pressure for a gas-lift oil well

Downhole pressure is a key variable in the operation of gas-lift oil wells. However, maintaining and replacing downhole sensors is a challenging task. In this context, we design and implement a data-driven soft sensor to estimate online the downhole pressure based on other (seabed and platform) available measurements. Such application is based on a two-step procedure. In the first step, discrete-time black-box and gray-box NARX models are identified offline and independently using historical data. Both polynomial and neural models are obtained. In the second step, recursive predictions of these multiple models are combined with current measured data (of variables other than the downhole pressure) by means of an interacting bank of unscented Kalman filters. In doing so, a closed-loop model prediction is performed. Three issues are investigated in this paper concerning: (i) the usage of a filter bank rather than a single filter approach, (ii) the availability of seabed variables as inputs of the models compared to the case where only platform variables are available, and (iii) the employment of gray-box models in the filters. Experimental results along 7 months of tests indicate that such closed-loop scheme improves estimation accuracy and robustness compared to the free-run model prediction or to the use of a single unscented Kalman filter. The method employed in this paper can also be applied to other soft sensing applications in industry.