A hierarchical optimization neural network for large-scale dynamic systems

A recurrent neural network for dynamical hierarchical optimization of nonlinear discrete large-scale systems is presented. The proposed neural network consists of hierarchically structured sub-networks: one coordination sub-network at the upper level and several local optimization sub-networks at the lower level. In particular, the coordination sub-network and the local optimization sub-networks work simultaneously. This feature makes the proposed method outperform in computational efficiency the conventional iterative algorithms where there usually exists an alternately waiting time during the coordination and local optimization processes. Moreover, the state equations of the subsystems of the large-scale system are imbedded into their corresponding local optimization sub-networks. This imbedding technique not only overcomes the difficulty in treating the constraints imposed by the state equations, but also leads to significant reduction in the network size. We present stability analysis to prove that the neural network is asymptotically stable and this stable state corresponds to the optimal solution to the original optimal control problem. Finally, we illustrate the performance of the proposed method by an example.     

Acquisition of state transitions and concept formation in neural networks

Searching of state transitions is an important subject of problem solving in artificial intelligence, computer science, engineering and operations research. In artificial intelligence, a breadth-first search is optimal, with uniform cost, but it takes considerable time to obtain a solution. Neural networks process state transitions in parallel with learning ability. The authors have developed a search procedure for state transitions, that resembles a breadth-first search, using neural networks. First, the input pattern states are self-organized in the neural network, which consists of a Kohonen layer followed by a state-planning layer. The state-planning layer makes lateral connections between the cells of transitions. Then, the initial and the target states are given as a problem. The network shows an optimal transition pathway of states in the neuron firings. Next, the state-transition procedure is developed for the formation of a concept for action planning. Here, as the action planning, an integration between the symbols and the action pattern is carried out in the extended neural network.     

Back-propagation neural networks for identification and control of a direct drive robot

A neural approach is proposed to estimate parameters in dynamics of a direct drive robot. Before the estimation, the input-output data for identification are generated in a sequential and term-by-term manner first. Then a two-layer neural network for parameter identification is proposed, in which the back-propagation training method is used to adjust the weights between neurons. The goal is to find the weights that minimize the root-mean-square error between the identification data and output of the network. With the estimated dynamics, existing trajectory-tracking algorithms, such as the well-known computed-torque method, can then be applied to make the robot move along a desired trajectory.     

Structure optimization of neural networks for evolutionary design optimization

We study the use of neural networks as approximate models for the fitness evaluation in evolutionary design optimization. To improve the quality of the neural network models, structure optimization of these networks is performed with respect to two different criteria: One is the commonly used approximation error with respect to all available data, and the other is the ability of the networks to learn different problems of a common class of problems fast and with high accuracy. Simulation results from turbine blade optimizations using the structurally optimized neural network models are presented to show that the performance of the models can be improved significantly through structure optimization.We would like to thank the BMBF, grant LOKI, number 01 IB 001 C, for their financial support of our research.     

Dynamic numerical reconstruction of a fungal biofiltration system using differential neural network

Biofiltration is an economical and environmentally friendly process to eliminate air pollutants. Results obtained by different authors showed the enhanced performance of the fungal biofiltering systems. Consequently, there is a necessity to develop methodologies not only to design more efficient reactors but to control the reaction behavior under different conditions: pollutants feeding, air flows, humidity and biomass production. In this study, a continuous neural network observer was designed to predict the toluene vapors elimination capacity (EC) in a fungal biofilter. The observer uses the carbon dioxide (CO₂) production and the pressure drop (DP) (on line measurements) as input information. The differential neural network observer proved to be a useful tool to reconstruct the immeasurable on-line variable (EC). The observer was successfully tested under different reaction conditions proving the robustness of estimation process. This software sensor may be helpful to derive adaptive control functions optimizing the biofilter reaction development.     

A neural network based multi-classifier system for gene identification in DNA sequences

The paper presents a neural network based multi-classifier system for the identification of Escherichia coli promoter sequences in strings of DNA. As each gene in DNA is preceded by a promoter sequence, the successful location of an E. coli promoter leads to the identification of the corresponding E. coli gene in the DNA sequence. A set of 324 known E. coli promoters and a set of 429 known non-promoter sequences were encoded using four different encoding methods. The encoded sequences were then used to train four different neural networks. The classification results of the four individual neural networks were then combined through an aggregation function, which used a variation of the logarithmic opinion pool method. The weights of this function were determined by a genetic algorithm. The multi-classifier system was then tested on 159 known promoter sequences and 171 non-promoter sequences not contained in the training set. The results obtained through this study proved that the same data set, when presented to neural networks in different forms, can provide slightly varying results. It also proves that when different opinions of more classifiers on the same input data are integrated within a multi-classifier system, we can obtain results that are better than the individual performances of the neural networks. The performances of our multi-classifier system outperform the results of other prediction systems for E. coli promoters developed so far.

New exponentially convergent state estimation method for delayed neural networks

The problem of designing a globally exponentially convergent state estimator for a class of delayed neural networks is investigated in this paper. The time-delay pattern is quite general and including fast time-varying delays. The activation functions are monotone nondecreasing with known lower and upper bounds. A linear estimator of Luenberger-type is developed and by properly constructing a new Lyapunov–Krasovskii functional coupled with the integral inequality, the global exponential stability conditions of the error system are derived. The unknown gain matrix is determined by solving a delay-dependent linear matrix inequality. The developed results are shown to be less conservative than previously published ones in the literature, which is illustrated by a representative numerical example.     

Development of moving window state and parameter estimators under maximum likelihood and Bayesian frameworks

Estimation of slowly varying model parameters/unmeasured disturbances is of paramount importance in process monitoring, fault diagnosis, model based advanced control and online optimization. The conventional approach to estimate drifting parameters is to artificially model them as a random walk process and estimate them simultaneously with the states. However, this may lead to a poorly conditioned problem, where the tuning of the random walk model becomes a non-trivial exercise. In this work, the moving window parameter estimator of Huang et al. [1] is recast as a moving window maximum likelihood (ML) estimator. The state can be estimated within the window using any recursive Bayesian estimator. It is assumed that, when the model parameters are perfectly known, the innovation sequence generated by the chosen Bayesian estimator is a Gaussian white noise process and is further used to construct a likelihood function that treats the model parameters as unknowns. This leads to a well conditioned problem where the only tuning parameter is the length of the moving window, which is much easier to select than selecting the covariance of the random walk model. The ML formulation is further modified to develop a maximum a posteriori (MAP) cost function by including arrival cost for the parameter. Efficacy of the proposed ML and MAP formulations has been demonstrated by conducting simulation studies and experimental evaluation. Analysis of the simulation and experimental results reveals that the proposed moving window ML and MAP estimators are capable of tracking the drifting parameters/unmeasured disturbances fairly accurately even when the measurements are available at multiple rates and with variable time delays.     

Design of state estimators for uncertain linear systems using quadratic boundedness

The notion of quadratic boundedness, which allows one to address the stability of a dynamic system in the presence of bounded disturbances, is applied to the design of state estimators for discrete-time linear systems with polytopic uncertainties. Necessary and sufficient stability conditions are stated and upper bounding sequences on the estimation error are derived. For the purpose of design, such conditions can be expressed in terms of linear matrix inequalities (LMIs), thus guaranteeing the numerical tractability. Simulation results are reported to show the effectiveness of the approach.     

Event prediction algorithm using neural networks for the power management system of electric vehicles

The power management system for electronic vehicles selectively activates Electronic Control Units (ECUs) in the electronic control system according to time-series vehicle data and predefined operation states. However, at an operation state transition, the energy overheads used for the selective ECU activation could be higher than the energy saved by deactivating ECUs. To prevent these energy-inefficient state transitions, we apply two main ideas to our proposed algorithm: (A) unacceptable state transitions and (B) adaptive training speed. For the unacceptable transitions, our energy model evaluates the breakeven time where energy saving equals to energy overheads. Based on the breakeven time, our algorithm classifies training dataset as unacceptable and acceptable event sets. Especially when the algorithm trains neural networks for the two event sets, the adaptive training speed expedites its training speed based on a history of training errors. Consequently, without violating in-vehicle time constraints, the algorithm could provide real-time predictions and save energy overheads by avoiding unacceptable transitions. In the simulation results on real driving datasets, our algorithm improves the energy dissipation of the electronic control system by 5% to 7%.     

Aspects of neural modeling of multidimensional chaotic attractors

This article discusses various aspects of neural modeling of multidimensional chaotic attractors. The Lorenz and Rosler attractors are considered as representative cases and are thoroughly examined. These two dynamical systems are expressed, within acceptable accuracy limits, by the corresponding systems of difference equations. Initially, the complete neural models of the attractors are examined. In this case, the neural networks are supplied with the values X_n , Y_n , Z_n of the systems to predict all the next components (X_{n +1}, Y_{n +1}, and Z_{n +1}) of the attractors. In the second case, named ‘component simulation’, the neural models are trained to predict only one of the values X_n , Y_n , Z_n , when they are fed with the complete input vector as in the first case. In the third case, the proposed neural networks are trained to predict only one component (X_{n +1}, Y_{n +1}, or Z_{n +1}) of the attractors, given a number of past values of the same component. Finally, the ability of the networks to predict the Y and Z components of an X time series of the dynamical systems is examined. Since the response of some networks is not satisfactory, the distribution of absolute error is considered in order to form a realistic picture of the networks’ performance.     

An expert system based on wavelet decomposition and neural network for modeling Chua’s circuit

This paper presents an expert system based on wavelet decomposition and neural network for modeling and simulation of Chua’s circuit which is used for chaos studies. The problems which arise in modeling Chua’s circuit by neural networks are high structural complexity and slow and difficult training. With this proposed method a new solutions is produced to solve these problems. Wavelet decomposition is used for new useful feature extracting from input signal and neural network is used for modeling. Test results of proposed wavelet decomposition and neural network model are compared with test results of neural network model. Desired performance is provided by this new model. Test results showed that the suggested method can be used efficiently for modeling nonlinear dynamical systems.     

Artificial neural networks and bankruptcy forecasting: a state of the art

The use of neural networks in finance began by the end of the 1980s and by the beginning of the 1990s, it developed specific applications related to forecasting on the failure of companies. In order to highlight the evolution of this research stream, we have retained and analysed 30 studies in which the authors use neural networks to solve companies’ classification problems (healthy and failing firms). This review of all these works gives us the opportunity to stress upon future trends in bankruptcy forecasting research.

Delay-dependent exponential state estimators for stochastic neural networks of neutral type with both discrete and distributed delays

This paper considered the state estimation for stochastic neural networks of neutral type with discrete and distributed delays. By using available output measurements, the state estimator can approximate the neuron states, and the asymptotic property of the state error is mean square exponential stable and also almost surely exponential stable in the presence of discrete and distributed delays. Under the Lipschitz assumptions for the activation functions and the measurement nonlinearity, a delay-dependent linear matrix inequality (LMI) criterion is proposed to guarantee the existence of the desired estimators by constructing an appropriate Lyapunov-Krasovskii function. It is shown that the existence conditions and the explicit expression of the state estimator can be parameterised in terms of the solution to a LMI. Finally, two numerical examples are presented to demonstrate the validity of the theoretical results and show that the theorem can provide less conservative conditions.     

Influence of the optimization methods on neural state estimation quality of the drive system with elasticity

The paper deals with the implementation of optimized neural networks (NNs) for state variable estimation of the drive system with an elastic joint. The signals estimated by NNs are used in the control structure with a state-space controller and additional feedbacks from the shaft torque and the load speed. High estimation quality is very important for the correct operation of a closed-loop system. The precision of state variables estimation depends on the generalization properties of NNs. A short review of optimization methods of the NN is presented. Two techniques typical for regularization and pruning methods are described and tested in detail: the Bayesian regularization and the Optimal Brain Damage methods. Simulation results show good precision of both optimized neural estimators for a wide range of changes of the load speed and the load torque, not only for nominal but also changed parameters of the drive system. The simulation results are verified in a laboratory setup.     

Design of artificial neural networks for tool wear monitoring

An on-line scheme for tool wear monitoring using artificial neural networks (ANNs) has been proposed. Cutting velocity, feed, cutting force and machining time are given as inputs to the ANN, and the flank wear is estimated using the ANN. Different ANN structures are designed and investigated to estimate the tool wear accurately. An existing analytical model is used to obtain the data for various cutting conditions in order to eliminate the huge cost and time associated with generation of training and evaluation data. Motivated by the fact that the tool wear at a given instance of time depends on the tool wear value at a previous instance of time, memory is included in the ANN. ANNs without memory, with one-phase memory, and with two-phase memory are investigated in this study. The effect of various training parameters, such as learning coefficient, momentum, temperature, and number of hidden neurons, on these architectures is studied. The findings and experience obtained should facilitate the design and implementation of reliable and economical real-time systems for tool wear monitoring and identification in intelligent manufacturing.     

Evaluation of neural network variable influence measures for process control

Decision-making frequently involves identifying how to change input parameters in a given process in order to effect a directed change in the process output. Artificial neural networks have been used extensively to model business and manufacturing processes and there are several existing neural network-based influence measures that allow a decision-maker to assess the relative impact of each variable on process performance. The purpose of this paper is to review those neural network-based measures of variable influence, and to identify the combination of those measures that results in a comprehensive approach to characterizing variable influence within a trained neural network model. We then demonstrate how this comprehensive approach can be used as a tool to guide decision makers in dynamic process control.     

Application of neural networks to temperature control in thermal power plants

In a thermal power plant with once-through boilers, it is important to control the temperature at the middle point where water becomes steam. However, there are many problems in the design of such a control system, due to a long system response delay, dead-zone and saturation of the actuator mechanisms, uncertainties in the system model and/or parameters, and process noise. To overcome these problems, an adaptive controller has been designed using neural networks, and tested extensively via simulations.

One of the key problems in designing such a controller is to develop an efficient training algorithm. Neural networks are usually trained using the output errors of the network, instead of using the output errors of the controlled plant. However, when a neural network is used to control a plant directly, the output errors of the network are unknown, since the desired control actions are unknown. This paper proposes a simple training algorithm for a class of nonlinear systems, which enables the neural network to be trained with the output errors of the controlled plant. The only a priori knowledge of the controlled plant is the direction of its output response. Due to its simple structure and algorithm, and good performance, the proposed controller has high potential for handling difficult problems in process-control systems.     

A genetic algorithm-based artificial neural network model for the optimization of machining processes

Artificial intelligent tools like genetic algorithm, artificial neural network (ANN) and fuzzy logic are found to be extremely useful in modeling reliable processes in the field of computer integrated manufacturing (for example, selecting optimal parameters during process planning, design and implementing the adaptive control systems). When knowledge about the relationship among the various parameters of manufacturing are found to be lacking, ANNs are used as process models, because they can handle strong nonlinearities, a large number of parameters and missing information. When the dependencies between parameters become noninvertible, the input and output configurations used in ANN strongly influence the accuracy. However, running of a neural network is found to be time consuming. If genetic algorithm-based ANNs are used to construct models, it can provide more accurate results in less time. This article proposes a genetic algorithm-based ANN model for the turning process in manufacturing Industry. This model is found to be a time-saving model that satisfies all the accuracy requirements.     

Adaptive neural control of MIMO nonlinear state time-varying delay systems with unknown dead-zones and gain signs

In this paper, adaptive neural control is proposed for a class of uncertain multi-input multi-output (MIMO) nonlinear state time-varying delay systems in a triangular control structure with unknown nonlinear dead-zones and gain signs. The design is based on the principle of sliding mode control and the use of Nussbaum-type functions in solving the problem of the completely unknown control directions. The unknown time-varying delays are compensated for using appropriate Lyapunov-Krasovskii functionals in the design. The approach removes the assumption of linear functions outside the deadband as an added contribution. By utilizing the integral Lyapunov function and introducing an adaptive compensation term for the upper bound of the residual and optimal approximation error as well as the dead-zone disturbance, the closed-loop control system is proved to be semi-globally uniformly ultimately bounded. Simulation results demonstrate the effectiveness of the approach.