An online fault tolerant actor-critic neuro-control for a class of nonlinear systems using neural network HJB approach

In this paper, we propose an actor-critic neuro-control for a class of continuous-time nonlinear systems under nonlinear abrupt faults, which is combined with an adaptive fault diagnosis observer (AFDO). Together with its estimation laws, an AFDO scheme, which estimates the faults in real time, is designed based on Lyapunov analysis. Then, based on the designed AFDO, a fault tolerant actor- critic control scheme is proposed where the critic neural network (NN) is used to approximate the value function and the actor NN updates the fault tolerant policy based on the approximated value function in the critic NN. The weight update laws for critic NN and actor NN are designed using the gradient descent method. By Lyapunov analysis, we prove the uniform ultimately boundedness (UUB) of all the states, their estimation errors, and NN weights of the fault tolerant system under the unpredictable faults. Finally, we verify the effectiveness of the proposed method through numerical simulations.     

Adaptive output feedback stabilisation for planar nonlinear systems with unknown control coefficients

The paper is concerned with the global adaptive stabilisation via output feedback for a class of uncertain planar nonlinear systems. Remarkably, the unknowns in the systems are rather serious: the control coefficients are unknown constants which do not belong to any known interval, and the growth of the systems heavily depends on the unmeasured states and has the rate of unknown polynomial of output. First, a delicate state transformation is introduced to collect the unknown control coefficients, and subsequently, a suitable state observer is successfully designed with two different dynamic gains. Then, an adaptive output feedback controller is proposed by flexibly combining the universal control idea and the backstepping technique. Meanwhile, an appropriate estimation law is constructed to overcome the negative effect caused by the unknown control coefficients. It is shown that, with the appropriate choice of the design parameters, all the states of the resulting closed-loop system are globally bounded, and furthermore, the states of the original system converge to zero.     

Hebb-Hopfield neural networks based on one-dimensional sets of neuron states

Neural Networks (NN), which interconnection matrix is the Hebb matrix of Hopfield (HH) [2,3] are considered. Quasi-continuos sets of neuron states are being used for network matrix production. It is shown, that in this case minima of Hopfield energy are at the bottom of deep ditches, corresponding to the basic set of network activity states for the HH NN. The corresponding states can be made to be stable states of the network. When neuron threshold fatigue is introduced, depending of its recent activity state, the network activity becomes cyclic, moving with a constant rate in one of the two possible directions in the ring, depending on the initial conditions. The phenomena described present novel robust types of NN behavior, which have a high probability to be encountered in living neural systems.     

基于参数估计误差的辊道窑温度场优化控制方法

辊道窑烧结过程的温度是决定锂离子电池正极材料产品质量的关键. 然而, 根据炉内有限个测温点的温度 建立起描述整个温度场的模型往往非常困难, 导致无法优化控制烧结过程的温度分布; 而控制方法的设计一般需要 进行参数估计, 已有参数估计方法大多依赖于观测器/预测器的状态误差信息, 无法直接反映待估计参数的变化特 征且方法的准确性取决于观测器/预测器的性能. 为此, 本文提出一种基于参数估计误差的温度场自适应动态规划 (adaptive dynamic programming, ADP)优化控制方法. 首先, 基于传热机理建立二维多孔介质能量守恒方程, 构建包 含角系数的边界条件以反映热辐射作用; 考虑到竖直方向温度变化较大, 通过转换边界条件建立起辊道窑一维温 度场模型, 并根据正极材料的特性获得模型参数. 然后, 采用ADP中的策略迭代(policy iteration, PI) 优化设计温度场 控制器, 神经网络(neural network, NN)用于PI中的评价网络以逼近代价函数; 基于权值参数的估计值与真实值之差 构建参数估计误差, 通过将估计误差的信息融入到评价NN参数更新过程, 提出基于参数估计误差的NN权值更新算 法, 以提高参数估计误差的收敛性, 实现有限时间内NN权值的快速收敛. 最后, 通过仿真验证所提建模和控制方法 的有效性.     

Reinforcement learning-based output feedback control of nonlinear systems with input constraints.

A novel neural network (NN)-based output feedback controller with magnitude constraints is designed to deliver a desired tracking performance for a class of multi-input and multi-output (MIMO) strict feedback nonlinear discrete-time systems. Reinforcement learning is proposed for the output feedback controller, which uses three NNs: 1) an NN observer to estimate the system states with the input-output data, 2) a critic NN to approximate certain strategic utility function, and 3) an action NN to minimize both the strategic utility function and the unknown dynamics estimation errors. Using the Lyapunov approach, the uniformly ultimate boundedness (UUB) of the state estimation errors, the tracking errors and weight estimates is shown.     

Discrete-time neural network output feedback control of nonlinear discrete-time systems in non-strict form

An adaptive neural network (NN)-based output feedback controller is proposed to deliver a desired tracking performance for a class of discrete-time nonlinear systems, which are represented in non-strict feedback form. The NN backstepping approach is utilized to design the adaptive output feedback controller consisting of: (1) an NN observer to estimate the system states and (2) two NNs to generate the virtual and actual control inputs, respectively. The non-causal problem encountered during the control design is overcome by using a dynamic NN which is constructed through a feedforward NN with a novel weight tuning law. The separation principle is relaxed, persistency of excitation condition (PE) is not needed and certainty equivalence principle is not used. The uniformly ultimate boundedness (UUB) of the closed-loop tracking error, the state estimation errors and the NN weight estimates is demonstrated. Though the proposed work is applicable for second order nonlinear discrete-time systems expressed in non-strict feedback form, the proposed controller design can be easily extendable to an nth order nonlinear discrete-time system.     

NEURAL NETWORK‐BASED OPTIMAL ADAPTIVE TRACKING USING GENETIC ALGORITHMS

This paper presents the use of neural networks (NNs) and genetic algorithms (GAs) to enhance the output tracking performance of partly known robotic systems. Two of the most potential approaches of adaptive control, i.e., the concept of variable structure control (VSC) and NN‐based adaptive control, are ingeniously combined using GAs to achieve high‐performance output tracking. GA is used to make the maximum use of different performance characteristics of two self‐adaptive NN modules by finding the switching function which best combines them. The method will be valid for any rigid revolute robot system. Computer simulations on our active binocular head are included for illustration and verification.     

Bayesian hybrid state estimation for unequal-length batch processes with incomplete observations

This paper investigates state estimation problem for batch processes with unequal-length batches as well as incomplete observations. A Bayesian hybrid state estimation method is proposed based on two dimensional (2D) correlations of states. The states of equal-length segment of time are estimated according to both within-a-batch and batch-to-batch correlations, and the states of unequal-length segment are obtained according to the correlations within the batch. In this way, the batch process states can be achieved in both equal-length and unequal-length situations, of which the latter one is a more general case. In order to approximate state distribution of nonlinear system and to deal with the problem of incomplete observations, particle filter (PF) is employed. The proposed method shows its superiority with a nonlinear system and a gas-phase reaction process. Compared to a typical existing method, the proposed method provides better estimation accuracy in the situation of equal-length batches, also it shows less sensitivity to incomplete observations.     

Neural networks for the recognition and pose estimation of 3D objects from a single 2D perspective view

In this paper we present a neural network (NN) based system for recognition and pose estimation of 3D objects from a single 2D perspective view. We develop an appearance based neural approach for this task. First the object is represented in a feature vector derived by a principal component network. Then a NN classifier trained with Resilient backpropagation (Rprop) algorithm is applied to identify it. Next pose parameters are obtained by four NN estimators trained on the same feature vector. Performance on recognition and pose estimation for real images under occlusions are shown. Comparative studies with two other approaches are carried out.     

Neural-network-observer-based optimal control for unknown nonlinear systems using adaptive dynamic programming

In this paper, an observer-based optimal control scheme is developed for unknown nonlinear systems using adaptive dynamic programming (ADP) algorithm. First, a neural-network (NN) observer is designed to estimate system states. Then, based on the observed states, a neuro-controller is constructed via ADP method to obtain the optimal control. In this design, two NN structures are used: a three-layer NN is used to construct the observer which can be applied to systems with higher degrees of nonlinearity and without a priori knowledge of system dynamics, and a critic NN is employed to approximate the value function. The optimal control law is computed using the critic NN and the observer NN. Uniform ultimate boundedness of the closed-loop system is guaranteed. The actor, critic, and observer structures are all implemented in real-time, continuously and simultaneously. Finally, simulation results are presented to demonstrate the effectiveness of the proposed control scheme.     

A parameter optimization method for radial basis function type models

This paper considers the nonlinear systems modeling problem for control. A structured nonlinear parameter optimization method (SNPOM) adapted to radial basis function (RBF) networks and an RBF network-style coefficients autoregressive model with exogenous variable model parameter estimation is presented. This is an off-line nonlinear model parameter optimization method, depending partly on the Levenberg-Marquardt method for nonlinear parameter optimization and partly on the least-squares method using singular value decomposition for linear parameter estimation. When compared with some other algorithms, the SNPOM accelerates the computational convergence of the parameter optimization search process of RBF-type models. The usefulness of this approach is illustrated by means of several examples.     

Self-Adaptive Output Tracking with Applications to Active Binocular Tracking

In this article we present a neurally-inspired self-adaptive active binocular tracking scheme and an efficient mathematical model for online computation of desired binocular-head trajectories. The self-adaptive neural network (NN) model is general and can be adopted in output tracking schemes of any partly known robotic systems. The tracking scheme ingeniously combines the conventional Resolved Velocity Control (RVC) technique and an adaptive compensating NN model constructed using SoftMax basis functions as nonlinear activation function. Desired trajectories to the servo controller are computed online by the use of a suitable linear kinematics mathematical model of the system. Online weight tuning algorithm guarantees tracking with small errors and error rates as well as bounded NN weights.     

Neural-network control of mobile manipulators

In this paper, a neural network (NN)-based methodology is developed for the motion control of mobile manipulators subject to kinematic constraints. The dynamics of the mobile manipulator is assumed to be completely unknown, and is identified online by the NN estimators. No preliminary learning stage of NN weights is required. The controller is capable of disturbance-rejection in the presence of unmodeled bounded disturbances. The tracking stability of the closed-loop system, the convergence of the NN weight-updating process and boundedness of NN weight estimation errors are all guaranteed. Experimental tests on a 4-DOF manipulator arm illustrate that the proposed controller significantly improves the performance in comparison with conventional robust control.     

A neural network controller for systems with unmodeled dynamicswith applications to wastewater treatment

This paper considers the use of neural networks (NN's) in controlling a nonlinear, stochastic system with unknown process equations. The approach here is based on using the output error of the system to train the NN controller without the need to assume or construct a separate model (NN or other type) for the unknown process dynamics. To implement such a direct adaptive control approach, it is required that connection weights in the NN be estimated while the system is being controlled. As a result of the feedback of the unknown process dynamics, however, it is not possible to determine the gradient of the loss function for use in standard (backpropagation-type) weight estimation algorithms. In principle, stochastic approximation algorithms in the standard (Kiefer-Wolfowitz) finite-difference form can be used for this weight estimation since they are based on gradient approximations from available system output errors. However, these algorithms will generally require a prohibitive number of observed system outputs. Therefore, this paper considers the use of a new stochastic approximation algorithm for this weight estimation, which is based on a "simultaneous perturbation" gradient approximation. It is shown that this algorithm can greatly enhance the efficiency over more standard stochastic approximation algorithms based on finite-difference gradient approximations. The approach is illustrated on a simulated wastewater treatment system with stochastic effects and nonstationary dynamics.     

Modeling of first-order photobleaching kinetics using Krylov subspace spectral methods

We solve the first order 2-D reaction–diffusion equations which describe binding-diffusion kinetics using the photobleaching scanning profile of a confocal laser scanning microscope, approximated by a Gaussian laser profile. We show how to solve the first-order photobleaching kinetics partial differential equations (PDEs) using a time-stepping method known as a Krylov subspace spectral (KSS) method. KSS methods are explicit methods for solving time-dependent variable-coefficient partial differential equations. They approximate Fourier coefficients of the solution using Gaussian quadrature rules in the spectral domain. In this paper, we show how a KSS method can be used to obtain not only an approximate numerical solution, but also an approximate analytical solution when using initial conditions that come from pre-bleach steady states and also general initial conditions, to facilitate asymptotic analysis. Analytical and numerical results are presented. It is observed that although KSS methods are explicit, it is possible to use a time step that is far greater than what the CFL condition would indicate.     

Identification of tool and product effects in a mixed product and parallel tool environment

In the semiconductor manufacturing industry, production resembles an automated assembly line in which many similar products with slightly different specifications are manufactured step-by-step, with each step being a complicated physiochemical batch process performed by a number of tools. This constitutes a high-mix production system for which effective run-to-run control (RtR) and fault detection control (FDC) can be carried out only if the states of different tools and different products can be estimated. However, since in each production run, a specific product is performed on a specific tool, absolute individual states of products and tools are not observable. In this work, a novel state estimation method based on analysis of variance (ANOVA) is developed to estimate the relative states of each product and tool to the grand average performance of this station in the fab. The method is formulated in the form of a recursive state estimation using the Kalman filter. The advantages of this method are demonstrated using simulations to show that the correct relative states can be estimated in production scenarios such as tool-shift, tool-drift, product ramp-up, tool/product-offline and preventive maintenance (PM). Furthermore, application of this state estimation method in RtR control scheme shows that substantial improvements in process capabilities can be gained, especially for products with small lot counts. The proposed algorithm is also evaluated by an industrial application.     

Robust adaptive neural network control of uncertain nonholonomic systems with strong nonlinear drifts

In this paper, robust adaptive neural network (NN) control is presented to solve the control problem of nonholonomic systems in chained form with unknown virtual control coefficients and strong drift nonlinearities. The robust adaptive NN control laws are developed using state scaling and backstepping. Uniform ultimate boundedness of all the signals in the closed-loop are guaranteed, and the system states are proven to converge to a small neighborhood of zero. The control performance of the closed-loop system is guaranteed by appropriately choosing the design parameters. The proposed adaptive NN control is free of control singularity problem. An adaptive control based switching strategy is used to overcome the uncontrollability problem associated with x0 (t0) = 0. The simulation results demonstrate the effectiveness of the proposed controllers.     

Dynamic optimal learning rates of a certain class of fuzzy neuralnetworks and its applications with genetic algorithm

The stability analysis of the learning rate for a two-layer neural network (NN) is discussed first by minimizing the total squared error between the actual and desired outputs for a set of training vectors. The stable and optimal learning rate, in the sense of maximum error reduction, for each iteration in the training (back propagation) process can therefore be found for this two-layer NN. It has also been proven in this paper that the dynamic stable learning rate for this two-layer NN must be greater than zero. Thus it Is guaranteed that the maximum error reduction can be achieved by choosing the optimal learning rate for the next training iteration. A dynamic fuzzy neural network (FNN) that consists of the fuzzy linguistic process as the premise part and the two-layer NN as the consequence part is then illustrated as an immediate application of our approach. Each part of this dynamic FNN has its own learning rate for training purpose. A genetic algorithm is designed to allow a more efficient tuning process of the two learning rates of the FNN. The objective of the genetic algorithm is to reduce the searching time by searching for only one learning rate, which is the learning rate of the premise part, in the FNN. The dynamic optimal learning rates of the two-layer NN can be found directly using our innovative approach. Several examples are fully illustrated and excellent results are obtained for the model car backing up problem and the identification of nonlinear first order and second order systems.     

Kalman filtering approach to multi-rate information fusion in the presence of irregular sampling rate and variable measurement delay

State estimation for a system with irregular rate and delayed measurements is studied using fusion Kalman filter. Lab data in process plants is usually more accurate compared to other measurements. However, it is often slow rate and subject to variable delay and irregularity in sampling time. Fast rate state estimation can be conducted using fast rate measurement, while the slow rate lab data can be used to improve the accuracy of estimation whenever it is available. For this purpose, two Kalman filters are used to estimate the states based on each type of measurement. The estimates are fused in the next step by considering the correlation between them. An iterative algorithm to obtain the cross-covariance matrix between the estimation errors of the two Kalman filters is presented and employed in the fusion process. The improvement on the accuracy of estimation and comparison with other optimal fusion state estimation techniques are discussed through a simulation example, a pilot-scale experiment and an industrial case study.     

Estimating degradation model parameters using neighborhood pattern distributions: an optimization approach

Noise models are crucial for designing image restoration algorithms, generating synthetic training data, and predicting algorithm performance. There are two related but distinct estimation scenarios. The first is model calibration, where it is assumed that the input ideal bitmap and the output of the degradation process are both known. The second is the general estimation problem, where only the image from the output of the degradation process is given. While researchers have addressed the problem of calibration of models, issues with the general estimation problems have not been addressed in the literature. In this paper, we describe a parameter estimation algorithm for a morphological, binary, page-level image degradation model. The inputs to the estimation algorithm are 1) the degraded image and 2) information regarding the font type (italic, bold, serif, sans serif). We simulate degraded images using our model and search for the optimal parameter by looking for a parameter value for which the local neighborhood pattern distributions in the simulated image and the given degraded image are most similar. The parameter space is searched using a direct search optimization algorithm. We use the p-value of the Kolmogorov-Smirnov test as the measure of similarity between the two neighborhood pattern distributions. We show results of our algorithm on degraded document images.