Neural dynamic optimization for control systems.II. Theory

The paper presents neural dynamic optimization (NDO) as a method of optimal feedback control for nonlinear multi-input-multi-output (MIMO) systems. The main feature of NDO is that it enables neural networks to approximate the optimal feedback solution whose existence dynamic programming (DP) justifies, thereby reducing the complexities of computation and storage problems of the classical methods such as DP. This paper mainly describes the theory of NDO, while the two other companion papers of this topic explain the background for the development of NDO and demonstrate the method with several applications including control of autonomous vehicles and of a robot arm, respectively.     

Neural dynamic optimization for control systems. I. Background

The paper presents neural dynamic optimization (NDO) as a method of optimal feedback control for nonlinear multi-input-multi-output (MIMO) systems. The main feature of NDO is that it enables neural networks to approximate the optimal feedback solution whose existence dynamic programming (DP) justifies, thereby reducing the complexities of computation and storage problems of the classical methods such as DP. This paper mainly describes the background and motivations for the development of NDO, while the two other subsequent papers of this topic present the theory of NDO and demonstrate the method with several applications including control of autonomous vehicles and of a robot arm, respectively.     

Design of a class of new nonlinear disturbance observers based on tracking differentiators for uncertain dynamic systems

The main contribution of this paper is to present a general design method of new nonlinear disturbance observer (NDO) based on tracking differentiator (TD) for uncertain dynamic system. The stability and convergence of the proposed NDO can be guaranteed by TD. This new NDO can be used to estimate many types of uncertain disturbances, and can overcome the disadvantages of existing NDOs that need the priori information concerning the upper and lower bounds of the disturbance and its ith derivative’s Lipschitz upper bound. It can be also applied in uncertain dynamic system for various purposes such as disturbance estimate and compensation, solving the problem of control input constraint, and reducing even eliminating chattering of control input. Simulation results are presented to show the effectiveness of the developed NDO.     

Approximation-based disturbance observer approach for adaptive tracking of uncertain pure-feedback nonlinear systems with unmatched disturbances

This paper presents an approximation-based nonlinear disturbance observer (NDO) methodology for adaptive tracking of uncertain pure-feedback nonlinear systems with unmatched external disturbances. Compared with existing control results using NDO for nonlinear systems in lower-triangular form, the major contribution of this study is to develop an NDO-based control framework in the presence of non-affine nonlinearities and disturbances unmatched in the control input. An approximation-based NDO scheme is designed to attenuate the effect of compounded disturbance terms consisting of external disturbances, approximation errors and control coefficient nonlinearities. The function approximation technique using neural networks is employed to estimate the unknown nonlinearities derived from the recursive design procedure. Based on the designed NDO scheme, an adaptive dynamic surface control system is constructed to ensure that all signals of the closed-loop system are semi-globally uniformly ultimately bounded and the tracking error converges to a neighbourhood of the origin. Simulation examples including a mechanical system are provided to show the effectiveness of the proposed theoretical result.     

Optimal prescribed performance tracking control of nonlinear motor driven systems via adaptive dynamic programming

Although optimal regulation problem has been well studied, resolving optimal tracking control via adaptive dynamic programming (ADP) has not been completely resolved, particularly for nonlinear uncertain systems. In this paper, an online adaptive learning method is developed to realize the optimal tracking control design for nonlinear motor driven systems (NMDSs), which adopts the concept of ADP, unknown system dynamic estimator (USDE), and prescribed performance function (PPF). To this end, the USDE in a simple form is first proposed to address the NMDSs with bounded disturbances. Then, based on the estimated unknown dynamics, we define an optimal cost function and derive the optimal tracking control. The derived optimal tracking control is divided into two parts, that is, steady-state control and optimal feedback control. The steady-state control can be obtained with the tracking commands directly. The optimal feedback control can be obtained via the concept of ADP based on the PPF; this contributes to improving the convergence of critic neural network (CNN) weights and tracking accuracy of NMDSs. Simulations are provided to display the feasibility of the designed control method.     

基于自适应模糊控制器和非线性扰动观测器的永磁直线同步电机反馈线性化控制

由于永磁直线同步电机(PMLSM)伺服系统应用于一些高精密场合,因此克服系统存在的负载扰动、参数变化等不确定性影响是提高系统性能的关键.针对不确定性问题,采用一种基于自适应模糊控制器(AFC)和非线性扰动观测器(NDO)的反馈线性化控制方法.首先设计反馈线性化控制器(FLC)实现系统的线性化,便于位置跟踪;其次采用NDO估计并补偿系统的不确定性,提高跟踪精度.但在实际运行过程中观测器增益较难选取,极易产生较大的观测误差,为此,采用AFC方法逼近NDO的观测误差,通过自适应律动态调整模糊规则,改善模糊控制器的学习能力,增强系统的鲁棒性,并用李雅普诺夫定理保证系统闭环稳定性.实验结果表明,与基于DOB和NDO的反馈线性化位置控制相比,该方法能够明显提高系统的跟踪性和鲁棒性.     

A robust constrained control approach for flexible air‐breathing hypersonic vehicles

This article investigates the robust adaptive control system design for the longitudinal dynamics of a flexible air‐breathing hypersonic vehicle (FAHV) subject to parametric uncertainties and control input constraints. A combination of back‐stepping and nonlinear disturbance observer (NDO) is utilized for exploiting an adaptive output‐feedback controller to provide robust tracking of velocity and altitude reference trajectories in the presence of flexible effects and system uncertainties. The dynamic surface control is introduced to solve the problem of “explosion of terms.” A new NDO is developed to guarantee the proposed controller's disturbance attenuation ability and to performance robustness against uncertain aerodynamic coefficients. To deal with the problem of actuator saturation, a novel auxiliary system is exploited to compensate the desired control laws. The stability of the presented NDO and controller is analyzed. Simulation results are given to demonstrate the effectiveness of the presented control strategy.     

Parallel Control for Optimal Tracking via Adaptive Dynamic Programming

This paper studies the problem of optimal parallel tracking control for continuous-time general nonlinear systems. Unlike existing optimal state feedback control, the control input of the optimal parallel control is introduced into the feedback system. However, due to the introduction of control input into the feedback system, the optimal state feedback control methods can not be applied directly. To address this problem, an augmented system and an augmented performance index function are proposed firstly. Thus, the general nonlinear system is transformed into an affine nonlinear system. The difference between the optimal parallel control and the optimal state feedback control is analyzed theoretically. It is proven that the optimal parallel control with the augmented performance index function can be seen as the suboptimal state feedback control with the traditional performance index function. Moreover, an adaptive dynamic programming (ADP) technique is utilized to implement the optimal parallel tracking control using a critic neural network (NN) to approximate the value function online. The stability analysis of the closed-loop system is performed using the Lyapunov theory, and the tracking error and NN weights errors are uniformly ultimately bounded (UUB). Also, the optimal parallel controller guarantees the continuity of the control input under the circumstance that there are finite jump discontinuities in the reference signals. Finally, the effectiveness of the developed optimal parallel control method is verified in two cases.      

基于ELM的永磁直线同步电机位移跟踪动态面反步滑模控制

针对永磁直线同步电机(PMLSM)易受到参数摄动、负载扰动等不确定因素的影响,进而影响其位移跟踪控制精度的问题,提出一种基于非线性干扰观测器(NDO)和极限学习机(ELM)的动态面反步滑模控制方法.首先,通过构造NDO对系统模型中的非匹配不确定项进行动态观测,并将反步控制、动态面控制与滑模控制相结合,完成PMLSM位移跟踪控制器的设计,在提高系统抗干扰能力的同时,避免常规反步控制中的“微分爆炸”问题;其次,采用ELM神经网络对系统模型中的匹配不确定项进行逼近估计,并将输出的估计值引入设计的动态面反步滑模控制器中进行补偿;再次,采用人工鱼群-蛙跳混合算法对所设计控制器的主要参数进行优化设计,提高系统的收敛速度和稳定精度;最后,将所提出控制方法与其他控制方法进行仿真对比,仿真结果表明了所提出方法的有效性.     

Integrated connection-level and packet-level QoS controls over wireless mesh networks

This paper focuses on integrating connection-level and packet-level QoS controls over wireless mesh network (WMN) to support applications with diverse QoS performance requirements. At the connection-level, the dynamic guard based prioritized connection admission control (DG-PCAC) provides prioritized admission with relative connection blocking probabilities and end-to-end deterministic minimum bandwidth allocation guarantees. DG-PCAC is enabled by dynamic guard based logical link configuration controls (LCCs), which provides relative differentiated capacity limits for prioritized admission classes. At the packet-level, the optimal rate delay scheduler (ORDS) dynamically allocates link bandwidth to the admitted flows of prioritized traffic classes; with the objective to minimize deviation from relative delay targets with minimum bandwidth guarantees according to traffic classes. Two realizations of the ORDS are presented, namely optimal scheduling policy via dynamic programming (DP) algorithm, and neural network (NN) based heuristic to alleviate computational complexity. Performance results show that the DG-PCAC enables consistent performance guarantees under non-stationary arrivals of connection requests. Performance results also show that the performance of the NN based scheduling heuristic approaches to that of the DP based optimal ORDS scheme.     

Neural networks applied to optimal flight control

This paper presents a method for developing control laws for nonlinear systems based on an optimal control formulation. Due to the nonlinearities of the system, no analytical solution exists. The method proposed here uses the ‘black box’ structure of a neural network to model a feedback control law. The network is trained with the back-propagation learning method by using examples of optimal control produced with a differential dynamic programming technique. Two different optimal control problems from flight control are studied. The produced control laws are simulated and the results analyzed. Neural networks show promise for application to optimal control problems with nonlinear systems.     

Approximation-based adaptive control for a class of mobile robots with unknown skidding and slipping

An adaptive tracking control approach using function approximation technique is proposed for trajectory tracking of Type (2,0) wheeled mobile robots with unknown skidding and slipping in polar coordinates and at the dynamic level. The nonlinear disturbance observer (NDO) is used to estimate a nonlinear disturbance term including unknown skidding and slipping. The adaptive control system is designed via the function approximation technique using neural networks employed to compensate the NDO error. It is proved that all signals of the controlled closed-loop system are uniformly bounded and the point tracking errors converge to an adjustable neighborhood of the origin regardless of large initial tracking errors and unknown skidding and slipping. Simulation results are presented to validate the good tracking performance and robustness of the proposed control system against unknown skidding and slipping.     

On the design of optimal output feedback control systems†

For a linear control system with quadratic performance index the optimal control takes a feedback form of all state variables. However, if there are some states which are not fed in the control system, it is impossible to obtain the optimal feedback control by using the usual mathematical optimization technique such as dynamic programming or the maximum principle.

This paper presents the optimal control of output feedback systems for a quadratic performance index by using a new parameter optimization technique.

Since the optimal feedback gains depend on the initial states in the output feedback control system, two cases where (1) the initial states are known, and (2) the statistical properties of initial states such as mean and covariance matrices are known, are considered here. Furthermore, the proposed method for optimal output feedback control is also applied to sampled-data systems.     

Neural Network Based Algorithm for Dynamic System Optimization

A class of artificial neural networks with a two‐layer feedback topology to solve nonlinear discrete dynamic optimization problems is developed. Generalized recurrent neuron models are introduced. A direct method to assign the weights of neural networks is presented. The method is based on Bellmann's Optimality Principle and on the interchange of information which occurs during the synaptic chemical processing among neurons. A comparative analysis of the computational requirements is made. The analysis shows advantages of this approach as compared to the standard dynamic programming algorithm. The technique has been applied to several important optimization problems, such as shortest path and control optimal problems.     

Subgradient-based feedback neural networks for non-differentiable convex optimization problems

1 Introduction Optimization problems arise in a broad variety of scientific and engineering applica- tions. For many practice engineering applications problems, the real-time solutions of optimization problems are mostly required. One possible and very pr…     

Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints

In this paper, the near-optimal control problem for a class of nonlinear discrete-time systems with control constraints is solved by iterative adaptive dynamic programming algorithm. First, a novel nonquadratic performance functional is introduced to overcome the control constraints, and then an iterative adaptive dynamic programming algorithm is developed to solve the optimal feedback control problem of the original constrained system with convergence analysis. In the present control scheme, there are three neural networks used as parametric structures for facilitating the implementation of the iterative algorithm. Two examples are given to demonstrate the convergence and feasibility of the proposed optimal control scheme.     

Neural network solution for finite-horizon H-infinity constrained optimal control of nonlinear systems

In this paper, neural networks are used to approximately solve the finite-horizon constrained input H-infinity state feedback control problem. The method is based on solving a related Hamilton-Jacobi-Isaacs equation of the corresponding finite-horizon zero-sum game. The game value function is approximated by a neural network with time- varying weights. It is shown that the neural network approximation converges uniformly to the game-value function and the resulting almost optimal constrained feedback controller provides closed-loop stability and bounded L2 gain. The result is an almost optimal H-infinity feedback controller with time-varying coefficients that is solved a priori off-line. The effectiveness of the method is shown on the Rotational/Translational Actuator benchmark nonlinear control problem.     

IMPROVED CONVERGENCE THROUGH DYNAMIC PROGRAMMING APPROACH

New hybrid methods for solving the multiplayer perceptron optimization problem are proposed which use the computation capabilities of Bellman's dynamic programming (DP) method. To solve the neural network optimization problem, we consider the case of output neurons differently from that of hidden neurons. For the neurons of the output layer we apply the conventional DP and for the hidden neurons we apply a method based on gradient approach. Computer simulation shows that the new hybrid methods outperform the gradient-based optimization methods in converging speed and avoiding the local minimum.     

Neural networks for feedback feedforward nonlinear control systems

This paper deals with the problem of designing feedback feedforward control strategies to drive the state of a dynamic system (in general, nonlinear) so as to track any desired trajectory joining the points of given compact sets, while minimizing a certain cost function (in general, nonquadratic). Due to the generality of the problem, conventional methods are difficult to apply. Thus, an approximate solution is sought by constraining control strategies to take on the structure of multilayer feedforward neural networks. After discussing the approximation properties of neural control strategies, a particular neural architecture is presented, which is based on what has been called the "linear-structure preserving principle". The original functional problem is then reduced to a nonlinear programming one, and backpropagation is applied to derive the optimal values of the synaptic weights. Recursive equations to compute the gradient components are presented, which generalize the classical adjoint system equations of N-stage optimal control theory. Simulation results related to nonlinear nonquadratic problems show the effectiveness of the proposed method.     

Stability and monotone convergence of generalised policy iteration for discrete-time linear quadratic regulations

In this paper, we analyse the convergence and stability properties of generalised policy iteration (GPI) applied to discrete-time linear quadratic regulation problems. GPI is one kind of the generalised adaptive dynamic programming methods used for solving optimal control problems, and is composed of policy evaluation and policy improvement steps. To analyse the convergence and stability of GPI, the dynamic programming (DP) operator is defined. Then, GPI and its equivalent formulas are presented based on the notation of DP operator. The convergence of the approximate value function to the exact one in policy evaluation is proven based on the equivalent formulas. Furthermore, the positive semi-definiteness, stability, and the monotone convergence (PI-mode and VI-mode convergence) of GPI are presented under certain conditions on the initial value function. The online least square method is also presented for the implementation of GPI. Finally, some numerical simulations are carried out to verify the effectiveness of GPI as well as to further investigate the convergence and stability properties.