Suboptimal control for nonlinear slow-fast coupled systems using reinforcement learning and Takagi–Sugeno fuzzy methods

In this article, by using singular perturbation theory, reinforcement learning (RL), and Takagi–Sugeno (T-S) fuzzy methods, a RL-fuzzy-based composite suboptimal control method is proposed for nonlinear slow-fast coupled systems (SFCSs) with unknown slow dynamics. First, the SFCSs is decomposed into slow and fast subsystems and the original optimal control problem is reduced to two subproblems. Then, for the slow subsystem, a nonlinear coordinate transformation is introduced to transform the nonquadratic slow utility function into the quadratic form. Unmeasurable virtual slow subsystem state is reconstructed by the state measurements of original system and slow controller design algorithm is proposed in the framework of RL by utilizing the actor-critic neural networks to approximate the controller and cost function. For the fast subsystem, T-S fuzzy model is established and state measurements of the original system are exploited to reconstruct the unmeasurable fast subsystem state. Fast controller is designed with the approach of parallel distributed compensation. The obtained slow and fast controllers form the composite suboptimal controller for the original SFCSs. Considering the state reconstruction error, convergence of the slow controller design algorithm, suboptimality of the composite controller, and stability of the closed-loop SFCSs are analyzed. Finally, the effectiveness of our proposed method is illustrated by examples.