Stable reinforcement learning with recurrent neural networks

In this paper, we present a technique for ensuring the stability of a large class of adaptively controlled systems. We combine IQC models of both the controlled system and the controller with a method of filtering control parameter updates to ensure stable behavior of the controlled system under adaptation of the controller. We present a specific application to a system that uses recurrent neural networks adapted via reinforcement learning techniques. The work presented extends earlier works on stable reinforcement learning with neural networks. Specifically, we apply an improved IQC analysis for RNNs with time-varying weights and evaluate the approach on more complex control system.