On the selection of a subgoal and the use of a priori information in learning control systems

Numerous methods have been proposed for the design of control systems which learn to function in unknown or partially known environments. Most learning schemes are radical departures from the techniques using continuous adjustment of parameters which grew out of early developments in model reference systems. Principal contributions to the area have been controller models and algorithms. In studying these models, the system is abstracted to such an extent that there is quite often a loss of contact with practical considerations. The objective of this paper is to present some results in the theory of learning control, but also to look again at some of the practical problems encountered in applying a learning controller to a problem.

This paper defines the subgoal as a subordinate to the primary goal of minimizing the performance index. It must evaluate each decision one control interval after it is instituted. The subgoal problem is to choose a subgoal which will direct the learning process to the optimal as prescribed by the given performance index. An analytical solution is presented and extended heuristically for the general case. This extended method makes use of the a priori information about the plant.

Two other problems are also discussed. A fixed grid is used to partition the state space into control situations, and a method of extending the grid is proposed and evaluated. The controller is initialized using the a priori information, too. A full scale simulation confirms that the proposed methods of choosing the subgoal, extending the fixed grid and initializing the controller are improvements over previous methods.