基于神经网络的强化学习算法研究

BP神经网络在非线性控制系统中被广泛运用，但作为有导师监督的学习算法，要求批量提供输入输出对神经网络训练，而在一些并不知道最优策略的系统中，这样的输入输出对事先并无法得到，另一方面，强化学习从实际系统学习经验来调整策略，并且是一个逼近最优策略的过程，学习过程并不需要导师的监督。提出了将强化学习与BP神经网络结合的学习算法-RBP模型。该模型的基本思想是通过强化学习控制策略，经过一定周期的学习后再用学到的知识训练神经网络，以使网络逐步收敛到最优状态。最后通过实验验证了该方法的有效性及收敛性。

RESEARCH ON A REINFORCEMENT LEARNING ALGORITHM BASED ON NEURAL NETWORK

BP neural network has been used in nonlinear system controller widely. But as a supervised training algorithm, it requires the input-output pairs to be trained. But in some systems such input-output pairs cannot be received under the optimal control policy. On the other hand, reinforcement learning (RL) learns behavior through trial-and-error interaction with a dynamic environment. It is unsupervised and on-line. This paper provides the RBP model which adapts the BP network to be used in RL. The main idea of RBP is: RL learns optimal policy from the environment and stores the policy into the network. Instead of updating weights instantly, network weights are updated in batch mode periodically. A simple example is used to illustrate the validity of the algorithm.