神经网络BP学习算法动力学分析

研究神经网络BP学习算法与微分动力系统的关系.指出BP学习算法的迭代式与相 应的微分动力系统数值解Euler方法在一定条件下等价,且二者在解的渐近性方面是一致的. 给出了神经网络BP学习算法与相应的微分动力系统解的存在性、唯一性定理和微分动力系统 的零解稳定性定理.从理论上证明了神经网络的学习在一定条件下与微分动力系统的数值方法 所得的数值解在渐近意义下是等价的,从而借助于微分动力系统的数值方法可以解决神经网络 的学习问题.最后给出了用改进Euler方法训练BP网的例子.

DYNAMIC ANALYSIS OF BP ALGORITHM FOR NEURAL NETWORKS

This paper deals with the relationship between BP algorithm for neural networks and differential dynamic systems. It is proposed that the iteration formula of BP algorithm is equivalent to Euler method of differential dynamic system under certain conditions, and the asymptotic solutions of the two formulas are consistent. For neural networks' BP algorithm and their corresponding differential dynamics, the solution existence theorem, exclusiveness theorem and zero solution stability theorem of dynamic system are presented. It is also theoretically proved that asymptotic solutions given by neural networks' BP algorithm are equivalent to that computed by any numerical method for differential dynamic systems under certain conditions. Therefore, training a neural network can be converted to computing numerical solution of differential dynamic systems. Also, an example to train the BP network by modified Euler method is presented.