Boundedness and Convergence of Online Gradient Method with Penalty for Linear Output Feedforward Neural Networks |
| |
Authors: | Huisheng Zhang Wei Wu |
| |
Affiliation: | (1) Department of Applied Mathematics, Dalian University of Technology, Dalian, 116024, People’s Republic of China;(2) Department of Mathematics, Dalian Maritime University, Dalian, 116026, People’s Republic of China |
| |
Abstract: | This paper investigates an online gradient method with penalty for training feedforward neural networks with linear output. A usual penalty is considered, which is a term proportional to the norm of the weights. The main contribution of this paper is to theoretically prove the boundedness of the weights in the network training process. This boundedness is then used to prove an almost sure convergence of the algorithm to the zero set of the gradient of the error function. |
| |
Keywords: | Feedforward neural networks Linear output Online gradient method Penalty Boundedness Convergence |
本文献已被 SpringerLink 等数据库收录! |
|