Deterministic convergence of conjugate gradient method for feedforward neural networks |
| |
| Authors: | Jian Wang Wei Wu Jacek M Zurada |
| |
| Affiliation: | 1. School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, PR China;2. Department of Electrical and Computer Engineering, University of Louisville, Louisville, KY 40292, USA;3. School of Mathematics and Computational Sciences, China University of Petroleum, Dongying 257061, PR China;1. Automation Department, Technical University of Cluj-Napoca, Memorandumului 28, 400114 Cluj-Napoca, Romania;2. Université de Lorraine, CRAN, UMR 7039, 2 av. Forêt de Haye, Vand?uvre-lès-Nancy, France;3. CNRS, CRAN, UMR 7039, 2 av. Forêt de Haye, Vand?uvre-lès-Nancy, France;1. College of Computer Science & Technology, Zhejiang University, Hangzhou 310027, PR China;2. Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USA;3. Department of Mathematics, Zhejiang University, Hangzhou 310027, PR China;1. State Key Lab. of Integrated Services Networks, Xidian University, Xi?an, Shaanxi, China;2. Department of Electronic Information Engineering, Nanchang University, Nanchang, Jianxi, 330031, China;3. National Lab. of Radar Signal Processing, Xidian University, Xi?an, Shaanxi, China;4. Luoyang Institute of Electro Optical Equipment of AVIC, Luoyang, Henan, China;5. Institute of Systems Science and Control Engineering, Hangzhou Dianzi University, Hangzhou, Zhejiang, China;1. Graduate School of Science and Engineering, Ritsumeikan University Shiga 525-8577, Japan;2. State Key Laboratory of CAD & CG, Zhejiang University, Hangzhou 310058, China;3. Hitachi (China) Research & Development Corporation, Shanghai 200020, China;1. School of Information Science and Technology, Xiamen University, Xiamen 361005, China;2. Fujian Key Laboratory of the Brain-like Intelligent Systems (Xiamen University), Xiamen 361005, China;3. Cognitive Science Department, Xiamen University, Xiamen 361005, China;4. School of Information Management, Hubei University of Economics, Hubei 430205, China;1. Control and Simulation Center, Harbin Institute of Technology, Harbin 150001, China;2. Department of Control Science and Engineering, Harbin Institute of Technology, Harbin 150001, China;3. Space Control and Inertial Technology Research Center, Harbin Institute of Technology, Harbin 150001, China |
| |
| Abstract: | Conjugate gradient methods have many advantages in real numerical experiments, such as fast convergence and low memory requirements. This paper considers a class of conjugate gradient learning methods for backpropagation neural networks with three layers. We propose a new learning algorithm for almost cyclic learning of neural networks based on PRP conjugate gradient method. We then establish the deterministic convergence properties for three different learning modes, i.e., batch mode, cyclic and almost cyclic learning. The two deterministic convergence properties are weak and strong convergence that indicate that the gradient of the error function goes to zero and the weight sequence goes to a fixed point, respectively. It is shown that the deterministic convergence results are based on different learning modes and dependent on different selection strategies of learning rate. Illustrative numerical examples are given to support the theoretical analysis. |
| |
| Keywords: | |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
