| An exploration of the uncertainty relation satisfied by BP network learning ability and generalization ability |
| |
| 引用本文： | LI Zuoyong1 & PENG Lihong2 1. Chengdu University of Information Technology,Chengdu 610041,China, 2. Environmental Science Research Center,Xiamen University,Xiamen 361005,China Correspondence should be addressed to Peng Lihong (email: lhpeng@xmu.edu.cn). An exploration of the uncertainty relation satisfied by BP network learning ability and generalization ability[J]. 中国科学F辑(英文版), 2004, 47(2): 137-150. DOI: 10.1360/02yf0331 |
| |
| 作者姓名： | LI Zuoyong1 & PENG Lihong2 1. Chengdu University of Information Technology Chengdu 610041 China 2. Environmental Science Research Center Xiamen University Xiamen 361005 China Correspondence should be addressed to Peng Lihong (email: lhpeng@xmu.edu.cn) |
| |
| 基金项目： | 国家自然科学基金，the National Science and Technology Program of the Ninth Five-Year Plan |
| |
| 摘 要： | An overfit phenomenon exists in the BP network. The so-called overfit means that as long as the network is allowed to be sufficiently complicated, the BP network can minimize the error of the training sample set; however, in the case of a limited number of samples, the generalization ability of the network will decrease. This indicates that there is a relation between the learning ability and the generalization ability. Therefore, studying the relationship between the learning ability is the…
|
An exploration of the uncertainty relation satisfied by BP network learning ability and generalization ability |
| |
| Li Zuoyong,Peng Lihong. An exploration of the uncertainty relation satisfied by BP network learning ability and generalization ability[J]. Science in China(Information Sciences), 2004, 47(2): 137-150. DOI: 10.1360/02yf0331 |
| |
| Authors: | Li Zuoyong Peng Lihong |
| |
| Affiliation: | 1. Chengdu University of Information Technology, Chengdu 610041, China
2. Environmental Science Researn Center,Xiamen University,Xiamen 361005,China |
| |
| Abstract: | This paper analyses the intrinsic relationship between the BP network learning ability and generalization ability and other influencing factors when the overfit occurs, and introduces the multiple correlation coefficient to describe the complexity of samples; it follows the calculation uncertainty principle and the minimum principle of neural network structural design, provides an analogy of the general uncertainty relation in the information transfer process, and ascertains the uncertainty relation between the training relative error of the training sample set, which reflects the network learning ability, and the test relative error of the test sample set, which represents the network generalization ability; through the simulation of BP network overfit numerical modeling test with different types of functions, it is ascertained that the overfit parameter q in the relation generally has a span of 7 × 10−3 to 7 × 10−2; the uncertainty relation then helps to obtain the formula for calculating the number of hidden nodes of a network with good generalization ability under the condition that multiple correlation coefficient is used to describe sample complexity and the given approximation error requirement is satisfied; the rationality of this formula is verified; this paper also points out that applying the BP network to the training process of the given sample set is the best method for stopping training that improves the generalization ability. |
| |
| Keywords: | BP network learning ability generalization ability overfit relation network structure optimiza-tion. |
| 本文献已被 CNKI 万方数据 SpringerLink 等数据库收录！ |
