A successive projection method for binary pattern recognition with multilayer feedforward neural networks

Error back-propagation (BP) is one of the most popular ideas used in learning algorithms for multilayer neural networks. In BP algorithms, there are two types of learning schemes, online learning and batch learning. The online BP has been applied to various problems in practice, because of its simplicity of implementation. However, efficient implementation of the online BP usually requires an ad hoc rule for determining the learning rate of the algorithm. In this paper, we propose a new learning algorithm called SPM, which is derived from the successive projection method for solving a system of nonlinear inequalities. Although SPM can be regarded as a modification of online BP, the former algorithm determines the learning rate (step-size) adoptively based on the output for each input pattern. SPM may also be considered a modification of the globally guided back-propagation (GGBP) proposed by Tang and Koehler. Although no theoretical proof of the convergence for SPM is given, some simulation results on pattern classification problems indicate that SPM is more effective and robust than the standard online BP and GGBP