A sum-of-product neural network (SOPNN)

This paper presents a sum-of-product neural network (SOPNN) structure. The SOPNN can learn to implement static mapping that multilayer neural networks and radial basis function networks normally perform. The output of the neural network has the sum-of-product form ∑Npi=1∏Nvj=1 fij (xj), where xj's are inputs, Nv is the number of inputs, fij( ) is a function generated through network training, and Np is the number of product terms. The function fij(xj) can be expressed as ∑kwijkBjk(xj), where Bjk( ) is a single-variable basis function and Wijk's are weight values. Linear memory arrays can be used to store the weights. If Bjk( ) is a Gaussian function, the new neural network degenerates to a Gaussian function network. This paper focuses on the use of overlapped rectangular pulses as the basis functions. With such basis functions, WijkBjk(xj) will equal either zero or Wijk, and the computation of fij(xj) becomes a simple addition of some retrieved Wijk's. The structure can be viewed as a basis function network with a flexible form for the basis functions. Learning can start with a small set of submodules and have new submodules added when it becomes necessary. The new neural network structure demonstrates excellent learning convergence characteristics and requires small memory space. It has merits over multilayer neural networks, radial basis function networks and CMAC in function approximation and mapping in high-dimensional input space. The technique has been tested for function approximation, prediction of a time series, learning control, and classification.