Automatic training of a min-max neural network for function approximation by using a second feed forward network

This research is concerned with a gradient descent training algorithm of a min-max network which we will refer to as the target network. Training makes use of a helper feed-forward network (FFN) to represent the performance function used in training the target network. A helper FFN is trained because the mathematical form of the performance function for the target network in terms of its trainable parameters, p, is not differentiable. Values for the parameter vector, p, of the target network are generated randomly and performance values are determined to produce the data for training the helper FFN with its own connection matrices. Thus we find an approximation to the mathematical relationship between performance values and p by training an FFN. The input to this FFN is a value for p and the output is a performance measure. The transfer function of the helper FFN provides a differentiable function for the performance function of the parameter vector, p, for the target network allowing gradient search methods for finding the optimum p for the target network. The method is successfully tried in approximating a given function and also on training data produced by a randomly selected min-max network.