通过最小二乘拟合方法删除神经网络的隐节点

提出了一种基于隐节点输出行向量的最小二乘拟合来删除神经网络隐节点的方法。它分成两步,首先分析隐节点输出行向量的正交投影间的关系,通过最小二乘拟合找出可以最准确地被其它隐节点输出行向量表达的隐节点输出行向量,然后将该隐节点的作用利用最小二乘拟合系数进行纵向传播分摊到其它隐节点。最后删除该隐节点,并进行再训练,实验结果表明再训练所需时间很短,因而它的实用性很强。

Removal of Hidden Neurons Based on Least Square Approximation

A method of removing hidden neurons based on least square approximation among hidden layer output vectors is presented. It is divided into two stages, first, by analyzing the relations of orthogonal projections among the hidden layer output vectors, the hidden neuron which can be best expressed by the other hidden output vectors is found. Then based on the coefficients of the least square approximation, the influence of the removed hidden neuron is distributed to other hidden neurons. Finally the new network is re-trained with the initial values of the current weights and thresholds. Experimental results show that the re-training time is pretty short, and hence it is very practical.