权值初始化与激励函数调整相结合的学习算法

提出了一种基于独立元分析(ICA)方法的权值初始化方法和动态调整S型激励函数的斜率相结合的神经网络学习算法。该方法利用ICA从输入数据中提取显著的特征信息来初始化输入层到隐含层权值。而且通过使神经网络的输出位于激励函数的活动区域,对隐含层到输出层的权值进行初始化。在学习过程中,再对每个隐单元和输出单元的激励函数的斜率进行自动调整。最后通过计算机仿真实际的基准问题,验证了论文提出的方法的有效性。实验结果表明,所提出的方法能有效地加快多层前向神经网络的训练过程。

Learning Algorithm based on the Combination of Weight Initialization and Activation Function Adjustment

A novel learning algorithm is proposed that is based on the combination of independent component analysis(ICA)based weight initialization and automatically adjusting the gain parameter of sigmoid activation function.The algorithm is able to initialize the weights from input layer to hidden layer that extract the salient feature components from the input data.The initial weights from hidden layer to output layer are evaluated in such a way that the output neurons are kept inside the active region.In the process of learning,the each neuron's gain parameter of the activation function is dynamically tuned.The real-world benchmark problems are used for validating the proposed algorithms.The simulation results show that the proposed algorithm is able to speed up the learning process of feedforward neural network effectively.