基于自适应动量优化算法的正则化极限学习机

针对极限学习机(extreme learning machine,ELM)隐节点不确定性导致的系统不稳定,以及对大型数据计算负担过重的问题,提出了基于自适应动量优化算法(adaptive and momentum method,AdaMom)的正则化极限学习机.算法主要思想是构造连续可微的目标函数,在梯度下降过程中计算自适应学习率,求自适应学习率与梯度乘积的指数加权平均值,通过迭代得到损失函数最小值对应的隐层输出权重矩阵.实验结果表明,在相同基准数据集的训练中,AdaMom-ELM算法具有非常良好的泛化性能和鲁棒性,提高了计算效率.

Adaptive and momentum method for regularized extreme learning machine

Aiming at the system instability caused by the uncertainty of the extreme learning machine(ELM) hidden nodes and the problem of overburdening large data calculations, this paper proposed an optimization algorithm based on the adaptive and momentum method(AdaMom). This algorithm constructed a continuously differentiable objective function, and calculated the adaptive learning rate during the gradient descent process, obtained the exponentially weighted average of the product of the adaptive learning rate and the gradient. At last it obtained the hidden layer output weight matrix corresponding to the minimum value of the loss function through iteration. The experimental results show that in the training of the same benchmark data set, the AdaMom-ELM algorithm has very good generalization performance and robustness, which improves the computational efficiency.