正则化超限学习机的多分块松弛交替方向乘子法

针对超限学习机在大数据环境下计算负担过重的问题,文中提出正则化超限学习机的多分块松弛交替方向乘子法及N-等分和N/2-等分情形的标量化实现.模型分块使算法具有高度的并行结构,与松弛技术结合提高算法的收敛速度.通过分析,建立算法收敛的充要条件,给出最优收敛率及最优参数.在基准数据集上仿真计算收敛率随分块数的变化关系,对比不同算法的收敛速率和GPU加速比.实验表明,文中算法具有较低的计算复杂度和较高的并行性.

Multi-partition Relaxed Alternating Direction Method of Multipliers for Regularized Extreme Learning Machine

To address the issue of overly heavy computational load of extreme learning machine(ELM) in the big data environment, parallel optimization for ELM is studied. A multi-partition relaxed alternating direction method of multipliers(ADMM) for regularized ELM along with two scalarwise implementations in the N- and N/2-equipartition cases is proposed. By the multi-partition, the proposed algorithm has a highly parallel structure and the combination with relaxation technique improves the convergence rate of the proposed algorithm. Through analysis, a necessary and sufficient convergence condition is established, and optimal convergence ratio and optimal parameters are obtained. Through simulations on bench-mark datasets, the relationship between the convergence ratio and the number of partitioned blocks is calculated, and convergence rates and GPU acceleration ratios of different algorithms are compared. Experimental results demonstrate that the proposed algorithm has low computational complexity and high parallelism.