分布式在线交替方向乘子法

针对如何对分布式网络采集的数据进行在线学习的问题,提出了一种基于交替方向乘子法(ADMM)的分布式在线学习优化算法--分布式在线交替方向乘子法(DOM)。首先,针对分布式在线学习需要各节点根据新采集的数据来更新本地估计,同时保持网络中所有节点的估计趋于一致这一问题,建立了数学模型并设计DOM算法对其进行求解。其次,针对分布式在线学习问题定义了Regret 界,用以表征在线估计的性能;证明了当本地即时损失函数是凸函数时,DOM算法是收敛的,并给出了其收敛速度。最后,通过数值仿真实验结果表明,相比现有的分布式在线梯度下降法(DOGD)和分布式在线自主学习算法(DAOL),所提出的DOM算法具有更快的收敛性能。

Decentralized online alternating direction method of multipliers

Aiming at the problem of learning streaming data collected by a decentralized network in an online manner, a decentralized online learning algorithm - Decentralized Online alternating direction method of Multipliers (DOM) was proposed based on the Alternating Direction Method of Multipliers (ADMM). Firstly, observing that decentralized online learning required each node to update its local iterate based on new local data while keeping the estimation of all the nodes converged to a consensual iterate, a mathematical model was developed and solved by DOM. Secondly, a Regret bound of decentralized online learning was defined to evaluate performance of online estimation. DOM was proved to be convergent when the instantaneous local cost functions were convex, and the convergence rate was given. Finally, the results of numerical experiments show that, compared with existing algorithms such as Distributed Online Gradient Descent (DOGD) and Distributed Autonomous Online Learning (DAOL), the proposed algorithm DOM has the fastest convergence rate.