有向网络分布式优化的Barzilai-Borwein梯度跟踪方法

本文研究有向网络上的分布式优化问题, 其全局目标函数是网络上所有光滑强凸局部目标函数的平均值.受Barzilai-Borwein步长改善梯度方法表现的启发, 本文提出了一种分布式Barzilai-Borwein梯度跟踪方法. 与文献中使用固定步长的分布式梯度算法不同, 所提出的方法中每个智能体利用其局部梯度信息自动地计算其步长. 通过同时使用行随机和列随机权重矩阵, 该方法避免了由特征向量估计引起的计算和通信. 当目标函数是光滑和强凸函数时, 本文证明了该算法产生的迭代序列可以线性地收敛到最优解. 对分布式逻辑回归问题的仿真结果验证了所提出的算法比使用固定步长的分布式梯度算法表现更好

Barzilai-Borwein gradient tracking method for distributed optimization over directed networks

This paper studies the distributed optimization problem over directed networks. The global objective function of this problem is the average of all smooth and strongly convex local objective functions on the networks. Motivated by the capability of Barzilai-Borwein step sizes in improving the performance of gradient methods, a distributed Barzilai-Borwein gradient tracking method is proposed. Different from the distributed gradient algorithms using fixed step sizes in the literature, the proposed method allows each agent to calculate its step size automatically using its local gradient information. By using row- and column-stochastic weights simultaneously, the method can avoid the computation and communication on eigenvector estimation. It is proved that the iterative sequence generated by the proposed method converges linearly to the optimal solution for smooth and strongly convex functions. Simulation results on the distributed logistic regression problem show that the proposed method performs better than some advanced distributed gradient algorithms with fixed step sizes.