New directions in distributed Nash equilibrium seeking based on monotone operator theory

There has been an increasing research interest in modeling, optimization and control of various multi-agent networks that have wide applications in industry, defense, security, and social areas, such as computing clusters, interconnected micro-grid systems, unmanned vessel swarms \cite{ChenJie}, power systems\cite{MeiS}, multiple UAV systems\cite{KolaricP} and sensor networks\cite{LiuR}. For non-cooperative agents that only concern selfish profit-maximizing, the decision making problem can be modelled and solved with the help of game theory, while Nash equilibrium (NE) seeking is at the core to solve the non-cooperative multi-agent games \cite{HenrikSandberg, IsraelAlvarez, Jong-ShiPang}. Distributed NE seeking methods are appealing compared with the center-based methods in large-scale networks due to its scalability, privacy protection, and adaptability. Recently, monotone operator theory is explored for distributed NE seeking, which is shown to provide an uniform framework for various algorithms in different scenarios. It has been gradually developing into a cutting-edge research field, with the prospect and necessity of future in-depth research. In non-cooperative multi-agent games, each agent has different characteristics and pursues maximizing its own benefit. Hence, there is no centralized manager that can force all agents to adopt specified strategies to optimize the overall benefits. Under the NE, no player can decrease its cost by unilaterally changing its local decision to another feasible one. To seek an NE, the agent is required to optimize its own objective function given the opponent''s countermeasures. Therefore, various optimization-based methods have been investigated for distributed NE seeking, such as the gradient flow method and the best response method....     

Game theoretical applications for multi-agent systems

We consider game-theoretic principles for design of cooperative and competitive (non-cooperative self-interested) multi-agent systems. Using economic concepts of tâtonnement and economic core, we show that cooperative multi-agent systems should be designed in games with dominant strategies that may lead to social dilemmas. Non-cooperative multi-agent systems, on the other hand, should be designed for games with no clear dominant strategies and high degree of problem complexity. Further, for non-cooperative multi-agent systems, the number of learning agents should be carefully managed so that solutions in the economic core can be obtained. We provide experimental results for the design of cooperative and non-cooperative MAS from telecommunication and manufacturing industries.     

基于切换拓扑的多智能体协作控制研究综述

多智能体系统一直是众多学科领域研究的主要研究对象,基于切换拓扑的多智能体协作控制理论研究作为多智能体系统研究的重要部分,一直是近年来的热点。为了推进基于切换拓扑的多智能体协作控制理论研究,在广泛调研现有文献和最新成果的基础上,从一致性问题、分布式优化问题和分布式估计问题三个方面对该领域的发展现状进行了总结;探讨了诸如一致性协议的设计、一致性协议的性能分析方法及其优缺点、分布式优化的实现方式和分布式估计的实际应用。最后指出当前该领域尚未解决的问题和未来的研究方向。     

A game theoretic approach for the distributed control of multi-agent systems under directed and time-varying topology

This paper presents a new game theory based method to control multi-agent systems under directed and time varying interaction topology. First, the sensing/communication matrix is introduced to cope with information sharing among agents, and to provide the minimal information requirement which ensures the system level objective is desirable. Second, different from traditional methods of controlling multi-agent systems, non-cooperative games are investigated to enforce agents to make rational decisions. And a new game model, termed stochastic weakly acyclic game, is developed to capture the optimal solution to the distributed optimization problem for multi-agent systems with directed topology. It is worth noting that the system level objective can be achieved at the points of the corresponding equilibriums of the new game model. The proposed method is illustrated with an example in smart grid where multiple distributed generators are controlled to reach the fair power utilization profile in the game formulation to ensure the aggregated power output are optimal.     

动态博弈框架下的分布式动态优化

为了实时在线求解复杂的大规模动态优化问题,本文基于动态博弈理论提出了一种分布式动态优化方案,滚动合作博弈优化(RCGO).首先基于滚动时域优化框架,该方案将原本复杂的大规模动态优化问题分解为若干简单的小规模局部优化子问题,使得计算复杂度降低从而保证优化求解的实时性.之后本文基于动态博弈提出了分解迭代法求解各局部动态优化子问题,并对RCGO优化方案下系统稳定性进行分析.最后本文选择一个化工过程网络作为仿真案例,基于RCGO方案得到了极大化经济效益下该网络的最优操作.优化结果表明在求解复杂大规模动态优化问题时, RCGO方案较传统的集中式优化方案在由系统经济效益、闭环控制性能及优化求解实时性等组成的综合指标上有较大优势.     

具有切换拓扑的多智能体系统协同控制的差分隐私保护

网络信息技术的不断发展与普及使得各类数据的发布采集变得方便与便捷, 但数据的直接发布势必会造 成个网络信息的泄露和敏感信息的失密, 因此敏感信息的保护成为了各行各业关注的问题. 本文研究了基于固定拓 扑和切换拓扑的多智能体系统协同控制的差分隐私保护问题, 将差分隐私算法与传统平均一致性算法结合, 提出了 具有隐私保护的协同控制算法, 分析了隐私保护算法对分布式协同控制闭环系统稳定性的影响. 基于所提算法, 应 用矩阵论和概率统计对隐私保护协同控制算法的收敛性和隐私性进行理论分析, 该算法可以保护智能个体的数据 隐私, 同时可以使得系统运动实现均方一致. 在系统拓扑结构动态变化的情况下, 本文对该算法的收敛性和隐私性 进行理论分析, 讨论了切换拓扑对隐私保护的影响. 最后的仿真示例验证了理论结果的正确性.     

移动边缘网络中联邦学习效率优化综述

联邦学习(federated learning)将模型训练任务部署在移动边缘设备,参与者只需将训练后的本地模型发送到服务器参与全局聚合而无须发送原始数据,提高了数据隐私性.然而,解决效率问题是联邦学习落地的关键.影响效率的主要因素包括设备与服务器之间的通信消耗、模型收敛速率以及移动边缘网络中存在的安全与隐私风险.在充分调研后,首先将联邦学习的效率优化归纳为通信、训练与安全隐私保护3类.具体来说,从边缘协调与模型压缩的角度讨论分析了通信优化方案;从设备选择、资源协调、聚合控制与数据优化4个方面讨论分析了训练优化方案;从安全与隐私的角度讨论分析了联邦学习的保护机制.其次,通过对比相关技术的创新点与贡献,总结了现有方案的优点与不足,探讨了联邦学习所面临的新挑战.最后,基于边缘计算的思想提出了边缘化的联邦学习解决方案,在数据优化、自适应学习、激励机制和隐私保护等方面给出了创新理念与未来展望.     

基于一般和随机对策论框架下的多智能体学习

将Q-learning从单智能体框架上扩展到非合作的多智能体框架上,建立了在一般和随机对策框架下的多智能体理论框架和学习算法,提出了以Nash平衡点作为学习目标.给出了对策结构的约束条件,并证明了在此约束条件下算法的收敛性,对多智能体系统的研究与应用有重要意义.     

一种虚拟人作业行为的自主优化模型

人体自适应行为仿真是实现人机工程学评估的前提条件.针对已有技术存在的不足,提出了一种基于多Agent合作式博弈的虚拟人作业行为自主优化模型.该模型将工作环境中人体自适应行为定义为一个多目标优化问题,提出了人体工作状态空间和人体行为元素的概念,以实现人体行为的离散化设计了人体行为仿真算法以求解上述模型.算法采用梯度上升的策略来搜索满足模糊多目标Nash谈判条件的人体作业姿态的Pareto最优解.仿真实验表明,该方法可以在缺少相关数据的情况下推导出舒适的人体工作姿态,在工程领域中表现出较好的适用性.     

An Optimal Control-Based Distributed Reinforcement Learning Framework for A Class of Non-Convex Objective Functionals of the Multi-Agent Network

This paper studies a novel distributed optimization problem that aims to minimize the sum of the non-convex objective functionals of the multi-agent network under privacy protection, which means that the local objective of each agent is unknown to others. The above problem involves complexity simultaneously in the time and space aspects. Yet existing works about distributed optimization mainly consider privacy protection in the space aspect where the decision variable is a vector with finite dimensions. In contrast, when the time aspect is considered in this paper, the decision variable is a continuous function concerning time. Hence, the minimization of the overall functional belongs to the calculus of variations. Traditional works usually aim to seek the optimal decision function. Due to privacy protection and non-convexity, the Euler-Lagrange equation of the proposed problem is a complicated partial differential equation. Hence, we seek the optimal decision derivative function rather than the decision function. This manner can be regarded as seeking the control input for an optimal control problem, for which we propose a centralized reinforcement learning (RL) framework. In the space aspect, we further present a distributed reinforcement learning framework to deal with the impact of privacy protection. Finally, rigorous theoretical analysis and simulation validate the effectiveness of our framework.

     

联邦学习通信开销研究综述

为了解决数据共享需求与隐私保护要求之间不可调和的矛盾,联邦学习应运而生.联邦学习作为一种分布式机器学习,其中的参与方与中央服务器之间需要不断交换大量模型参数,而这造成了较大通信开销;同时,联邦学习越来越多地部署在通信带宽有限、电量有限的移动设备上,而有限的网络带宽和激增的客户端数量会使通信瓶颈加剧.针对联邦学习的通信瓶...     

化工企业优化多智能体系统的通信机制与协调规则

分析了用于复杂化工企业生产优化的多智能体系统。采取将大部分运算时间用于各智能体自身的局部优化求解，而只花少量时间将优化任务和局部优化结果通过网络进行交换作为基本准则，提出了适合化工企业多层优化多智能体系统的通信机制与协调规则。     

多Agent协同工作环境MACE

在ＣＡＤ／ＣＡＭ和ＣＩＭＳ等领域的分布协同计算中,分布人工智能领域的多Ａｇｅｎｔ技术已逐步得到越来越多的应用,本文阐述了ＭＡＣＥ（Ｍｕｌｔｉ－ＡｇｅｎｔＣｏｏｐｅｒａｔｉｖｅＥｎｖｉｒｏｎｍｅｎｔ）多Ａｇｅｎｔ协同工作环境中有关Ａｇｎｅｔ的概念,分类和结构,多Ａｇｅｎｔ系统结构,人与人交互界面,公用语言以及运行模式等问题,最后,以一个简单的机械组合件的交互和自动两种方式修改参数的协同设计为实例,论     

Distributed data mining and agents

Multi-agent systems (MAS) offer an architecture for distributed problem solving. Distributed data mining (DDM) algorithms focus on one class of such distributed problem solving tasks—analysis and modeling of distributed data. This paper offers a perspective on DDM algorithms in the context of multi-agents systems. It discusses broadly the connection between DDM and MAS. It provides a high-level survey of DDM, then focuses on distributed clustering algorithms and some potential applications in multi-agent-based problem solving scenarios. It reviews algorithms for distributed clustering, including privacy-preserving ones. It describes challenges for clustering in sensor-network environments, potential shortcomings of the current algorithms, and future work accordingly. It also discusses confidentiality (privacy preservation) and presents a new algorithm for privacy-preserving density-based clustering.     

基于数据分布的聚类联邦学习

联邦学习(federated learning)可以解决分布式机器学习中基于隐私保护的数据碎片化和数据隔离问题。在联邦学习系统中,各参与者节点合作训练模型,利用本地数据训练局部模型,并将训练好的局部模型上传到服务器节点进行聚合。在真实的应用环境中,各节点之间的数据分布往往具有很大差异,导致联邦学习模型精确度较低。为了解决非独立同分布数据对模型精确度的影响,利用不同节点之间数据分布的相似性,提出了一个聚类联邦学习框架。在Synthetic、CIFAR-10和FEMNIST标准数据集上进行了广泛实验。与其他联邦学习方法相比,基于数据分布的聚类联邦学习对模型的准确率有较大提升,且所需的计算量也更少。     

Consensus control of discrete-time multi-agent systems with privacy preservation

Due to the dependence of consensus theory on multi-agent systems (MAS) and the need to openly exchange status information with neighbors on each agent, private status and information are inevitably exposed to the public. In this paper, we present a consensus privacy protection algorithm to solve the consensus privacy problem of second-order MAS. The proposed approach is based on the Paillier algorithm with semi-homomorphic characteristics while combining a distributed control perspective to solve the privacy issues of sharing system status information. Under the conditions of our design, as long as there is at least one trustworthy neighbor node, the privacy of the topology nodes can be protected, and thus, the interconnected nodes are unable to detect the state information of systems from one another. We provide a new method for the privacy of the state information of the discrete-time MAS and use restricted hardware devices to implement the consensus control. Finally, experiments verify the effectiveness of our method, while meeting the actual applicability     

基于足球机器人决策系统的协作模型

运用多智能体系统的思想,提出了一种多智能体协作控制模型,通过对多智能体的规划提高了足球机器人系统决策系统的连贯性,系统利用改进的黑板结构有效地解决了角色分配及通信问题。通过实例分析了模型及其策略在系统协作控制方面的实用性。     

基于博弈论优化的高效联邦学习方案

随着网络信息技术与互联网的发展,数据的隐私与安全问题亟待解决,联邦学习作为一种新型的分布式隐私保护机器学习技术应运而生。针对在联邦学习过程中存在个人数据信息泄露的隐私安全问题,结合Micali-Rabin随机向量表示技术,基于博弈论提出一种具有隐私保护的高效联邦学习方案。根据博弈论激励机制,构建联邦学习博弈模型,通过设置合适的效用函数和激励机制保证参与者的合理行为偏好,同时结合Micali-Rabin随机向量表示技术设计高效联邦学习方案。基于Pedersen承诺机制实现高效联邦学习的隐私保护,以保证联邦学习各参与者的利益和数据隐私,并且全局达到帕累托最优状态。在数字分类数据集上的实验结果表明,该方案不仅提高联邦学习的通信效率,而且在通信开销和数据精确度之间实现平衡。     

具有外部干扰的二阶多自主体系统的协同运动

为解决多自主体系统在群集运动过程受到外部干扰影响的问题,本文研究了具有外部干扰的二阶多自主体系统的分布式协同控制.本文中的外部干扰包括匹配干扰和不匹配干扰,针对系统中的匹配干扰,设计了状态观测器和干扰观测器,对系统的未知状态和干扰进行估计,并且构造了基于干扰观测器的多自主体协同控制算法.对于系统中的不匹配干扰,设计了与匹配干扰不同的干扰观测器,构造了基于主动抗干扰观测器的协同控制算法.运用矩阵论和现代控制理论等方法,研究了基于干扰观测器的二阶多自主体系统的协同控制.应用计算机仿真分别验证在多自主体系统具有匹配干扰和不匹配干扰的情况下结论的有效性,仿真结果表明,本文所设计的多自主体协同控制算法可以使跟随者最终都收敛到领导者的状态,实现了具有匹配干扰和不匹配干扰的二阶多自主体系统的状态一致性.     

A survey on distributed online optimization and online games

Distributed online optimization and online games have been increasingly researched in the last decade, mostly motivated by their wide applications in sensor networks, robotics (e.g., distributed target tracking and formation control), smart grids, deep learning, and so forth. In these problems, there is a network of agents which interact with each other in a collaborative manner (i.e., distributed online optimization) or noncooperative manner (i.e., online games) through local information exchanges. And the local cost function of each agent is time-varying in dynamic and adversarial environments. At each time, a decision must be made by each agent based on historical information at hand without knowing its future cost functions. For these problems, a comprehensive survey is still lacking. This paper aims to provide a thorough overview of distributed online optimization and online games from the perspective of problem settings, algorithms, communication and computation requirements, and performances. In addition, some potential future directions are also discussed.