Integrating domain and constraint privacy reasoning in the distributed stochastic algorithm with breakouts

Privacy has traditionally been a major motivation of distributed problem solving. One popular approach to enable privacy in distributed environments is to implement complex cryptographic protocols. In this paper, we propose a different, orthogonal approach, which is to control the quality and the quantity of publicized data. We consider the Open Constraint Programming model and focus on algorithms that solve Distributed Constraint Optimization Problems (DCOPs) using a local search approach. Two such popular algorithms exist to find good solutions to DCOP: DSA and GDBA. In this paper, we propose DSAB, a new algorithm that merges ideas from both algorithms to allow extensive handling of constraint privacy. We also study how algorithms behave when solving Utilitarian DCOPs, where utilitarian agents want to reach an agreement while reducing the privacy loss. We experimentally study how the utilitarian approach impacts the quality of the solution and of publicized data.

     

Leximin Asymmetric Multiple Objective Distributed Constraint Optimization Problem

The Distributed Constraint Optimization Problem (DCOP) lies at the foundations of multiagent cooperation. With DCOPs, the optimization in distributed resource allocation problems is formalized using constraint optimization problems. The solvers for the problem are designed based on decentralized cooperative algorithms that are performed by multiple agents. In a conventional DCOP, a single objective is considered. The Multiple Objective Distributed Constraint Optimization Problem (MODCOP) is an extension of the DCOP framework, where agents cooperatively have to optimize simultaneously multiple objective functions. In the conventional MODCOPs, a few objectives are globally defined and agents cooperate to find the Pareto optimal solution. However, such models do not capture the interests of each agent. On the other hand, in several practical problems, the share of each agent is important. Such shares are modeled as preference values of agents. This class of problems can be defined using the MODCOP on the preferences of agents. In particular, we define optimization problems based on leximin ordering and Asymmetric DCOPs (Leximin AMODCOPs). The leximin defines an ordering among vectors of objective values. In addition, Asymmetric DCOPs capture the preferences of agents. Because the optimization based on the leximin ordering improves the equality among the satisfied preferences of the agents, this class of problems is important. We propose several solution methods for Leximin AMODCOPs generalizing traditional operators into the operators on sorted objective vectors and leximin. The solution methods applied to the Leximin AMODCOPs are based on pseudo trees. Also, the investigated search methods employ the concept of boundaries of the sorted vectors.     

Constructing a unifying theory of dynamic programming DCOP algorithms via the generalized distributive law

In this paper we propose a novel message-passing algorithm, the so-called Action-GDL, as an extension to the generalized distributive law (GDL) to efficiently solve DCOPs. Action-GDL provides a unifying perspective of several dynamic programming DCOP algorithms that are based on GDL, such as DPOP and DCPOP algorithms. We empirically show how Action-GDL using a novel distributed post-processing heuristic can outperform DCPOP, and by extension DPOP, even when the latter uses the best arrangement provided by multiple state-of-the-art heuristics.     

低约束密度分布式约束优化问题的求解算法

多Agent协作过程中的许多挑战都可以建模为分布式约束优化问题.针对低约束密度的分布式约束优化问题,提出了一种基于贪婪和回跳思想的求解算法.在该算法中,各Agent基于贪婪原则进行决策,能够利用低约束密度问题中大量赋值组合代价为0这一特点来加快求解速度.同时,Agent间的回跳机制可以在贪婪原则陷入局部最优时保证算法的完全性.相对于已有主流算法,该算法可以在保持多项式级别的消息长度/空间复杂度的前提下,以较少的消息数目求解低约束密度的分布式约束优化问题.给出了算法关键机制的正确性证明,并通过实验验证了算法的上述性能优势.     

基于约束图分片求解DCOP的Agent组织结构

MAS中许多分布式推理问题可以建模为分布式约束优化问题(DCOP),解决DCOP的分布式算法已经成为MAS中的重要基础.已有的Adopt等算法通过对等的Agent之间的平等协商完成求解,强调了异步通信、分布计算与对解质量的保证,在求解问题的组织结构方面仍有改进余地.可以采用一种基于分散与集中相结合的思路,基于对约束图分片的方法及核心结点、通信主干道等概念,构造新颖的Agent组织结构,完成DCOP问题的异步、分布求解.在该组织结构下求解DCOP的算法可在效率、适应动态性方面得到改善,并将一个Agent一个变量和一个Agent多个变量的DCOP求解方法统一起来.     

DCSP和DCOP求解研究进展

分布式约束满足问题（DCSP）和分布式约束最优问题（DCOP）的研究是分布式人工智能领域的基础性工作。本文首先介绍了卿和DCOP的形式化描述及对实际应用问题的建模方法。在DCSP和DCOP的求解中,通常对问题要进行限制和要求,同时要满足分布性、异步性、局部性、完备性的原则。异步回溯（ABT）、异步弱承诺搜索（AWC）和分布式逃逸（DB）算法是求解DCSP的有代表性的算法;DCSP算法对DCOP求解产生了影响,但由DCSP一般化到DCOP的算法,仅适用于解决部分特定的问题,DCOP的最优、异步算法有异步分布式约束最优算法（A—dopt）和最优异步部分交叉算法（OptAPO）。本文讨论了上述算法的性能。相关的研究工作在多局部变量的处理、超约束DCSP、算法性能度量、通信的保密等方面进行了扩充,在对问题本身的研究、建模方法学、算法、与其他方法的结合以及拓展应用领域等方面仍有许多问题需要进一步研究。     

Distributed constraint optimization with MULBS: A case study on collaborative meeting scheduling

This paper introduces MULBS, a new DCOP (distributed constraint optimization problem) algorithm and also presents a DCOP formulation for scheduling of distributed meetings in collaborative environments. Scheduling in CSCWD can be seen as a DCOP where variables represent time slots and values are resources of a production system (machines, raw-materials, hardware components, etc.) or management system (meetings, project tasks, human resources, money, etc). Therefore, a DCOP algorithm must find a set of variable assignments that maximize an objective function taking constraints into account. However, it is well known that such problems are NP-complete and that more research must be done to obtain feasible and reliable computational approaches. Thus, DCOP emerges as a very promising technique: the search space is decomposed into smaller spaces and agents solve local problems, collaborating in order to achieve a global solution. We show with empirical experiments that MULBS outperforms some of the state-of-the-art algorithms for DCOP, guaranteeing high quality solutions using less computational resources for the distributed meeting scheduling task.     

BD-ADOPT: a hybrid DCOP algorithm with best-first and depth-first search strategies

Distributed Constraint Optimization Problem (DCOP) is a promising framework for modeling a wide variety of multi-agent coordination problems. Best-First search (BFS) and Depth-First search (DFS) are two main search strategies used for search-based complete DCOP algorithms. Unfortunately, BFS often has to deal with a large number of solution reconstructions whereas DFS is unable to promptly prune sub-optimal branch. However, their weaknesses will be remedied if the two search strategies are combined based on agents’ positions in a pseudo-tree. Therefore, a hybrid DCOP algorithm with the combination of BFS and DFS, called BD-ADOPT, is proposed, in which a layering boundary is introduced to divide all agents into BFS-based agents and DFS-based agents. Furthermore, this paper gives a rule to find a suitable layering boundary with a new strategy for the agents near the boundary to realize the seamless joint between BFS and DFS strategies. Detailed experimental results show that BD-ADOPT outperforms some famous search-based complete DCOP algorithms on the benchmark problems.     

Balancing exploration and exploitation in incomplete Min/Max-sum inference for distributed constraint optimization

Distributed Constraint Optimization Problems (DCOPs) are NP-hard and therefore the number of studies that consider incomplete algorithms for solving them is growing. Specifically, the Max-sum algorithm has drawn attention in recent years and has been applied to a number of realistic applications. Unfortunately, in many cases Max-sum does not produce high-quality solutions. More specifically, Max-sum does not converge and explores solutions of low quality when run on problems whose constraint graph representation contains multiple cycles of different sizes. In this paper we advance the state-of-the-art in incomplete algorithms for DCOPs by: (1) proposing a version of the Max-sum algorithm that operates on an alternating directed acyclic graph (Max-sum_AD), which guarantees convergence in linear time; (2) solving a major weakness of Max-sum and Max-sum_AD that causes inconsistent costs/utilities to be propagated and affect the assignment selection, by introducing value propagation to Max-sum_AD (Max-sum_ADVP); and (3) proposing exploration heuristic methods that evidently improve the algorithms performance further. We prove that Max-sum_ADVP converges to monotonically improving states after each change of direction, and that it is guaranteed to converge in pseudo-polynomial time to a stable solution that does not change with further changes of direction. Our empirical study reveals a large improvement in the quality of the solutions produced by Max-sum_ADVP on various benchmarks, compared to the solutions produced by the standard Max-sum algorithm, Bounded Max-sum and Max-sum_AD with no value propagation. It is found to be the best guaranteed convergence inference algorithm for DCOPs. The exploration methods we propose for Max-sum_ADVP improve its performance further. However, anytime results demonstrate that their exploration level is not as efficient as a version of Max-sum, which uses Damping.     

基于局部并行搜索的分布式约束优化算法框架

针对当前局部搜索算法在求解大规模、高密度的分布式约束优化问题（DCOP）时,求解困难且难以跳出局部最优取得进一步优化等问题,提出一种基于局部并行搜索的分布式约束优化算法框架（LPOS）,算法中agent通过自身的取值并行地搜索局部所有邻居取值来进一步扩大对解空间的搜索,从而避免算法过早陷入局部最优。为了保证算法的收敛性与稳定性,设计了一种自适应平衡因子K来平衡算法对解的开发和继承能力,并在理论层面证明了并行搜索优化算法可以扩大对解空间的搜索,自适应平衡因子K可以实现平衡目的。综合实验结果表明,基于该算法框架的算法在求解低密度和高密度DCOP时性能都优于目前最新的算法。特别是在求解高密度DCOP中有显著的提升。     

Privacy Loss in Distributed Constraint Reasoning: A Quantitative Framework for Analysis and its Applications

It is critical that agents deployed in real-world settings, such as businesses, offices, universities and research laboratories, protect their individual users’ privacy when interacting with other entities. Indeed, privacy is recognized as a key motivating factor in the design of several multiagent algorithms, such as in distributed constraint reasoning (including both algorithms for distributed constraint optimization (DCOP) and distributed constraint satisfaction (DisCSPs)), and researchers have begun to propose metrics for analysis of privacy loss in such multiagent algorithms. Unfortunately, a general quantitative framework to compare these existing metrics for privacy loss or to identify dimensions along which to construct new metrics is currently lacking. This paper presents three key contributions to address this shortcoming. First, the paper presents VPS (Valuations of Possible States), a general quantitative framework to express, analyze and compare existing metrics of privacy loss. Based on a state-space model, VPS is shown to capture various existing measures of privacy created for specific domains of DisCSPs. The utility of VPS is further illustrated through analysis of privacy loss in DCOP algorithms, when such algorithms are used by personal assistant agents to schedule meetings among users. In addition, VPS helps identify dimensions along which to classify and construct new privacy metrics and it also supports their quantitative comparison. Second, the article presents key inference rules that may be used in analysis of privacy loss in DCOP algorithms under different assumptions. Third, detailed experiments based on the VPS-driven analysis lead to the following key results: (i) decentralization by itself does not provide superior protection of privacy in DisCSP/DCOP algorithms when compared with centralization; instead, privacy protection also requires the presence of uncertainty about agents’ knowledge of the constraint graph. (ii) one needs to carefully examine the metrics chosen to measure privacy loss; the qualitative properties of privacy loss and hence the conclusions that can be drawn about an algorithm can vary widely based on the metric chosen. This paper should thus serve as a call to arms for further privacy research, particularly within the DisCSP/DCOP arena.     

DCOP FOR SMART HOMES: A CASE STUDY

The aim of this article is to bring forth the issue of integrating the services provided by intelligent artifacts in Ambient Intelligence applications. Specifically, we propose a Distributed Constraint Optimization procedure for achieving a functional integration of intelligent artifacts in a smart home. To this end, we employ Adopt-N , a state-of-the-art algorithm for solving Distributed Constraint Optimization Problems (DCOP). This article attempts to state the smart home coordination problem in general terms, and provides the details of a DCOP-based approach by describing a case study taken from the RoboCare project. More specifically, we show how (1) DCOP is a convenient metaphor for casting smart home coordination problems, and (2) the specific features which distinguish Adopt-N from other algorithms for DCOP represent a strong asset in the smart home domain.     

ADOPT-ing: unifying asynchronous distributed optimization with asynchronous backtracking

This article presents an asynchronous algorithm for solving distributed constraint optimization problems (DCOPs). The proposed technique unifies asynchronous backtracking (ABT) and asynchronous distributed optimization (ADOPT) where valued nogoods enable more flexible reasoning and more opportunities for communication, leading to an important speed-up. While feedback can be sent in ADOPT by COST messages only to one predefined predecessor, our extension allows for sending such information to any relevant agent. The concept of valued nogood is an extension by Dago and Verfaille of the concept of classic nogood that associates the list of conflicting assignments with a cost and, optionally, with a set of references to culprit constraints. DCOPs have been shown to have very elegant distributed solutions, such as ADOPT, distributed asynchronous overlay (DisAO), or DPOP. These algorithms are typically tuned to minimize the longest causal chain of messages as a measure of how the algorithms will scale for systems with remote agents (with large latency in communication). ADOPT has the property of maintaining the initial distribution of the problem. To be efficient, ADOPT needs a preprocessing step consisting of computing a Depth-First Search (DFS) tree on the constraint graph. Valued nogoods allow for automatically detecting and exploiting the best DFS tree compatible with the current ordering. To exploit such DFS trees it is now sufficient to ensure that they exist. Also, the inference rules available for valued nogoods help to exploit schemes of communication where more feedback is sent to higher priority agents. Together they result in an order of magnitude improvement.     

Probabilistic optimal solution assessment for DCOPs

Distributed Constraint Optimization Problems (DCOPs) are widely used in Multi-Agent Systems for coordination and scheduling. The present paper proposes a heuristic algorithm that uses probabilistic assessment of the optimal solution in order to quickly find a solution that is not far from the optimal one. The heuristic assessment uses two passes by the agents to produce a high-quality solution. Extensive performance evaluation demonstrates that the solution of the proposed probabilistic assessment algorithm is indeed very close to the optimum, on smaller problems where this could be measured. In larger set-ups, the quality of the solution is demonstrated relatively to standard incomplete search algorithms.     

On game theoretic DSA-driven MAC for cognitive radio networks

The game theoretic dynamic spectrum allocation (DSA) technique is an efficient approach to coordinate cognitive radios sharing the spectrum. However, existing game based DSA algorithms lack a platform to support the game process. On the other hand, existing medium access control (MAC) protocols for cognitive radio networks do not fully utilize the adaptability and intelligence of the cognitive radio (CR) to achieve efficient spectrum utilization, let alone fairness and QoS support. Therefore it is necessary to develop DSA-driven MAC protocols with the game theoretic DSA embedded into the MAC layer. In this paper, based on the analysis of challenges for the game theoretic DSA in realistic applications, we conclude that a unified game theoretic DSA-driven MAC framework should constitute of four integral components: (1) DSA algorithm, deriving the spectrum access strategy for data communication; (2) negotiation mechanism, coordinating players to follow the right game policy; (3) clustering algorithm, limiting the negotiation within one cluster for scalability; (4) collision avoidance mechanism, eliminating collisions among clusters. With our MAC framework, DSA-driven MAC protocols can be conveniently developed, as illustrated in the design process of a concrete QoSe-DSA-driven MAC protocol. The game theoretic DSA-driven MAC framework can fulfill merits of game theoretic DSA algorithms including high spectrum utilization, collision-free channel access for data communication, QoS and fairness support. Through simulations, the merits of the DSA-driven MAC framework are demonstrated.     

基于分布式约束优化的武器目标分配问题研究

为解决舰艇编队协同防空中的武器目标分配(WTA)问题,提出一种将WTA问题建模为分布式约束优化问题的方法。介绍求解分布式约束优化问题的2个典型算法ADOPT和DPOP。通过Frodo软件平台对舰艇拦截多批反舰导弹过程进行仿真,比较2个算法在仿真时间、通信量等方面的性能,结果证明了该方法求解WTA问题的可行性。     

Exploring selfish reinforcement learning in repeated games with stochastic rewards

In this paper we introduce a new multi-agent reinforcement learning algorithm, called exploring selfish reinforcement learning (ESRL). ESRL allows agents to reach optimal solutions in repeated non-zero sum games with stochastic rewards, by using coordinated exploration. First, two ESRL algorithms for respectively common interest and conflicting interest games are presented. Both ESRL algorithms are based on the same idea, i.e. an agent explores by temporarily excluding some of the local actions from its private action space, to give the team of agents the opportunity to look for better solutions in a reduced joint action space. In a latter stage these two algorithms are transformed into one generic algorithm which does not assume that the type of the game is known in advance. ESRL is able to find the Pareto optimal solution in common interest games without communication. In conflicting interest games ESRL only needs limited communication to learn a fair periodical policy, resulting in a good overall policy. Important to know is that ESRL agents are independent in the sense that they only use their own action choices and rewards to base their decisions on, that ESRL agents are flexible in learning different solution concepts and they can handle both stochastic, possible delayed rewards and asynchronous action selection. A real-life experiment, i.e. adaptive load-balancing of parallel applications is added.     

Node and Link Allocation in Network Virtualization Based on Distributed Constraint Optimization

Virtual Networks (VNs) offer a flexible and economic approach to deploy customer suited networks. However, defining how resources of a physical network are used to support VNs requirements is a NP-hard problem. For this reason, heuristics have been used on mapping of virtual networks. Although heuristics do not ensure the optimal solution, they implement fast solutions and showed satisfactory results. This work presents a modeling of the node and link allocation problem using Distributed Constraint Optimization Problem (DCOP) with factor graphs, which is a formalism widely used in real distributed optimization problems. In our approach, we use the max-sum algorithm to solve the DCOP. Correctness criteria for this approach are discussed and verifications are conducted through model checking.     

On Randomization in On-Line Computation

This paper concerns two fundamental but somewhat neglected issues, both related to the design and analysis of randomized on-line algorithms. Motivated by early results in game theory we define several types of randomized on-line algorithms, discuss known conditions for their equivalence, and give a natural example distinguishing between two kinds of randomizations. In particular, we show thatmixedrandomized memoryless paging algorithms can achieve strictly better competitive performance thanbehavioralrandomized algorithms. Next we summarize known—and derive new—“Yao principle” theorems for lower bounding competitive ratios of randomized on-line algorithms. This leads to four different theorems for bounded/unbounded and minimization/maximization problems.     

Online solution of nonlinear two‐player zero‐sum games using synchronous policy iteration

The two‐player zero‐sum (ZS) game problem provides the solution to the bounded L₂‐gain problem and so is important for robust control. However, its solution depends on solving a design Hamilton–Jacobi–Isaacs (HJI) equation, which is generally intractable for nonlinear systems. In this paper, we present an online adaptive learning algorithm based on policy iteration to solve the continuous‐time two‐player ZS game with infinite horizon cost for nonlinear systems with known dynamics. That is, the algorithm learns online in real time an approximate local solution to the game HJI equation. This method finds, in real time, suitable approximations of the optimal value and the saddle point feedback control policy and disturbance policy, while also guaranteeing closed‐loop stability. The adaptive algorithm is implemented as an actor/critic/disturbance structure that involves simultaneous continuous‐time adaptation of critic, actor, and disturbance neural networks. We call this online gaming algorithm ‘synchronous’ ZS game policy iteration. A persistence of excitation condition is shown to guarantee convergence of the critic to the actual optimal value function. Novel tuning algorithms are given for critic, actor, and disturbance networks. The convergence to the optimal saddle point solution is proven, and stability of the system is also guaranteed. Simulation examples show the effectiveness of the new algorithm in solving the HJI equation online for a linear system and a complex nonlinear system. Copyright © 2011 John Wiley & Sons, Ltd.