Autonomous Adaptive Agents for Single Seller Sealed Bid Auctions

In developing open, heterogeneous and distributed multi-agent systems researchers often face a problem of facilitating negotiation and bargaining amongst agents. It is increasingly common to use auction mechanisms for negotiation in multi-agent systems. The choice of auction mechanism and the bidding strategy of an agent are of central importance to the success of the agent model. Our aim is to determine the best agent learning algorithm for bidding in a variety of single seller auction structures in both static environments where a known optimal strategy exists and in complex environments where the optimal strategy may be constantly changing. In this paper we present a model of single seller auctions and describe three adaptive agent algorithms to learn strategies through repeated competition. We experiment in a range of auction environments of increasing complexity to determine how well each agent performs, in relation to an optimal strategy in cases where one can be deduced, or in relation to each other in other cases. We find that, with a uniform value distribution, a purely reactive agent based on Cliff’s ZIP algorithm for continuous double auctions (CDA) performs well, although is outperformed in some cases by a memory based agent based on the Gjerstad Dickhaut agent for CDA.     

Efficient Management of Multi-Linked Negotiation Based on a Formalized Model

A Multi-linked negotiation problem occurs when an agent needs to negotiate with multiple other agents about different subjects (tasks, conflicts, or resource requirements), and the negotiation over one subject has influence on negotiations over other subjects. The solution of the multi-linked negotiations problem will become increasingly important for the next generation of advanced multi-agent systems. However, most current negotiation research looks only at a single negotiation and thus does not present techniques to manage and reason about multi-linked negotiations. In this paper, we first present a technique based on the use of a partial-order schedule and a measure of the schedule, called flexibility, which enables an agent to reason explicitly about the interactions among multiple negotiations. Next, we introduce a formalized model of the multi-linked negotiation problem. Based on this model, a heuristic search algorithm is developed for finding a near-optimal ordering of negotiation issues and their parameters. Using this algorithm, an agent can evaluate and compare different negotiation approaches and choose the best one. We show how an agent uses this technology to effectively manage interacting negotiation issues. Experimental work is presented which shows the efficiency of this approach.     

STRATUM: A METHODOLOGY FOR DESIGNING HEURISTIC AGENT NEGOTIATION STRATEGIES

Automated negotiation is a powerful (and sometimes essential) means for allocating resources among self-interested autonomous software agents. A key problem in building negotiating agents is the design of the negotiation strategy, which is used by an agent to decide its negotiation behavior. In complex domains, there is no single, obvious optimal strategy. This has led to much work on designing heuristic strategies, where agent designers usually rely on intuition and experience. In this article, we introduce STRATUM, a methodology for designing strategies for negotiating agents. The methodology provides a disciplined approach to analyzing the negotiation environment and designing strategies in light of agent capabilities and acts as a bridge between theoretical studies of automated negotiation and the software engineering of negotiation applications. We illustrate the application of the methodology by characterizing some strategies for the Trading Agent Competition and for argumentation-based negotiation.     

Predicate control: synchronization in distributed computations with look-ahead

The predicate control problem involves synchronizing a distributed computation to maintain a given global predicate. In contrast with many popular distributed synchronization problems such as mutual exclusion, readers writers, and dining philosophers, predicate control assumes a look-ahead, so that the computation is an off-line rather than an on-line input. Predicate control is targeted towards applications such as rollback recovery, debugging, and optimistic computing, in which such computation look-ahead is natural.We define predicate control formally and show that, in its full generality, the problem is NP-complete. We find efficient solutions for some important classes of predicates including “disjunctive predicates”, “mutual exclusion predicates”, and “readers writers predicates”. For each class of predicates, we determine the necessary and sufficient conditions for solving predicate control and describe an efficient algorithm for determining a synchronization strategy. In the case of “independent mutual exclusion predicates”, we determine that predicate control is NP-complete and describe an efficient algorithm that finds a solution under certain constraints.     

基于多Agent协商的虚拟企业伙伴选择方法

伙伴选择是虚拟企业建立过程中的核心问题,分析了虚拟企业的特点、虚拟企业环境下协商问题的特点,提出了一个适合于虚拟企业环境的多Agent协商模型。该模型支持多Agent多议题的多轮谈判,并将Agent类型引入到协商中来,作为指导协商Agent提议的一个重要因素。在不完全信息的条件下,应用贝叶斯学习的方法,更新既有信息,并通过分析对方Agent的历史提议序列,推测其类型,来指导自身的提议策略和战术,使自己的提议更具有针对性,避免了盲目性,从而节约协商时间,提高了协商的效率,使得盟主企业能在尽短的时间里寻找到理想的合作伙伴。     

一种多agent系统框架与协商机制研究*

从基于动态、异构网络上快速构建稳健的多agent系统出发,设计了多agent远程过程调用通信模型,定义了三种基本类型的agent,对KQML消息规范进行扩展,增加了对消息生存周期的控制,设计了双缓存消息推送器以实现agent消息的主动推送,并在WCF的基础上实现了该通信框架。针对同目标多agent协作系统提出了基于开销均衡的agent系统交互协商策略,通过实例证明相对于独立运行和基于正交互协商策略的agent系统,本协商策略可有效降低系统总开销,并可使运行负载更为均衡。     

Active-Vision for the Autonomous Surveillance of Dynamic,Multi-Object Environments

This paper presents a novel method for active-vision-based sensing-system reconfiguration for the autonomous surveillance of an object-of-interest as it travels through a multi-object dynamic workspace with an a priori unknown trajectory. Several approaches have been previously proposed to address the problem of sensor selection and control. However, these have primarily relied on off-line planning methods and rarely utilized on-line planning to compensate for unexpected variations in a target’s trajectory. The method proposed in this paper, on the other hand, uses a multi-agent system for on-line sensing-system reconfiguration, eliminating the need for any a priori knowledge of the target’s trajectory. Thus, it is robust to unexpected variations in the environment. Simulations and experiments have shown that the use of dynamic sensors with the proposed on-line reconfiguration algorithm can tangibly improve the performance of an active-surveillance system.      

A two-layered multi-agent reinforcement learning model and algorithm

Multi-agent reinforcement learning technologies are mainly investigated from two perspectives of the concurrence and the game theory. The former chiefly applies to cooperative multi-agent systems, while the latter usually applies to coordinated multi-agent systems. However, there exist such problems as the credit assignment and the multiple Nash equilibriums for agents with them. In this paper, we propose a new multi-agent reinforcement learning model and algorithm LMRL from a layer perspective. LMRL model is composed of an off-line training layer that employs a single agent reinforcement learning technology to acquire stationary strategy knowledge and an online interaction layer that employs a multi-agent reinforcement learning technology and the strategy knowledge that can be revised dynamically to interact with the environment. An agent with LMRL can improve its generalization capability, adaptability and coordination ability. Experiments show that the performance of LMRL can be better than those of a single agent reinforcement learning and Nash-Q.     

Hybrid active learning for reducing the annotation effort of operators in classification systems

Active learning is understood as any form of learning in which the learning algorithm has some control over the input samples due to a specific sample selection process based on which it builds up the model. In this paper, we propose a novel active learning strategy for data-driven classifiers, which is based on unsupervised criterion during off-line training phase, followed by a supervised certainty-based criterion during incremental on-line training. In this sense, we call the new strategy hybrid active learning. Sample selection in the first phase is conducted from scratch (i.e. no initial labels/learners are needed) based on purely unsupervised criteria obtained from clusters: samples lying near cluster centers and near the borders of clusters are expected to represent the most informative ones regarding the distribution characteristics of the classes. In the second phase, the task is to update already trained classifiers during on-line mode with the most important samples in order to dynamically guide the classifier to more predictive power. Both strategies are essential for reducing the annotation and supervision effort of operators in off-line and on-line classification systems, as operators only have to label an exquisite subset of the off-line training data resp. give feedback only on specific occasions during on-line phase. The new active learning strategy is evaluated based on real-world data sets from UCI repository and collected at on-line quality control systems. The results show that an active learning based selection of training samples (1) does not weaken the classification accuracies compared to when using all samples in the training process and (2) can out-perform classifiers which are built on randomly selected data samples.     

求解包含复杂关联约束的JSSP 的二级嵌套混合算法

作业车间调度问题(JSSP)包含“设备分配”和“工序排序” 2 个相互耦合的子问题,目 前的研究主要集中于工序串行的小规模问题。如果工序之间还存在并行、甚至嵌套等复杂关联 约束,则可行域性状非常复杂,当规模较大时,甚至难以求得可行解。针对以上难点问题,在 分别发挥遗传算法求解“分配问题”和蚁群算法求解“排序问题”的优势基础上,提出了二级嵌套 模型及其基本思路。通过一系列改进策略,如：基于工序的整数编码策略、基于设备类型的多 节点交叉策略、设备类别区间内基因互换的变异策略、基于逆向遍历的可行路径形成策略、基 于最短加工时间的信息素播洒与更新策略等等,构造了集成遗传算法与蚁群算法于同一循环体 的二级嵌套混合算法。针对中等规模问题,分别采用遗传算法、蚁群算法、二级嵌套蚁群算法、 遗传算法与蚁群算法相结合的二级嵌套混合算法,进行了对比试验研究。结果验证了所提算法 的可靠性和优越性,为求解包含复杂关联约束的JSSP 提供了新思路和新方法。     

基于时间约束的不完全信息的战略协商模型

在实证的一对一协商中，协商Agent不仅要面临自己的最后期限的压力，同时又要预测协商对手的最后期限和其类型，协商Agent的协商战略必须满足理性与均衡的要求。提出了通过形式化的方法建立轮流出价协商模型，给出了轮流出价协商战略均衡的条件定义，求出了基于时间限制的不完全信息环境下满足均衡组合的协商战略，建立了依据均衡战略的实用化协商算法，最后分析了该算法产生的实验数据，并在相同环境下与Zeus协商模型比较显示，依从本模型的均衡战略的协商Agent能根据对对手的不确定信息的信念动态地采取行动，以获得最大的期望收益。     

A posteriori versus a priori access strategies in slotted all-optical WDM rings

The paper presents the concept of “a posteriori” access strategy, as opposed to the previously studied concept of “a priori” strategy, to provide a fair and efficient access technique in slotted wavelength division multiplexing (WDM) rings under a large variety of traffic patterns. Using a posteriori strategies, the source node selects the packet for transmission based on the state of the WDM channels in the arriving slot, thus adapting the access decision to the instantaneous traffic in the various channels. The packet selection process of a posteriori strategies, when contrasted with a priori strategies, yields fair and efficient utilization of the ring bandwidth without requiring the equalization of the ring latency, nor imposing restrictions on the traffic patterns allowed in the system. Since the same features cannot be provided by a priori access strategies, a posteriori strategies offers a more versatile access control that justifies the higher implementation complexity due to the multi-channel sensing and on-line packet selection.     

基于博弈论及Q学习的多Agent协作追捕算法

多Agent协作追捕问题是多Agent协调与协作研究中的一个典型问题。针对具有学习能力的单逃跑者追捕问题，提出了一种基于博弈论及Q学习的多Agent协作追捕算法。首先,建立协作追捕团队，并构建协作追捕的博弈模型；其次,通过对逃跑者策略选择的学习，建立逃跑者有限的Step-T累积奖赏的运动轨迹，并把运动轨迹调整到追捕者的策略集中；最后,求解协作追捕博弈得到Nash均衡解，每个Agent执行均衡策略完成追捕任务。同时,针对在求解中可能存在多个均衡解的问题，加入了虚拟行动行为选择算法来选择最优的均衡策略。C#仿真实验表明，所提算法能够有效地解决障碍环境中单个具有学习能力的逃跑者的追捕问题，实验数据对比分析表明该算法在同等条件下的追捕效率要优于纯博弈或纯学习的追捕算法。     

基于交互历史的多Agent自动协商研究

在多Agent协商过程中,初始信念起到了至关重要的作用.而初始信念的形成是由设计者给予的部分专家知识和策略集,成功的交互历史是Agent在复杂环境中最后达成一致的提议集.通过学习机制从交互历史中获得知识,形成协商的初始信念,将更加有效地预测对方的策略,缩短协商过程的时间,再通过在线学习来协调己方Agent的行为.在此基础上优化协商模型,提高协商的效率和成功率.     

多Agent信念修正研究

Belief Revision is a theory that studies how to integrate new information into original belief set.Classical BR theory uses AGM frame,but it only resolves problems in single agent BR system.Multi-agent BR faces problems such as the collision of many information sources and how to maximize the logic consistence of multi-agent system.On the basis of game theory model,we form profit matrix under different BR strategies in muhi-agent system and try to get the best strategy that satisfies logic consistence of the system through negotiation.     

Learning Situation-Specific Coordination in Cooperative Multi-agent Systems

Achieving effective cooperation in a multi-agent system is a difficult problem for a number of reasons such as limited and possibly out-dated views of activities of other agents and uncertainty about the outcomes of interacting non-local tasks. In this paper, we present a learning system called COLLAGE, that endows the agents with the capability to learn how to choose the most appropriate coordination strategy from a set of available coordination strategies. COLLAGE relies on meta-level information about agents' problem solving situations to guide them towards a suitable choice for a coordination strategy. We present empirical results that strongly indicate the effectiveness of the learning algorithm.     

基于多Agent的复合模型求解自适应QoS机制

在基于网络的分布式系统应用基础上,分析了大型复杂问题复合模型协作求解的过程特征描述,提出基于多Agent 的领域问题协作求解的主动控制策略,探讨了用户交互Agent、系统主控Agent、协作Agent以及模型Agent和数据Agent等复合模型协作求解的4种Agent类型。应用多Agent层次结构,提出一种复合模型协作求解的自适应QoS体系结构,通过实现复合模型协作求解的主动调度规划算法对其进行了验证,支持分布式网络环境下实现模型资源和数据资源的共享,以提高协同计算环境分布式问题协作求解的运行效率和服务质量。     

一种基于信任的多Agent供应链协商关系网

Agent技术已被广泛用于供应链伙伴的协商。协商前如何选择协商Agent对提高协商效率有着重要的意义。提出了一种基于信任的多Agent协商关系网及其形成和更新算法,并对该协商关系网的特点进行了深入的研究。模拟表明,提出的协商关系网能有效地促进Agent之间的协商,提高协商成功率。     

基于影响度及状态预测的多agent协作算法

针对灾难环境中多agent协作问题进行了研究，提出了一种基于影响度与状态预测的多agent协作算法。首先该算法根据协作任务对信息的需求，使用影响度函数对agent感知到的信息进行处理；其次利用预测算法对任务的后续状态和agent的行为进行预测并根据预测结果制定协作策略；最后执行协作任务的agent根据动作效果和触发条件动态调整协作策略。为了验证算法的有效性，在Unity3D中搭建仿真平台，对比不同协作算法的收敛率、救援人数和整体得分，结果表明该算法的收敛速度快、救援人数多和整体得分最优，可以效地指导agent间的协作，能给实际救援协作策略的制定提供理论支持。     

Off-line synthesis of evolutionarily stable normative systems

Within the area of multi-agent systems, normative systems are a widely used framework for the coordination of interdependent activities. A crucial problem associated with normative systems is that of synthesising norms that will effectively accomplish a coordination task and that the agents will comply with. Many works in the literature focus on the on-line synthesis of a single, evolutionarily stable norm (convention) whose compliance forms a rational choice for the agents and that effectively coordinates them in one particular coordination situation that needs to be identified and modelled as a game in advance. In this work, we introduce a framework for the automatic off-line synthesis of evolutionarily stable normative systems that coordinate the agents in multiple interdependent coordination situations that cannot be easily identified in advance nor resolved separately. Our framework roots in evolutionary game theory. It considers multi-agent systems in which the potential conflict situations can be automatically enumerated by employing MAS simulations along with basic domain information. Our framework simulates an evolutionary process whereby successful norms prosper and spread within the agent population, while unsuccessful norms are discarded. The outputs of such a natural selection process are sets of codependent norms that, together, effectively coordinate the agents in multiple interdependent situations and are evolutionarily stable. We empirically show the effectiveness of our approach through empirical evaluation in a simulated traffic domain.