基于Agent联盟的协作学习系统研究

为了在协作学习系统中实现学习者Agent之间的有效合作,通过引入一种新的合作机制--同学关系网模型(Schoolmate Relation Web Model),来构建学习系统中学习者Agent之间的同学联盟,并且基于学习者Agent之间的同学联盟来实现多个学习者Agent之间的协作学习.在每个同学联盟中,任意两个Agent之间都具有同学关系,并且联盟中的所有Agent相互协作,共同完成学习任务.另外,联盟中的学习者Agent之间的通信不是直接进行的,而是通过一个黑板来进行,这可以显著地提高Agent之间的通信效率.由于同学关系网模型可以避免Agent联盟形成的盲目性,并且可以提高学习者Agent之间的交互效率,从而使得我们基于Agent同学联盟的协作学习系统可以实现学习者Agent之间的有效合作,弥补了现有协作学习系统的不足.     

Agent联盟通信机制的动态构造及其Ambient演算验证分析

为了提高多Agent系统中的通信效率,Agent在通信过程中可以形成若干个联盟,在每个联盟内设置一块黑板,Agent之间的通信通过黑板进行.本文针对当前Agent运行的网络拓扑结构经常变化的情况,提出了Agent联盟通信机制的动态构造模型.当网络的拓扑结构发生变化时,该模型可以重新调整Agent联盟通信机制,各Agent根据调整后的通信机制进行有效通信,从而适应新的网络拓扑结构的要求.最后,本文采用Ambient演算对该模型进行了分析验证,结果证明模型是正确可行的.     

一种新的动态Agent联盟形成机制

动态Agent联盟的形成机制是当前MAS研究的一个重要方向.为了克服目前Agent联盟形成机制存在的Agent利用率不高等缺陷,本文提出了一种基于"二次招/投标"和NOAH规划的动态Agent联盟形成机制.该机制采用"二次招/投标"法形成动态联盟,采用NOAH规划进行任务规划和分解,能够在满足子任务时间约束的条件下,充分发挥Agent的并行执行能力;并使得Agent能够动态加入和退出联盟,从而提高了Agent的利用率,保证了自利Agent自身利益的最大化.最后通过一个典型的实例验证了其有效性.     

多Agent对抗环境下联盟形成的信任模型

提出了一种用于多Agent对抗环境下联盟形成的信任模型CORE,模型从能力和名誉两个方面来描述Agent的信任度,用能力向量空间中的距离公式来度量Agent胜任具体任务的能力大小,用隶属度函数来描述Agent的名誉。任务开始时,模型按信任度大小选取合适的Agent形成联盟,任务中Agent能力动态增长,名誉动态变化,任务结束后,模型根据联盟中Agent的表现情况计算新的信任度,作为下一个任务来临时联盟形成的依据。最后,在NetLogo平台上模拟实现了该模型。     

基于协作度的分布式自动协商联盟形成机制

现有联盟形成的研究中大都没有考虑到不同Agent的协作资源和协作态度不同的异质性,而是假定所有Agent具有相同的协作资源和协作态度.为此提出一种基于协作度的分布式自动协商联盟形成机制(collaborative degree-based distributed automatic negotiation coalition formation mechanism,CDBDN),通过对处在网络拓扑结构中Agent的协作资源和协作态度的描述建立Agent协作度的概念.以分布式的应用环境为背景,建立分布式协商协议(distributed negotiation protocal,DNP)来实现分布式自动协商方式构建联盟.该机制建立了分布式协商协议和引入了Agent协作度,提出基于Agent 协作度的协商策略.实验结果表明,该机制在联盟形成的效率、Agent协商效率和个体效用方面都表现出有较好的性能.     

基于关系网模型的Agent联盟形成策略*

从研究Agent社会合作机制入手,引入了一个表示Agent之间联系的社会关系网模型,并以该模型中的熟人集为基础提出了一种Agent联盟形成策略。该策略能有效地减少系统中的联盟数,避免联盟形成过程中的盲目性,节省协商时间并提高协商效率。     

基于FCM的多Agent移动学习教学框架模式研究

移动学习是远程教育新的发展阶段,对于满足人们的学习要求、平衡教育资源、构建学习型社会都有很大的帮助。但传统的远程教学模式将学生和教师分离为单独个体,忽略了"教"与"学"之间的交互性和"学"应该具有的主动性。为了更好地提高学生学习的兴趣和积极性,提出了一种新的移动学习教学模式,增加"学-教角色转化Agent群",通过测试Agent,采用FCM的模糊推理,分析学生Agent的认知水平,结合知识表示模型,实现自动筛选合格学生和自动角色转化机制。     

一种基于角色跟踪的群体Agent再励学习算法

在多Agent系统中,通过学习可以使Agent不断增加和强化已有的知识与能力,并选择合理的动作最大化自己的利益．但目前有关Agent学习大都限于单Agent模式,或仅考虑Agent个体之间的对抗,没有考虑Agent的群体对抗,没有考虑Agent在团队中的角色,完全依赖对效用的感知来判断对手的策略,导致算法的收敛速度不高．因此,将单Agent学习推广到在非通信群体对抗环境下的群体Agent学习．考虑不同学习问题的特殊性,在学习模型中加入了角色属性,提出一种基于角色跟踪的群体Agent再励学习算法,并进行了实验分析．在学习过程中动态跟踪对手角色,并根据对手角色与其行为的匹配度动态决定学习速率,利用minmax-Q算法修正每个状态的效用值,最终加快学习的收敛速度,从而改进了Bowling和Littman等人的工作．     

分布式环境下面向复杂任务的Agent联盟构建

在分布式环境下,现有Agent联盟构建算法不能解决带有相互依赖关系和转移成本的任务流程问题。为此,利用Agent协商构建联盟,在协商过程中设定方案发布Agent和参与Agent,并对应设计以成本信息调整和盈利任务争取为主的决策算法。在Agent的反馈信息中加入争取信息,允许参与Agent在多轮协商中采用可控制的信息泄露机制,通过泄露自己的成本信息向方案发布Agent争取可获利的任务,经过多轮协商,形成最优联盟结构。实验结果表明,在按劳分配联盟总收益的模式下,相比传统的信息不泄露机制,该信息泄露机制能够更快地形成联盟,并且具有更高的联盟净收益和Agent平均收益率。     

基于角色和CSCL的智能网络协作模型

为深入研究智能Agent在开放动态的网络环境中的应用，把角色机制应用到网络学习环境中，提出了一种新型的基于CSCL的智能网络协作模型，给出了智能Agent的结构表示及功能，并从多角色的角度给出了模型中Agent的分类。最后以共同学习活动为例，对Agent之间基于角色的协作过程进行了形式化描述。     

An Agent Based Framework for Internetware Computing

Internetware intends to be a paradigm of Web-based software development. At present, researches on Internetware have gained daily expanding attentions and interests. This paper proposes an agent based framework for Internetware computing. Four principles are presented that are followed by this framework. They are the autonomy principle, the abstract principle, the explicitness principle and the competence principle. Three types of agents with di?erent responsibilities are designed and specified. They are the capability providing agents, the capability planning agents and the capability consuming agents. In this sense, capability decomposition and satisfaction turns to be a key issue for this framework and becomes a communication protocol among these distributed and heterogenous agents. A capability conceptualization is proposed and based on the conceptualization, an agent coalition formation mechanism has been developed. This mechanism features that (1) all the participants make their one decisions on whether or not joining the coalition based on the capability realization pattern generated by the capability planning agents as well as the benefits they can obtain; and (2) the coalition selection is conducted by a negotiation process for satisfying the expectations of all the participants as the complexity of this problem has been proven to be NP-complete.     

Multi-agent cooperation pursuit based on an extension of AALAADIN organisational model

An approach of cooperative pursuit for multiple mobile targets based on multi-agents system is discussed. In this kind of problem the pursuit process is divided into two kinds of tasks. The first one (coalition problem) is designed to solve the problem of the pursuit team formation. To achieve this mission, we used an innovative method based on a dynamic organisation and reorganisation of the pursuers’ groups. We introduce our coalition strategy extended from the organisational agent, group, role model by assigning an access mechanism to the groups inspired by fuzzy logic principles. The second task (motion problem) is the treatment of the pursuers’ motion strategy. To manage this problem we applied the principles of the Markov decision process. Simulation results show the feasibility and validity of the given proposal.     

A reinforcement learning optimized negotiation method based on mediator agent

This paper firstly proposes a bilateral optimized negotiation model based on reinforcement learning. This model negotiates on the issue price and the quantity, introducing a mediator agent as the mediation mechanism, and uses the improved reinforcement learning negotiation strategy to produce the optimal proposal. In order to further improve the performance of negotiation, this paper then proposes a negotiation method based on the adaptive learning of mediator agent. The simulation results show that the proposed negotiation methods make the efficiency and the performance of the negotiation get improved.     

Agent long-term coalition credit

Agent trust researches become more and more important because they will ensure good interactions among the software agents in large-scale open systems. Moreover, individual agents often interact with long-term coalitions such as some E-commerce web sites. So the agents should choose a coalition based on utility and trust. Unfortunately, few studies have been done on agent coalition credit and there is a need to do it in detail. To this end, a long-term coalition credit model (LCCM) is presented. Furthermore, the relationship between coalition credit and coalition payoff is also attended. LCCM consists of internal trust based on agent direct interactions and external reputation based on agent direct observation. Generalization of LCCM can be demonstrated through experiments applied in both cooperative and competitive domain environment. Experimental results show that LCCM is capable of coalition credit computation efficiently and can properly reflect various factors effect on coalition credit. Another important advantage that is a useful and basic property of credit is that LCCM can effectively filter inaccurate or lying information among interactions.     

Sequentially optimal repeated coalition formation under uncertainty

Coalition formation is a central problem in multiagent systems research, but most models assume common knowledge of agent types. In practice, however, agents are often unsure of the types or capabilities of their potential partners, but gain information about these capabilities through repeated interaction. In this paper, we propose a novel Bayesian, model-based reinforcement learning framework for this problem, assuming that coalitions are formed (and tasks undertaken) repeatedly. Our model allows agents to refine their beliefs about the types of others as they interact within a coalition. The model also allows agents to make explicit tradeoffs between exploration (forming “new” coalitions to learn more about the types of new potential partners) and exploitation (relying on partners about which more is known), using value of information to define optimal exploration policies. Our framework effectively integrates decision making during repeated coalition formation under type uncertainty with Bayesian reinforcement learning techniques. Specifically, we present several learning algorithms to approximate the optimal Bayesian solution to the repeated coalition formation and type-learning problem, providing tractable means to ensure good sequential performance. We evaluate our algorithms in a variety of settings, showing that one method in particular exhibits consistently good performance in practice. We also demonstrate the ability of our model to facilitate knowledge transfer across different dynamic tasks.     

多任务联盟形成中的Agent行为策略研究

Agent联盟是多Agent系统中一种重要的合作方式,联盟形成是其研究的关键问题.本文提出一种串行多任务联盟形成中的Agent行为策略,首先论证了Agent合作求解多任务的过程是一个Markov决策过程,然后基于Q-学习求解单个Agent的最优行为策略.实例表明该策略在面向多任务的领域中可以快速、有效地串行形成多个任务求解联盟.     

Agent Reasoning Mechanism for Long–Term Coalitions Based on Decision Making and Trust

We address long–term coalitions that are formed of both customer and vendor agents. We present a coalition formation mechanism designed at the agent level as a decision problem. The proposed mechanism is analyzed at both system and agent levels. Our results show that the coalition formation mechanism is beneficial for both the system—it reaches an equilibrium state—and for the agents—their gains highly increase over time.     

基于直觉模糊关系的多主体联盟机制

提出一种新的联盟形成机制,以模糊数学为理论基础,着重研究个体间"满意度"和任务问"相似性"对联盟形成的影响,并将其表示为直觉模糊关系,通过模糊关系合成运算,得到新任务条件下agent间的"满意度".在此基础上,按不同阶段在不同范围内进行协商,从而有效降低了联盟形成的复杂性,并对其基本性质进行了分析.仿真实验表明了该机制的有效性.     

贝叶斯学习与强化学习结合技术的研究

强化学习的研究需要解决的重要难点之一是:探索未知的动作和采用已知的最优动作之间的平衡。贝叶斯学习是一种基于已知的概率分布和观察到的数据进行推理,做出最优决策的概率手段。因此,把强化学习和贝叶斯学习相结合,使 Agent 可以根据已有的经验和新学到的知识来选择采用何种策略:探索未知的动作还是采用已知的最优动作。本文分别介绍了单 Agent 贝叶斯强化学习方法和多 Agent 贝叶斯强化学习方法:单 Agent 贝叶斯强化学习包括贝叶斯 Q 学习、贝叶斯模型学习以及贝叶斯动态规划等;多 Agent 贝叶斯强化学习包括贝叶斯模仿模型、贝叶斯协同方法以及在不确定下联合形成的贝叶斯学习等。最后,提出了贝叶斯在强化学习中进一步需要解决的问题。