单agent强化学习与多agent强化学习比较研究

学习、交互及其结合是建立健壮、自治agent的关键必需能力。强化学习是agent学习的重要部分,agent强化学习包括单agent强化学习和多agent强化学习。文章对单agent强化学习与多agent强化学习进行了比较研究,从基本概念、环境框架、学习目标、学习算法等方面进行了对比分析,指出了它们的区别和联系,并讨论了它们所面临的一些开放性的问题。     

几种agent强化学习方法的比较研究

强化学习使agent具有在线自主学习能力,该文介绍了MDP模型下的自适应动态规划、时序差分学习、Q-学习等几种典型agent强化学习方法,并从基本思想、学习内容、收敛速度、可扩展性等方面对它们进行了对比分析。     

利用聚类分析法改进的多Agent协作强化学习方法

针对多agent系统强化学习中,状态空间和动作空间随着agent个数的增加成指数倍增长,进而导致维数灾难、学习速度慢和收敛性差的问题,提出了一种新型的混合强化学习方法,用于改进传统的多agent协作强化学习;该算法基于Friend-or-Foe Q-学习,事先采用聚类分析法对状态空间和动作空间进行预处理,降低空间维数后再进行强化学习,这就避免了同等状态环境下的重复劳动和对动作集的盲目搜索,理论上大大提高了agent的学习速度和算法的收敛性;文章首先进行改进算法的思想概述,然后给出了改进算法的学习框架和算法的一般描述。     

随机博弈框架下的多agent强化学习方法综述

多agent学习是在随机博弈的框架下,研究多个智能体间通过自学习掌握交互技巧的问题.单agent强化学习方法研究的成功,对策论本身牢固的数学基础以及在复杂任务环境中广阔的应用前景,使得多agent强化学习成为目前机器学习研究领域的一个重要课题.首先介绍了多agent系统随机博弈中基本概念的形式定义;然后介绍了随机博弈和重复博弈中学习算法的研究以及其他相关工作;最后结合近年来的发展,综述了多agent学习在电子商务、机器人以及军事等方面的应用研究,并介绍了仍存在的问题和未来的研究方向.     

强化学习研究综述

在未知环境中,关于agent的学习行为是一个既充满挑战又有趣的问题,强化学习通过试探与环境交互获得策略的改进,其学习和在线学习的特点使其成为机器学习研究的一个重要分支。介绍了强化学习在理论、算法和应用研究三个方面最新的研究成果,首先介绍了强化学习的环境模型和其基本要素;其次介绍了强化学习算法的收敛性和泛化有关的理论研究问题;然后结合最近几年的研究成果,综述了折扣型回报指标和平均回报指标强化学习算法;最后列举了强化学习在非线性控制、机器人控制、人工智能问题求解、多agent 系统问题等若干领域的成功应用和未来的发展方向。     

分层强化学习研究进展*

首先介绍了半马尔可夫决策过程、分层与抽象等分层强化学习的理论基础;其次,较全面地比较HAM、options、MAXQ和HEXQ四种典型的学习方法,从典型学习方法的拓展、学习分层、部分感知马尔可夫决策过程、并发和多agent合作等方面讨论分层强化学习的研究现状;最后指出分层强化学习未来的发展方向。     

一种基于案例推理的多agent 强化学习方法研究

提出一种基于案例推理的多agent 强化学习方法．构建了系统策略案例库,通过判断agent 之间的协作 关系选择相应案例库子集．利用模拟退火方法从中寻找最合适的可再用案例策略,agent 按照案例指导执行动作选 择．在没有可用案例的情况下,agent 执行联合行为学习（JAL）．在学习结果的基础上实时更新系统策略案例库．追 捕问题的仿真结果表明所提方法明显提高了学习速度与收敛性．     

一种共享经验元组的多agent协同强化学习算法

Q学习算法是一种最受欢迎的模型无关强化学习算法.本文通过对Q学习算法进行合适的扩充,提出了一种适合于多agent协作团队的共享经验元组的多agent协同强化学习算法,其中采用一种新的状态行为的知识表示方法使得状态行为空间得到缩减,采用相似性变换和经验元组的共享使得学习的效率得到提高.最后将该算法应用于猎人捕物问题域.实验结果表明该算法能够加快多个猎人合作抓捕猎物的进程,有利于协作任务的成功执行,并能提高多agent协作团队的协作效率,因此该算法是有效的.     

AODE中的多agent协商与学习

AODE是我们研制的一个面向agent的智能系统开发环境,本文以AODE为平台研究了多agent环境下的协商与学习本文利用协商-协商过程-协商线程的概念建立了多边-多问题协商模型MMN,该协商模型支持多agent环境中的多种协商形式及agent在协商过程中的学习,系统中的学习agent采用状态概率聚类空间上的多agent强化学习算法.该算法通过使用状态聚类方法减少Q值表存储所需空间,降低了经典Q-学习算法由于使用Q值表导致的对系统计算资源的要求,且该算法仍然可以保证收敛到最优解.     

强化学习中异构反馈信号的分析与集成

探讨了在高度危险行业的游戏式专业救援培训系统中,视觉与听觉信号能否协同作用以提高人们的记忆和推理能力问题;运用半马尔科夫博弈模型(semi-Markov game,SMG)提出了合作型多agent分层强化学习框架和算法,构建了由视觉处理agent、听觉处理agent以及人类agent组成的异构异质多agent系统;指出分析和归纳视觉听觉相干反馈信号的性质和特点是非常具有挑战性的任务,其决定了强化学习中异构信号的集成方法和途径。在此基础上,提出了将异构反馈信号进行集成的偏信息学习算法,大大缩小了状态搜索空间,缓解了强化学习固有的"维数灾难"问题;根据心理治疗的"系统脱敏"原理,设计了"情绪-个性-刺激-调节"(mood-personality-stimulus-regulation,MPSR)模型和恐怖场景个性化呈现算法(personalized rendering algorithm for terrorist scene,PRATS),用于提升救援队员的心理承受能力,并通过实验验证了算法的有效性。     

基于SNS的美术院校网络学习平台的研究

国家新课程改革中重点提出对教学模式的改革,21世纪是网络学习的时代,基于SNS的学习模式已经颠覆传统的学习模式,这种崭新的学习模式能够提高学生的团队协作与沟通能力,发展创新性与批判性思维,成为学生获取知识的有效途径。将SNS引入高校的教学与学习当中,具有重要的意义。     

Incremental online sparsification for model learning in real-time robot control

For many applications such as compliant, accurate robot tracking control, dynamics models learned from data can help to achieve both compliant control performance as well as high tracking quality. Online learning of these dynamics models allows the robot controller to adapt itself to changes in the dynamics (e.g., due to time-variant nonlinearities or unforeseen loads). However, online learning in real-time applications - as required in control - cannot be realized by straightforward usage of off-the-shelf machine learning methods such as Gaussian process regression or support vector regression. In this paper, we propose a framework for online, incremental sparsification with a fixed budget designed for fast real-time model learning. The proposed approach employs a sparsification method based on an independence measure. In combination with an incremental learning approach such as incremental Gaussian process regression, we obtain a model approximation method which is applicable in real-time online learning. It exhibits competitive learning accuracy when compared with standard regression techniques. Implementation on a real Barrett WAM robot demonstrates the applicability of the approach in real-time online model learning for real world systems.     

机器学习安全及隐私保护研究进展

机器学习作为实现人工智能的一种重要方法,在数据挖掘、计算机视觉、自然语言处理等领域得到广泛应用。随着机器学习应用的普及发展,其安全与隐私问题受到越来越多的关注。首先结合机器学习的一般过程,对敌手模型进行了描述。然后总结了机器学习常见的安全威胁,如投毒攻击、对抗攻击、询问攻击等,以及应对的防御方法,如正则化、对抗训练、防御精馏等。接着对机器学习常见的隐私威胁,如训练数据窃取、逆向攻击、成员推理攻击等进行了总结,并给出了相应的隐私保护技术,如同态加密、差分隐私。最后给出了亟待解决的问题和发展方向。     

Learning style Identifier: Improving the precision of learning style identification through computational intelligence algorithms

Identifying students’ learning styles has several benefits such as making students aware of their strengths and weaknesses when it comes to learning and the possibility to personalize their learning environment to their learning styles. While there exist learning style questionnaires for identifying a student's learning style, such questionnaires have several disadvantages and therefore, research has been conducted on automatically identifying learning styles from students’ behavior in a learning environment. Current approaches to automatically identify learning styles have an average precision between 66% and 77%, which shows the need for improvements in order to use such automatic approaches reliably in learning environments. In this paper, four computational intelligence algorithms (artificial neural network, genetic algorithm, ant colony system and particle swarm optimization) have been investigated with respect to their potential to improve the precision of automatic learning style identification. Each algorithm was evaluated with data from 75 students. The artificial neural network shows the most promising results with an average precision of 80.7%, followed by particle swarm optimization with an average precision of 79.1%. Improving the precision of automatic learning style identification allows more students to benefit from more accurate information about their learning styles as well as more accurate personalization towards accommodating their learning styles in a learning environment. Furthermore, teachers can have a better understanding of their students and be able to provide more appropriate interventions.     

A service component based CAT system with SCORM for advanced learning effects

In the web-based learning system, a variety of learning materials such as text, video, sound, and picture are employed with the objective of improving the learning effect. One of the main problems facing such systems is the systems’ ability to support the materials and integrate them into the system in such a way that they can be tailored to the user’s ability level. Accordingly, it is very difficult for the system to support user-oriented learning materials or courses efficiently. In this research, we applied the CAT (Computerized Adaptive Testing) system with SCORM interfaces to improve the learning effect. CAT is a method which focuses on analysis and evaluation of learning outcomes. SCORM is an international standard used to manage the learning materials. To support user-oriented learning and testing, we made a learning process using IRT (Item response theory). Throughout the process in this system, we implemented logic as a service component. The system process performs the learning test program’s logic by interface between the service components. The structure of this system interlinks SCORM API with the learning contents and items through a separate e-learning server.     

机器学习隐私保护研究综述

机器学习已成为大数据、物联网和云计算等领域核心技术.机器学习模型训练需要大量数据,这些数据通常通过众包方式收集,里面含有大量隐私数据包括个人身份信息（如电话号码、身份证号等）、敏感信息（如金融财务、医疗健康等信息）.如何低成本且高效地保护这些数据是一个重要的问题.介绍了机器学习及其隐私定义和隐私威胁,重点对机器学习隐私保护主流技术的工作原理和突出特点进行了阐述,并分别按照差分隐私、同态加密和安全多方计算等机制对机器学习隐私保护领域的研究成果进行了综述.在此基础上,对比分析了机器学习不同隐私保护机制的主要优缺点.最后,对机器学习隐私保护的发展趋势进行展望,并提出了该领域未来可能的研究方向.     

一种本体学习中分类关系提取方法的研究

本体学习技术是利用本体工程技术和机器学习技术等众多学科技术来实现本体的自动半自动构建,可解决本体手工构建的不足。根据本体学习目前的研究现状,提出了一种从文本中半自动获取本体中分类关系的实现,讨论了本体学习中概念抽取和概念间分类关系抽取等关键技术。实现了本体中分类关系提取,对于非分类关系的提取还有待研究。     

IMORL: Incremental Multiple-Object Recognition and Localization

This paper proposes an incremental multiple-object recognition and localization (IMORL) method. The objective of IMORL is to adaptively learn multiple interesting objects in an image. Unlike the conventional multiple-object learning algorithms, the proposed method can automatically and adaptively learn from continuous video streams over the entire learning life. This kind of incremental learning capability enables the proposed approach to accumulate experience and use such knowledge to benefit future learning and the decision making process. Furthermore, IMORL can effectively handle variations in the number of instances in each data chunk over the learning life. Another important aspect analyzed in this paper is the concept drifting issue. In multiple-object learning scenarios, it is a common phenomenon that new interesting objects may be introduced during the learning life. To handle this situation, IMORL uses an adaptive learning principle to autonomously adjust to such new information. The proposed approach is independent of the base learning models, such as decision tree, neural networks, support vector machines, and others, which provide the flexibility of using this method as a general learning methodology in multiple-object learning scenarios. In this paper, we use a neural network with a multilayer perceptron (MLP) structure as the base learning model and test the performance of this method in various video stream data sets. Simulation results show the effectiveness of this method.