进化强化学习及其在机器人路径跟踪中的应用

研究了一种基于自适应启发评价(AHC)强化学习的移动机器人路径跟踪控制方法.AHC的评价单元(ACE)采用多层前向神经网络来实现,将TD(λ)算法和梯度下降法相结合来更新神经网络的权值.AHC的动作选择单元(ASE)由遗传算法优化的模糊推理系统(FIS)构成.ACE网络的输出构成二次强化信号,用于指导ASE的学习.最后将所提出的算法应用于移动机器人的行为学习,较好地解决了机器人的复杂路径跟踪问题.

     

进化操作行为学习模型及在移动机器人避障上的应用

针对移动机器人避障上存在的自适应能力较差的问题,结合遗传算法(GA)的进化思想,以自适应启发评价(AHC)学习和操作条件反射(OC)理论为基础,提出了一种基于进化操作行为学习模型(EOBLM)的移动机器人学习避障行为的方法。该方法是一种改进的AHC学习模式,评价单元采用多层前向神经网络来实现,利用TD算法和梯度下降法进行权值更新,这一阶段学习用来生成取向性信息,作为内在动机决定进化的方向;动作选择单元主要用来优化操作行为以实现状态到动作的最佳映射。优化过程分两个阶段来完成,第一阶段通过操作条件反射学习算法得到的信息熵作为个体适应度,执行GA学习算法搜索最优个体;第二阶段由OC学习算法选择最优个体内的最优操作行为,并得到新的信息熵值。通过移动机器人避障仿真实验,结果表明所设计的EOBLM能使机器人通过不断与外界未知环境进行交互主动学会避障的能力,与传统的AHC方法相比其自学习自适应的能力得到加强。     

动态环境下移动机器人路径跟踪与避障

采用模糊神经网络并结合模糊逻辑控制的方法,研究在动态环境下移动机器人的在线路径跟踪与实时避障问题:针对移动机器人的运动学模型,依据点与直线间的距离关系设计了移动机器人的路径跟踪算法,提出了一种基于模糊神经网络的移动机器人在线路径跟踪方案,采用改进的BP算法对网络进行学习与训练,利用梯度下降法调整网络的权值与阈值,使其实际输出与期望输出的误差总均方差最小.同时,运用模糊逻辑控制,实现了移动机器人的实时避障.仿真实例证实了控制方案的有效性,表明了所提出的跟踪算法与控制方案具有良好的动态路径跟踪与实时避障能力.     

一种基于小脑模型关节控制器评论-策略家的机器人跟踪控制算法

本文提出了一种基于小脑模型关节控制器(CMAC)的评论–策略家算法,设计不依赖模型的跟踪控制器,来解决机器人的跟踪问题.该跟踪控制器包含位置控制器和角度控制器,其输出分别为线速度和角速度.位置控制器由评价单元和策略单元组成,每个单元都采用CMAC算法,按改进δ学习规则在线调整权值.策略单元产生控制量;评判单元在线调整策略单元学习速率.以双轮驱动自主移动机器人为例,与固定学习速率CMAC做比较,仿真数据表明,基于CMAC的评论–策略家算法的跟踪控制器具有跟踪速度快,自适应能力强,配置参数范围宽,不依赖数学模型等特点.     

神经网络在移动机器人路径规划中的应用研究

移动机器人路径规划可分为两种类型(1)全局路径规划;(2)局部路径规划.本文分析了Kohonen神经网络算法及其识别机理,提出了Kohonen神经网络和BP神经网络结合起来进行路径规划的方法,最后给出了具体的算法,探讨了在神经网络技术中引入学习机制的特点.     

基于模糊神经网络的强化学习及其在机器人导航中的应用

研究基于行为的移动机器人控制方法.将模糊神经网络与强化学习理论相结合,构成模糊强化系统.它既可获取模糊规则的结论部分和模糊隶属度函数参数,也可解决连续状态空间和动作空间的强化学习问题.将残差算法用于神经网络的学习,保证了函数逼近的快速性和收敛性.将该系统的学习结果作为反应式自主机器人的行为控制器,有效地解决了复杂环境中的机器人导航问题.     

神经网络在移动机器人路径规划中的应用研究

移动机器人路径规划可分为两种类型:(1)全局路径规划;(2)局部路径规划。本文分析了Kohonen神经网络算法及其识别机理,提出了Kohonen神经网络和BP神经网络结合起来进行路径规划的方法,最后给出了具体的算法,探讨了在神经网络技术中引入学习机制的特点。     

一种基于强化学习的ART2神经网络——RL-ART2

提出一种基于强化学习的ART2神经网络(RL-ART2),使其利用强化学习的特性通过与环境交互而无需训练样本即可进行在线学习,同时给出该神经网络的学习算法.当ART2神经网络运行时,通过内部竞争学习得到输出的分类模式,随后通过与环境交互得到神经网络分类模式的运行效果并对其进行评价.通过这种不断与环境的交互学习,当经过在线学习足够的时间和次数后,ART2神经网络即具有相当的识别率.移动机器人路径规划仿真实验表明,使用RL-ART2后与未使用前相比大大减少了机器人与障碍物的碰撞次数,实践证明该方法的合理性和有效性.     

基于Q学习算法和BP神经网络的倒立摆控制

Q学习是Watkins[1]提出的求解信息不完全马尔可夫决策问题的一种强化学习方 法.将Q学习算法和BP神经网络有效结合,实现了状态未离散化的倒立摆的无模型学习控 制.仿真表明:该方法不仅能成功解决确定和随机倒立摆模型的平衡控制,而且和Anderson[2] 的AHC(Adaptive Heuristic Critic)等方法相比,具有更好的学习效果.     

基于NN改进PSO算法的机器人路径规划

本文介绍了基于神经网络和微粒群优化算法的移动机器人动态避障路径规划算法.通过神经网络改进的微粒群算法,充分利用了神经网络的融合性和并行性来提高微粒群算法中适应度函数的准确性.通过神经网络描述机器人工作空间的动态环境约束并找到最优的适应度函数,在微粒群算法中使用该函数,求得微粒群算法最优无碰路径.     

煤矿井下安全监控分站的设计及其仿真实现

为了设计一种具有低成本、低功耗、易操作、功能强且可靠性高的煤矿井下安全分站,针对煤矿安全生产实际,文章提出了采用MCS-51系列单片机为核心、具有CAN总线通信接口的煤矿井下安全监控分站的设计方案;首先给出煤矿井下安全监控分站的整体构架设计,然后着重阐述模拟量输入信号处理系统的设计过程,最后说明单片机最小系统及其键盘、显示、报警、通信等各个组成部分的设计;为验证设计方案的可行性与有效性,使用Proteus软件对设计内容进行仿真验证,设计的煤矿井下安全监控分站具有瓦斯、温度等模拟量参数超标报警功能和电机开停、风门开闭等开关量指示功能;仿真结果表明:设计的煤矿井下安全监控分站具有一定的实际应用价值.     

View-based model-driven architecture for enhancing maintainability of data access services

In modern service-oriented architectures, database access is done by a special type of services, the so-called data access services (DAS). Though, particularly in data-intensive applications, using and developing DAS are very common today, the link between the DAS and their implementation, e.g. a layer of data access objects (DAOs) encapsulating the database queries, still is not sufficiently elaborated, yet. As a result, as the number of DAS grows, finding the desired DAS for reuse and/or associated documentation can become an impossible task. In this paper we focus on bridging this gap between the DAS and their implementation by presenting a view-based, model-driven data access architecture (VMDA) managing models of the DAS, DAOs and database queries in a queryable manner. Our models support tailored views of different stakeholders and are scalable with all types of DAS implementations. In this paper we show that our view-based and model driven architecture approach can enhance software development productivity and maintainability by improving DAS documentation. Moreover, our VMDA opens a wide range of applications such as evaluating DAS usage for DAS performance optimization. Furthermore, we provide tool support and illustrate the applicability of our VMDA in a large-scale case study. Finally, we quantitatively prove that our approach performs with acceptable response times.     

A pragmatic multi-method investigation of discrepant technological events: Coping,attributions, and ‘accidental’ learning

Discrepant technological events or situations that entail a problem, a misunderstanding or a difficulty with the Information Technology (IT) being employed, are common in the workplace, and can lead to frustration and avoidance behaviors. Little is known, however, about how individuals cope with these events. This paper examines these events by using a multi-method pragmatic approach informed by coping theory. The results of two studies – a critical incident study and an experiment – serve to build and test, respectively, a theoretical model that posits that individuals use a variety of strategies when dealing with these events: they experience negative emotions, make external attributions, and adopt engagement coping strategies directed at solving the event, eventually switching to a disengagement coping strategy when they feel they have no control over the situation. Furthermore, users’ efforts may result in ‘accidental’ learning as they try to overcome the discrepant IT events through engagement coping. The paper ends with a discussion of the results in light of existing literature, future opportunities for research, and implications for practice.     

Call for Papers The 6^th International Conference on Grid and Cooperative Computing

Recent years have seen rapid advances in various grid-related technologies, middleware, and applications. The GCC conference has become one of the largest scientific events worldwide in grid and cooperative computing. The 6th international conference on grid and cooperative computing (GCC2007) Sponsored by China Computer Federation (CCF),Institute of Computing Technology, Chinese Academy of Sciences (ICT) and Xinjiang University ,and in Cooperation with IEEE Computer Soceity ,is to be held from August 16 to 18, 2007 in Urumchi, Xinjiang, China.     

The Distributed Time-Triggered Simulation Scheme: Core Principles and Supporting Execution Engine

Distributed real-time simulation is a young technology field but its practice is under increasing demands. In recent years the author and his collaborators have been establishing a new approach called the distributed time-triggered simulation (DTS) scheme which is conceptually simple and easy to use but widely applicable. The concept was initiated in the course of developing a new-generation object-oriented real-time programming scheme called the time-triggered message-triggered object (TMO) programming scheme. Some fundamental issues inherent in distributed real-time simulation that were learned during recent experimental studies are discussed along with some approaches for resolving the issues. An execution engine developed to support both the TMOs engaged in control computation and the TMOs engaged in DTS is also discussed along with its possible extensions that will enable significantly larger-scale DTSs.     

Software-Konfigurationsmanagement in großen Softwareprojekten

Zusammenfassung Mit zunehmender Größe der Softwaresysteme verschärfen sich die für die Software-Herstellung typischen Probleme: Beherrschen großer Objektmengen, Erhalten der Systemkonsistenz, Kontrolle der ständigen Änderungseinflüsse und Gewährleisten einer langen Lebensdauer. Die Disziplin Konfigurationsmanagement bildet den methodischen Ansatz, diese Probleme besser zu beherrschen. Software-Konfigurationsmanagement faßt die Herstellung von Softwaresystemen als eine Abfolge von kontrollierten Änderungen an gesicherten Zwischen- und Endergebnissen auf. Dargestellt werden die Objekte und Funktionen des Software-Konfigurationsmanagements sowie die hierfür in großen Software-Projekten benötigten Methoden, Instanzen und Hilfsmittel.     

Information for Authors

正http://www.zju.edu.cn/jzus http://www.springerlink.com Aim The Journals of Zhejiang University-SCIENCE(A/B/C)are edited by the international board of distinguished Chinese and foreign scientists,and are aimed to present the latest developments and achievements in scientific research in China and     

Editorial:Brain-machine interface(BMI) and cyborg intelligence

正Brain-machine interfaces(BMIs)aim at building a direct communication pathway between the brain and an external device,and represent an area of research where significant progress has been made during the past decade.Based on BMIs,mind information can be read out by neural signals to control     

Information for Authors

正http://www.zju.edu.cn/jzus http://www.springerlink.com Aim The Journals of Zhejiang University-SCIENCE(A/B/C)are edited by the international board of distinguished Chinese and foreign scientists,and are aimed to present the latest developments and achievements in scientific research in China and overseas to the world’s scientific circles,especially to stimulate     

SCIENCE CHINA Information Sciences SUBJECT INDEX TO VOLUME 57(2014)

正~~