Cooperation of categorical and behavioral learning in a practical solution to the abstraction problem

Real robots should be able to adapt autonomously to various environments in order to go on executing their tasks without breaking down. They achieve this by learning how to abstract only useful information from a huge amount of information in the environment while executing their tasks. This paper proposes a new architecture which performs categorical learning and behavioral learning in parallel with task execution. We call the architectureSituation Transition Network System (STNS). In categorical learning, it makes a flexible state representation and modifies it according to the results of behaviors. Behavioral learning is reinforcement learning on the state representation. Simulation results have shown that this architecture is able to learn efficiently and adapt to unexpected changes of the environment autonomously. Atsushi Ueno, Ph.D.: He is a research associate in the Artificial Intelligence Laboratory at the Graduate School of Information Science at the Nara Institute of Science and Technology (NAIST). He received the B.E., the M.E., and the Ph.D. degrees in aeronautics and astronautics from the University of Tokyo in 1991, 1993, and 1997 respectively. His research interest is robot learning and autonomous systems. He is a member of Japan Association for Artificial Intelligence (JSAI). Hideaki Takeda, Ph.D.: He is an associate professor in the Artificial Intelligence Laboratory at the Graduate School of Information Science at the Nara Institute of Science and Technology (NAIST). He received his Ph.D. in precision machinery engineering from the University of Tokyo in 1991. He has conducted research on a theory of intelligent computer-aided design systems, in particular experimental study and logical formalization of engineering design. He is also interested in multiagent architectures and ontologies for knowledge base systems.     

Reinforcement Learning Based Algorithms for Average Cost Markov Decision Processes

This article proposes several two-timescale simulation-based actor-critic algorithms for solution of infinite horizon Markov Decision Processes with finite state-space under the average cost criterion. Two of the algorithms are for the compact (non-discrete) action setting while the rest are for finite-action spaces. On the slower timescale, all the algorithms perform a gradient search over corresponding policy spaces using two different Simultaneous Perturbation Stochastic Approximation (SPSA) gradient estimates. On the faster timescale, the differential cost function corresponding to a given stationary policy is updated and an additional averaging is performed for enhanced performance. A proof of convergence to a locally optimal policy is presented. Next, we discuss a memory efficient implementation that uses a feature-based representation of the state-space and performs TD(0) learning along the faster timescale. The TD(0) algorithm does not follow an on-line sampling of states but is observed to do well on our setting. Numerical experiments on a problem of rate based flow control are presented using the proposed algorithms. We consider here the model of a single bottleneck node in the continuous time queueing framework. We show performance comparisons of our algorithms with the two-timescale actor-critic algorithms of Konda and Borkar (1999) and Bhatnagar and Kumar (2004). Our algorithms exhibit more than an order of magnitude better performance over those of Konda and Borkar (1999).

分级移动IPv6中一种新的自适应MAP选择算法*

分级移动IPv6中存在单点故障和负荷集中问题。提出一种基于分布式MAP结构的自适应MAP选择算法,综合考虑移动节点的当前速度、会话到达率、MAP负荷及距离等因素,以MN注册后将产生的移动性管理信令开销最小为依据进行选择。仿真结果表明,自适应MAP选择算法能够根据移动节点和网络的当前特性优化地选择不同的MAP进行注册,使移动性管理信令开销最小,具有较好的负荷分担效果,并一定程度上增强了分级结构的鲁棒性。与最远/近MAP选择方案相比,自适应MAP选择算法能够提高网络的吞吐量及减少平均切换时延。     

Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach

In this paper, a finite-horizon neuro-optimal tracking control strategy for a class of discrete-time nonlinear systems is proposed. Through system transformation, the optimal tracking problem is converted into designing a finite-horizon optimal regulator for the tracking error dynamics. Then, with convergence analysis in terms of cost function and control law, the iterative adaptive dynamic programming (ADP) algorithm via heuristic dynamic programming (HDP) technique is introduced to obtain the finite-horizon optimal tracking controller which makes the cost function close to its optimal value within an ?-error bound. Three neural networks are used as parametric structures to implement the algorithm, which aims at approximating the cost function, the control law, and the error dynamics, respectively. Two simulation examples are included to complement the theoretical discussions.     

钦州市某水闸除险加固设计

钦州市某水闸工程是一座以灌溉为主,结合城市供水,兼顾发电、航运、旅游等综合效益的大(2)型水闸。工程运行50多年,其安全类别评定为四类。除险加固设计在充分考虑水文、地质、施工、运行、工程造价及既有重要建筑物的影响等因素,结合考虑枢纽各个建筑物的特点,初步设计阶段推荐了左岸船闸右岸厂房的枢纽布置方案,并提出了各主要建筑物的除险加固方案。     

基于Meta平衡的多Agent Q学习算法研究

多Agent强化学习算法的研究一直以来大多都是针对于合作策略,而NashQ算法的提出对非合作策略的研究无疑是一个重要贡献。针对在多Agent系统中,Nash平衡无法确保求得的解是Paret。最优解及其计算复杂度较高的问题,提出了基于Mcta平衡的MctaQ算法。与NashQ算法不同,MctaQ算法通过对自身行为的预处理以及对其它Agent行为的预测来获取共同行为的最优策略。最后通过研究及气候合作策略游戏实验,证明了MctaQ算法在解决非合作策略的问题中有着很好的理论解释和实验性能。     

基于强化学习的二级倒立摆控制

在模型未知和没有先验经验的条件下，采用一种改进的强化学习算法实现二级倒立摆系统的平衡控制。该学习算法不需要预测和辨识模型，能通过网络自身的联想和记忆，在线寻求最优策略。该学习算法采用基于神经网络的值函数逼近，并用直接梯度和适合度轨迹修正权值，有效实现对连续状态和行为空间任务的控制。计算机仿真证明了该强化学习算法在较短的时间内即可成功地学会控制直线二级倒立摆系统。     

斜轧锚杆纵裂缺陷分析及预防

在ANSYS/LS-DYNA中建模对斜轧锚杆的应力进行了模拟分析,由此提出了斜轧锚杆纵裂的力学模型。并分析了斜轧锚杆纵裂产生的原因及其影响因素,提出了在生产中预防纵裂应采取的措施。     

Damage Tolerant Magnesium Metal Matrix Composites: Influence of Reinforcement and Processing

A growing impetus to enhance our understanding of the behavior of magnesium-based alloys for use in weight critical applications resulted as a consequence of the low density of magnesium. In an attempt to enhance the applicability of magnesium for a wide spectrum of performance-critical applications, the addition of reinforcement to the alloy was considered as an economically affordable and potentially viable scientific alternative. In this paper are reported the results of a study aimed at understanding the influence of saffil alumina short fiber reinforcement on microstructural development of a squeeze-cast magnesium alloy. Preliminary results confirm promise of the reinforced alloy, which retains hardness, strength, and stiffness better at elevated temperatures compared to the unreinforced counterpart. However, impact strength and toughness of the reinforced alloy are inferior. The importance of the matrix alloy in governing the overall mechanical response of the composite microstructure is discussed based on fractographic observations. The importance of volume fraction of the reinforcing phase on properties of the composite microstructure is highlighted.     

碳纤维粘贴混凝土试件锚固长度的试验研究

将碳纤维粘贴在混凝土试件上，对其锚固长度进行试验研究，并对试验结果进行分析，从中找出了对应各种混凝土强度的锚固长度值，寻出其规律，对碳纤维锚固长度的取值和影响锚固长度的因素进行分析，提出了碳纤维锚固长度建议值，供工程应用参考。