基于虚拟静态锚节点的加权质心定位算法术

加权质心定位算法的实施需要布置较多的锚节点,较大的硬件成本限制了该算法在实际中的应用。本文结合余弦定理,利用原有的少量锚节点定位信息构造出虚拟静态锚节点VSAN（Virtual Static Anchor Node）参与定位,降低了该定位算法所需的硬件成本;此外,对加权质心算法进行数学推导和分析,从导数的角度阐述了权重系数和RSSI测距对加权质心算法的影响。仿真结果表明,基于VSAN的加权质心算法所需布置锚节点较少、定位精度较高;并通过实际环境下测试,验证了算法的有效性和可行性。     

XCSG在多机器人强化学习中的应用

XCS分类器在解决机器人强化学习方面已显示出较强的能力,但在多机器人领域仅局限于MDP环境,只能解决环境空间较小的学习问题。提出了XCSG来解决多机器人的强化学习问题。XCSG建立低维的逼近函数,梯度下降技术利用在线知识建立稳定的逼近函数,使Q-表格一直保持在稳定低维状态。逼近函数Q不仅所需的存储空间更小,而且允许机器人在线对已获得的知识进行归纳一般化。仿真实验表明,XCSG算法很好地解决了多机器人学习空间大、学习速度慢、学习效果不确定等问题。     

具有行为强化学习能力的人工生命进化模型设计

人工生命进化模型设计的关键问题是学习与进化之间的关系,在自主体生存期内的学习过程可以通过不同的遗传方式指导个体行为的进化。该文利用进化算法和人工神经网络的研究方法,设计了两种不同的人工生命自主体的进化模型,模型解决了先天的遗传进化和后天的个体神经系统强化学习的有机结合问题,并且得出结论认为,强化学习有助于自主体适应复杂的外部环境,同时学习也可以直接或间接地使该适应性成为自主体遗传基因上的固定成分。     

Adaptive target recognition

Target recognition is a multilevel process requiring a sequence of algorithms at low, intermediate and high levels. Generally, such systems are open loop with no feedback between levels and assuring their performance at the given probability of correct identification (PCI) and probability of false alarm (Pf) is a key challenge in computer vision and pattern recognition research. In this paper, a robust closed-loop system for recognition of SAR images based on reinforcement learning is presented. The parameters in model-based SAR target recognition are learned. The method meets performance specifications by using PCI and Pf as feedback for the learning system. It has been experimentally validated by learning the parameters of the recognition system for SAR imagery, successfully recognizing articulated targets, targets of different configuration and targets at different depression angles.     

Robot arm reaching through neural inversions and reinforcement learning

We present a neural method that computes the inverse kinematics of any kind of robot manipulators, both redundant and non-redundant. Inverse kinematics solutions are obtained through the inversion of a neural network that has been previously trained to approximate the manipulator forward kinematics. The inversion provides difference vectors in the joint space from difference vectors in the workspace. Our differential inverse kinematics (DIV) approach can be viewed as a neural network implementation of the Jacobian transpose method for arm kinematic control that does not require previous knowledge of the arm forward kinematics. Redundancy can be exploited to obtain a special inverse kinematic solution that meets a particular constraint (e.g. joint limit avoidance) by inverting an additional neural network The usefulness of our DIV approach is further illustrated with sensor-based multilink manipulators that learn collision-free reaching motions in unknown environments. For this task, the neural controller has two modules: a reinforcement-based action generator (AG) and a DIV module that computes goal vectors in the joint space. The actions given by the AG are interpreted with regard to those goal vectors.     

VRML虚拟场景跳转实现

基于VRML(Virtual Reality Modeling Language)构造复杂三维场景时,通常需要多个场景的组合和链接。如何实现三维场景空间的流畅跳转则十分关键。本文结合实例介绍了利用Anchor组节点实现虚拟场景间跳转的方法,从而有效服务于网络中复杂三维虚拟场景空间的跳转。     

Analyzing and visualizing multiagent rewards in dynamic and stochastic domains

The ability to analyze the effectiveness of agent reward structures is critical to the successful design of multiagent learning algorithms. Though final system performance is the best indicator of the suitability of a given reward structure, it is often preferable to analyze the reward properties that lead to good system behavior (i.e., properties promoting coordination among the agents and providing agents with strong signal to noise ratios). This step is particularly helpful in continuous, dynamic, stochastic domains ill-suited to simple table backup schemes commonly used in TD(λ)/Q-learning where the effectiveness of the reward structure is difficult to distinguish from the effectiveness of the chosen learning algorithm. In this paper, we present a new reward evaluation method that provides a visualization of the tradeoff between the level of coordination among the agents and the difficulty of the learning problem each agent faces. This method is independent of the learning algorithm and is only a function of the problem domain and the agents’ reward structure. We use this reward property visualization method to determine an effective reward without performing extensive simulations. We then test this method in both a static and a dynamic multi-rover learning domain where the agents have continuous state spaces and take noisy actions (e.g., the agents’ movement decisions are not always carried out properly). Our results show that in the more difficult dynamic domain, the reward efficiency visualization method provides a two order of magnitude speedup in selecting good rewards, compared to running a full simulation. In addition, this method facilitates the design and analysis of new rewards tailored to the observational limitations of the domain, providing rewards that combine the best properties of traditional rewards.     

Computer-aided design integration of a reinforced vibration isolator for electronic equipment’s system basedon experimental investigation

Electronic equipment’s system is always manufactured as a superprecision system. However, it will be used in harsh environment. For example, the computer in moving vehicles will be acted by vibrations. The objective of this paper is to provide a systematic investigation to test and computer-aided design of the vibration isolator for protection of electronic equipment’s system in harsh vibration environment. A micro-oil damping vibration isolator is designed and manufactured through coupling the oil and spring by ingenious tactics. The structure of the oil damping vibration isolator can achieve circulating oil damping function with an inner tube and an outer tube (some orifices are manufactured on upside and underside of the inner tube). The dynamics of the key model machine is systematically investigated. Based on the test, a nonlinear dynamic model for the vibration isolator is presented by analyzing the internal fluid dynamic phenomenon with respect to the vibration isolator. The model considers all the physical parameters of the structure. Comparisons with experimental data confirm the validity of the model. In the other, the model is integrated by introducing normalization measure. The normalization model shows the actual physical characteristics of the oil damping vibration isolator by considering quadratic damping, viscous damping, Coulomb damping, and nonlinear spring forces. An approximate solution is deduced by introducing harmonic transform method and Fourier transform method. Therefore, a parameter-matching optimal model for computer-aided design of the vibration isolator is build based on approximate solution. An example confirms the validity of the computer-aided design integration.

Cooperation of categorical and behavioral learning in a practical solution to the abstraction problem

Real robots should be able to adapt autonomously to various environments in order to go on executing their tasks without breaking down. They achieve this by learning how to abstract only useful information from a huge amount of information in the environment while executing their tasks. This paper proposes a new architecture which performs categorical learning and behavioral learning in parallel with task execution. We call the architectureSituation Transition Network System (STNS). In categorical learning, it makes a flexible state representation and modifies it according to the results of behaviors. Behavioral learning is reinforcement learning on the state representation. Simulation results have shown that this architecture is able to learn efficiently and adapt to unexpected changes of the environment autonomously. Atsushi Ueno, Ph.D.: He is a research associate in the Artificial Intelligence Laboratory at the Graduate School of Information Science at the Nara Institute of Science and Technology (NAIST). He received the B.E., the M.E., and the Ph.D. degrees in aeronautics and astronautics from the University of Tokyo in 1991, 1993, and 1997 respectively. His research interest is robot learning and autonomous systems. He is a member of Japan Association for Artificial Intelligence (JSAI). Hideaki Takeda, Ph.D.: He is an associate professor in the Artificial Intelligence Laboratory at the Graduate School of Information Science at the Nara Institute of Science and Technology (NAIST). He received his Ph.D. in precision machinery engineering from the University of Tokyo in 1991. He has conducted research on a theory of intelligent computer-aided design systems, in particular experimental study and logical formalization of engineering design. He is also interested in multiagent architectures and ontologies for knowledge base systems.     

Reinforcement Learning Based Algorithms for Average Cost Markov Decision Processes

This article proposes several two-timescale simulation-based actor-critic algorithms for solution of infinite horizon Markov Decision Processes with finite state-space under the average cost criterion. Two of the algorithms are for the compact (non-discrete) action setting while the rest are for finite-action spaces. On the slower timescale, all the algorithms perform a gradient search over corresponding policy spaces using two different Simultaneous Perturbation Stochastic Approximation (SPSA) gradient estimates. On the faster timescale, the differential cost function corresponding to a given stationary policy is updated and an additional averaging is performed for enhanced performance. A proof of convergence to a locally optimal policy is presented. Next, we discuss a memory efficient implementation that uses a feature-based representation of the state-space and performs TD(0) learning along the faster timescale. The TD(0) algorithm does not follow an on-line sampling of states but is observed to do well on our setting. Numerical experiments on a problem of rate based flow control are presented using the proposed algorithms. We consider here the model of a single bottleneck node in the continuous time queueing framework. We show performance comparisons of our algorithms with the two-timescale actor-critic algorithms of Konda and Borkar (1999) and Bhatnagar and Kumar (2004). Our algorithms exhibit more than an order of magnitude better performance over those of Konda and Borkar (1999).