Codevelopmental learning between human and humanoid robot using a dynamic neural-network model.

This paper examines characteristics of interactive learning between human tutors and a robot having a dynamic neural-network model, which is inspired by human parietal cortex functions. A humanoid robot, with a recurrent neural network that has a hierarchical structure, learns to manipulate objects. Robots learn tasks in repeated self-trials with the assistance of human interaction, which provides physical guidance until the tasks are mastered and learning is consolidated within the neural networks. Experimental results and the analyses showed the following: 1) codevelopmental shaping of task behaviors stems from interactions between the robot and a tutor; 2) dynamic structures for articulating and sequencing of behavior primitives are self-organized in the hierarchically organized network; and 3) such structures can afford both generalization and context dependency in generating skilled behaviors.     

SOIM: a self-organizing invertible map with applications in activevision

We propose a novel neural network, called the self-organized invertible map (SOIM), that is capable of learning many-to-one functionals mappings in a self-organized and online fashion. The design and performance of the SOIM are highlighted by learning a many-to-one functional mapping that exists in active vision for spatial representation of three-dimensional point targets. The learned spatial representation is invariant to changing camera configurations. The SOIM also possesses an invertible property that can be exploited for active vision. An efficient and experimentally feasible method was devised for learning this representation on a real active vision system. The proof of convergence during learning as well as conditions for invariance of the learned spatial representation are derived and then experimentally verified using the active vision system. We also demonstrate various active vision applications that benefit from the properties of the mapping learned by SOIM.     

Learning Dextrous Manipulation Skills for Multifingered Robot Hands Using the Evolution Strategy

We present a method for autonomous learning of dextrous manipulation skills with multifingered robot hands. We use heuristics derived from observations made on human hands to reduce the degrees of freedom of the task and make learning tractable. Our approach consists of learning and storing a few basic manipulation primitives for a few prototypical objects and then using an associative memory to obtain the required parameters for new objects and/or manipulations. The parameter space of the robot is searched using a modified version of the evolution strategy, which is robust to the noise normally present in real-world complex robotic tasks. Given the difficulty of modeling and simulating accurately the interactions of multiple fingers and an object, and to ensure that the learned skills are applicable in the real world, our system does not rely on simulation; all the experimentation is performed by a physical robot, in this case the 16-degree-of-freedom Utah/MIT hand. E xperimental results show that accurate dextrous manipulation skills can be learned by the robot in a short period of time. We also show the application of the learned primitives to perform an assembly task and how the primitives generalize to objects that are different from those used during the learning phase.     

Learning Dextrous Manipulation Skills for Multifingered Robot Hands Using the Evolution Strategy

We present a method for autonomous learning of dextrous manipulation skills with multifingered robot hands. We use heuristics derived from observations made on human hands to reduce the degrees of freedom of the task and make learning tractable. Our approach consists of learning and storing a few basic manipulation primitives for a few prototypical objects and then using an associative memory to obtain the required parameters for new objects and/or manipulations. The parameter space of the robot is searched using a modified version of the evolution strategy, which is robust to the noise normally present in real-world complex robotic tasks. Given the difficulty of modeling and simulating accurately the interactions of multiple fingers and an object, and to ensure that the learned skills are applicable in the real world, our system does not rely on simulation; all the experimentation is performed by a physical robot, in this case the 16-degree-of-freedom Utah/MIT hand. Experimental results show that accurate dextrous manipulation skills can be learned by the robot in a short period of time. We also show the application of the learned primitives to perform an assembly task and how the primitives generalize to objects that are different from those used during the learning phase.     

基于广义细胞自动机的网络信息自组织利用方法

目前的网络信息利用模式存在着严重缺陷，它将网络上发生的海量、随机、分布、并行的信息利用行为当作是没有后效的和彼此无关的．该文提出一种新的基于网络信息自组织的信息利用模式以及基于广义细胞自动机的网络信息自组织方法．按照本文的信息利用模式，网络信息利用行为总是伴随着信息内容在网络中的扩散，网络信息利用行为成为有后效的和相关的，从而导致不同信息内容和不同内容粒度的分布式的信息自组织结构，形成基于这种信息自组织结构的网络信息利用模式．文中进而提出一种广义细胞自动机的模型、结构和算法，通过群体智能，发现网络中的信息自组织结构．分析和实验表明，基于广义细胞自动机的的网络信息自组织利用模式，在效率、自适应性和可靠性等方面优于目前的网络信息利用方法．     

基于视觉监控的操作手势轨迹分布模式研究

为准确学习飞行员操作手势的轨迹分布模式,提出一种改进的层次自组织映射方法。引入 秩和检验技术,结合编辑距离判断内部网的匹配程度,通过交叉验证使验证集获得误差最小,从而自适应取得判断异常的阈值。根据训练得到的轨迹分布模式检测操作过程中的局部异常,判断运动轨迹所表示的事件是否为异常事件,并预测手势将来行为轨迹。实验结果验证了改进方法的有效性。     

Navigation behavior based on self-organized spatial representation in hierarchical recurrent neural network

ABSTRACT

A cognitive map is an internal model of the external world and contains the spatial representation of the surrounding environment. The existence of the cognitive map was first identified in rats; rats can navigate to their desired destination using cognitive maps while dealing with environmental uncertainty. We performed a mobile robot navigation experiment where obstacles were randomly placed using hierarchical recurrent neural network (HRNN) with multiple timescales. The HRNN was trained to navigate the mobile robot to the destination indicated by a snapshot image. After the training, the HRNN was able to successfully avoid the obstacles and navigate to the destination from any location in the environment. Analysis of the internal states of the HRNN showed that the module with fast timescale handles obstacle avoidance and the one with slow timescale has spatial representation corresponding to the spatial position of the destination. Moreover, in the experiment wherein the novel path appeared, the trained HRNN performed shortcut behavior. The shortcut behavior shows that the HRNN performed navigation using the self-organized spatial representation in the slow recurrent neural network. This indicates that training of goal-oriented navigation, i.e. the navigation motivated by a snapshot image of the destination results in the self-organization of cognitive map-like representation.     

Self-organization of Dynamic Object Features Based on Bidirectional Training

This paper presents a method to self-organize object features that describe object dynamics using bidirectional training. The model is composed of a dynamics learning module and a feature extraction module. Recurrent Neural Network with Parametric Bias (RNNPB) is utilized for the dynamics learning module, learning and self-organizing the sequences of robot and object motions. A hierarchical neural network is linked to the input of RNNPB as the feature extraction module for self-organizing object features that describe the object motions. The two modules are simultaneously trained through bidirectional training using image and motion sequences acquired from the robot's active sensing with objects. Experiments are performed with the robot's pushing motion with a variety of objects to generate sliding, falling over, bouncing and rolling motions. The results have shown that the model is capable of self-organizing object dynamics based on the self-organized features.     

Learning to generate proactive and reactive behavior using a dynamic neural network model with time-varying variance prediction mechanism

This paper discusses a possible neurodynamic mechanism that enables self-organization of two basic behavioral modes, namely a ‘proactive mode’ and a ‘reactive mode,’ and of autonomous switching between these modes depending on the situation. In the proactive mode, actions are generated based on an internal prediction, whereas in the reactive mode actions are generated in response to sensory inputs in unpredictable situations. In order to investigate how these two behavioral modes can be self-organized and how autonomous switching between the two modes can be achieved, we conducted neurorobotics experiments by using our recently developed dynamic neural network model that has a capability to learn to predict time-varying variance of the observable variables. In a set of robot experiments under various conditions, the robot was required to imitate other’s movements consisting of alternating predictable and unpredictable patterns. The experimental results showed that the robot controlled by the neural network model was able to proactively imitate predictable patterns and reactively follow unpredictable patterns by autonomously switching its behavioral modes. Our analysis revealed that variance prediction mechanism can lead to self-organization of these abilities with sufficient robustness and generalization capabilities.     

Learning of a controller for non-recurring fast movements

In this paper a learning method is described which enables a conventional industrial robot to accurately execute the teach-in path in the presence of dynamical effects and high speed. After training the system is capable of generating positional commands that in combination with the standard robot controller lead the robot along the desired trajectory. The mean path deviations are reduced to a factor of 20 for our test configuration. For low speed motion the learned controllers' accuracy is in the range of the resolution of the positional encoders. The learned controller does not depend on specific trajectories. It acts as a general controller that can be used for non-recurring tasks as well as for sensor-based planned paths. For repetitive control tasks accuracy can be even increased. Such improvements are caused by a three level structure estimating a simple process model, optimal a posteriori commands, and a suitable feedforward controller, the latter including neural networks for the representation of nonlinear behaviour. The learning system is demonstrated in experiments with a Manutec R2 industrial robot. After training with only two sample trajectories the learned control system is applied to other totally different paths which are executed with high precision as well.     

Model-based learning for mobile robot navigation from the dynamicalsystems perspective

This paper discusses how a behavior-based robot can construct a "symbolic process" that accounts for its deliberative thinking processes using models of the environment. The paper focuses on two essential problems; one is the symbol grounding problem and the other is how the internal symbolic processes can be situated with respect to the behavioral contexts. We investigate these problems by applying a dynamical system's approach to the robot navigation learning problem. Our formulation, based on a forward modeling scheme using recurrent neural learning, shows that the robot is capable of learning grammatical structure hidden in the geometry of the workspace from the local sensory inputs through its navigational experiences. Furthermore, the robot is capable of generating diverse action plans to reach an arbitrary goal using the acquired forward model which incorporates chaotic dynamics. The essential claim is that the internal symbolic process, being embedded in the attractor, is grounded since it is self-organized solely through interaction with the physical world. It is also shown that structural stability arises in the interaction between the neural dynamics and the environmental dynamics, which accounts for the situatedness of the internal symbolic process, The experimental results using a mobile robot, equipped with a local sensor consisting of a laser range finder, verify our claims.     

基于自结构动态递归模糊神经网络的无人机姿态控制*

针对无人机非线性、强耦合等特点,提出了基于该自结构动态递归模糊神经网络的姿态控制系统,给出了基于Lyapunov函数的系统稳定性证明。对四层模糊神经网络进行了优化和改进,设计了自结构动态递归模糊神经网络,该网络可以根据系统状态在线更新权值、创建/删除节点、优化网络结构。仿真表明：该控制方法的突出优点是,在兼顾考虑了系统中的不确定性因素、非线性因素及外部干扰并存的情况下,保证系统的稳定性和跟踪性能;同时此网络结构比固定结构的模糊神经网络响应速度快,因此更具优越性。     

Neural network models of haptic shape perception

Three different models of tactile shape perception inspired by the human haptic system were tested using an 8 d.o.f. robot hand with 45 tactile sensors. One model is based on the tensor product of different proprioceptive and tactile signals and a self-organizing map (SOM). The two other models replace the tensor product operation with a novel self-organizing neural network, the Tensor-Multiple Peak Self-Organizing Map (T-MPSOM). The two T-MPSOM models differ in the procedure employed to calculate the neural activation. The computational models were trained and tested with a set of objects consisting of hard spheres, blocks and cylinders. All the models learned to map different shapes to different areas of the SOM, and the tensor product model as well as one of the T-MPSOM models also learned to discriminate individual test objects.     

Human robot cooperation with compliance adaptation along the motion trajectory

In this paper we propose a novel approach for intuitive and natural physical human–robot interaction in cooperative tasks. Through initial learning by demonstration, robot behavior naturally evolves into a cooperative task, where the human co-worker is allowed to modify both the spatial course of motion as well as the speed of execution at any stage. The main feature of the proposed adaptation scheme is that the robot adjusts its stiffness in path operational space, defined with a Frenet–Serret frame. Furthermore, the required dynamic capabilities of the robot are obtained by decoupling the robot dynamics in operational space, which is attached to the desired trajectory. Speed-scaled dynamic motion primitives are applied for the underlying task representation. The combination allows a human co-worker in a cooperative task to be less precise in parts of the task that require high precision, as the precision aspect is learned and provided by the robot. The user can also freely change the speed and/or the trajectory by simply applying force to the robot. The proposed scheme was experimentally validated on three illustrative tasks. The first task demonstrates novel two-stage learning by demonstration, where the spatial part of the trajectory is demonstrated independently from the velocity part. The second task shows how parts of the trajectory can be rapidly and significantly changed in one execution. The final experiment shows two Kuka LWR-4 robots in a bi-manual setting cooperating with a human while carrying an object.     

自然语言训练的机器人基本行为控制器

本文探索了一种直接利用自然语言进行机器人运动技能训练的控制方法， 提出了利用模糊神经网络结构作为基本行为控制单元，通过教师的自然语言指令完成针对某 一特定行为的运动经验获取和控制器训练，这是一种更加自然的控制器构造方式，以基本运 动单元为基础，可以进一步实现机器人复杂任务的语言编程与控制．所提控制方法最终在一 个轮式移动机器人系统上进行了语言训练实验．     

Mobile robot programming using natural language

How will naive users program domestic robots? This paper describes the design of a practical system that uses natural language to teach a vision-based robot how to navigate in a miniature town. To enable unconstrained speech the robot is provided with a set of primitive procedures derived from a corpus of route instructions. When the user refers to a route that is not known to the robot, the system will learn it by combining primitives as instructed by the user. This paper describes the components of the Instruction-Based Learning architecture and discusses issues of knowledge representation, the selection of primitives and the conversion of natural language into robot-understandable procedures.     

Towards gesture-based programming: shape from motion primordial learning of sensorimotor primitives

Gesture-based programming (GBP) is a paradigm for the evolutionary programming of dextrous robotic systems by human demonstration. We call the paradigm “gesture-based” because we try to capture, in real-time, the intention behind the demonstratrator's fleeting, context-dependent hand motions, contact conditions, finger poses, and even cryptic utterances, rather than just recording and replaying movement. The paradigm depends on a pre-existing knowledge base of capabilities, collectively called “encapsulated expertise”, that comprise the real-time sensorimotor primitives from which the run-time executable is constructed as well as providing the basis for interpreting the teacher's actions during programming. In this paper we first describe the GBP environment, which is not fully implemented. We then present a technique based on principal components analysis, augmentable with model-based information, for learing and recognizing sensorimotor primitives. This paper describes simple applications of the technique to a small mobile robot and a PUMA manipulator. The mobile robot learned to escape from jams while the manipulator learned guarded moves and rotational accommodation that are composable to allow flat plate mating operations. While these initial applications are simple, they demonstrate the ability to extract primitives from demonstration, recognize the learned primitives in subsequent demonstrations, and combine and transform primitives to create different capabilities, which are all critical to the GBP paradigm.     

Adaptive Robot Biped Locomotion with Dynamic Motion Primitives and Coupled Phase Oscillators

In order to properly function in real-world environments, the gait of a humanoid robot must be able to adapt to new situations as well as to deal with unexpected perturbations. A promising research direction is the modular generation of movements that results from the combination of a set of basic primitives. In this paper, we present a robot control framework that provides adaptive biped locomotion by combining the modulation of dynamic movement primitives (DMPs) with rhythm and phase coordination. The first objective is to explore the use of rhythmic movement primitives for generating biped locomotion from human demonstrations. The second objective is to evaluate how the proposed framework can be used to generalize and adapt the human demonstrations by adjusting a few open control parameters of the learned model. This paper contributes with a particular view into the problem of adaptive locomotion by addressing three aspects that, in the specific context of biped robots, have not received much attention. First, the demonstrations examples are extracted from human gaits in which the human stance foot will be constrained to remain in flat contact with the ground, forcing the “bent-knee” at all times in contrast with the typical straight-legged style. Second, this paper addresses the important concept of generalization from a single demonstration. Third, a clear departure is assumed from the classical control that forces the robot’s motion to follow a predefined fixed timing into a more event-based controller. The applicability of the proposed control architecture is demonstrated by numerical simulations, focusing on the adaptation of the robot’s gait pattern to irregularities on the ground surface, stepping over obstacles and, at the same time, on the tolerance to external disturbances.     

Chaotic exploration and learning of locomotion behaviors

We present a general and fully dynamic neural system, which exploits intrinsic chaotic dynamics, for the real-time goal-directed exploration and learning of the possible locomotion patterns of an articulated robot of an arbitrary morphology in an unknown environment. The controller is modeled as a network of neural oscillators that are initially coupled only through physical embodiment, and goal-directed exploration of coordinated motor patterns is achieved by chaotic search using adaptive bifurcation. The phase space of the indirectly coupled neural-body-environment system contains multiple transient or permanent self-organized dynamics, each of which is a candidate for a locomotion behavior. The adaptive bifurcation enables the system orbit to wander through various phase-coordinated states, using its intrinsic chaotic dynamics as a driving force, and stabilizes on to one of the states matching the given goal criteria. In order to improve the sustainability of useful transient patterns, sensory homeostasis has been introduced, which results in an increased diversity of motor outputs, thus achieving multiscale exploration. A rhythmic pattern discovered by this process is memorized and sustained by changing the wiring between initially disconnected oscillators using an adaptive synchronization method. Our results show that the novel neurorobotic system is able to create and learn multiple locomotion behaviors for a wide range of body configurations and physical environments and can readapt in realtime after sustaining damage.