The Application of Connectionist Structures to Learning Impedance Control in Robotic Contact Tasks

The goal of this paper is to consider the synthesis of learning impedance control using recurrent connectionist structures for on-line learning of robot dynamic uncertainties in the case of robot contact tasks. The connectionist structures are integrated in non-learning impedance control laws that are intended to improve the transient dynamic response immediately after the contact. The recurrent neural network as a part of hybrid learning control algorithms uses fast learning rules and available sensor information in order to improve the robotic performance progressively for a minimum possible number of learning epochs. Some simulation results of deburring process with the MANUTEC r3 robot are presented here in order to verify the effectiveness of the proposed control learning algorithms.     

Human-robot collaboration while sharing production activities in dynamic environment: SPADER system

Interactive robot doing collaborative work in hybrid work cell need adaptive trajectory planning strategy. Indeed, systems must be able to generate their own trajectories without colliding with dynamic obstacles like humans and assembly components moving inside the robot workspace. The aim of this paper is to improve collision-free motion planning in dynamic environment in order to insure human safety during collaborative tasks such as sharing production activities between human and robot. Our system proposes a trajectory generating method for an industrial manipulator in a shared workspace. A neural network using a supervised learning is applied to create the waypoints required for dynamic obstacles avoidance. These points are linked with a quintic polynomial function for smooth motion which is optimized using least-square to compute an optimal trajectory. Moreover, the evaluation of human motion forms has been taken into consideration in the proposed strategy. According to the results, the proposed approach is an effective solution for trajectories generation in a dynamic environment like a hybrid workspace.     

Artificial neural network based robot control: An overview

The current thrust of research in robotics is to build robots which can operate in dynamic and/or partially known environments. The ability of learning endows the robot with a form of autonomous intelligence to handle such situations. This paper focuses on the intersection of the fields of robot control and learning methods as represented by artificial neural networks. An in-depth overview of the application of neural networks to the problem of robot control is presented. Some typical neural network architectures are discussed first. The important issues involved in the study of robotics are then highlighted. This paper concentrates on the neural network applications to the motion control of robots involved in both non-contact and contact tasks. The current state of research in this area is surveyed and the strengths and weakness of the present approaches are emphasized. The paper concludes by indentifying areas which need future research work.     

Dynamic motion learning for multi-DOF flexible-joint robots using active–passive motor babbling through deep learning

This paper proposes a learning strategy for robots with flexible joints having multi-degrees of freedom in order to achieve dynamic motion tasks. In spite of there being several potential benefits of flexible-joint robots such as exploitation of intrinsic dynamics and passive adaptation to environmental changes with mechanical compliance, controlling such robots is challenging because of increased complexity of their dynamics. To achieve dynamic movements, we introduce a two-phase learning framework of the body dynamics of the robot using a recurrent neural network motivated by a deep learning strategy. The proposed methodology comprises a pre-training phase with motor babbling and a fine-tuning phase with additional learning of the target tasks. In the pre-training phase, we consider active and passive exploratory motions for efficient acquisition of body dynamics. In the fine-tuning phase, the learned body dynamics are adjusted for specific tasks. We demonstrate the effectiveness of the proposed methodology in achieving dynamic tasks involving constrained movement requiring interactions with the environment on a simulated robot model and an actual PR2 robot both of which have a compliantly actuated seven degree-of-freedom arm. The results illustrate a reduction in the required number of training iterations for task learning and generalization capabilities for untrained situations.     

A method to learn the inverse kinematics of multi-link robots by evolving neuro-controllers

A general method to learn the inverse kinematic of multi-link robots by means of neuro-controllers is presented. We can find analytical solutions for the most used and well-known robots in the literature. However, these solutions are specific to a particular robot configuration and are not generally applicable to other robot morphologies. The proposed method is general in the sense that it is independent of the robot morphology. The method is based on the evolutionary computation paradigm and works obtaining incrementally better neuro-controllers. Furthermore, the proposed method solves some specific issues in robotic neuro-controller learning: it avoids any neural network learning algorithm which relies on the classical supervised input-target learning scheme and hence it lets to obtain neuro-controllers without providing targets. It can converge beyond local optimal solutions, which is one of the main drawbacks of some neural network training algorithms based on gradient descent when applied to highly redundant robot morphologies. Furthermore, using learning algorithms such as the neuro-evolution of augmenting topologies it is also possible to learn the neural network topology which is a common source of empirical testing in neuro-controllers design. Finally, experimental results are provided when applying the method to two multi-link robot learning tasks and a comparison between structural and parametric evolutionary strategies on neuro-controllers is shown.     

A Study on Acquiring Underlying Behavioral Criteria for Manipulator Motion by Focusing on Learning Efficiency

Conventional humanoid robotic behaviors are directly programmed depending on the programmer's personal experience. With this method, the behaviors usually appear unnatural. It is believed that a humanoid robot can acquire new adaptive behaviors from a human, if the robot has the criteria underlying such behaviors. The aim of this paper is to establish a method of acquiring human behavioral criteria. The advantage of acquiring behavioral criteria is that the humanoid robots can then autonomously produce behaviors for similar tasks with the same behavioral criteria but without transforming data obtained from morphologically different humans every time for every task. In this paper, a manipulator robot learns a model behavior, and another robot is created to perform the model behavior instead of being performed by a person. The model robot is presented some behavioral criteria, but the learning manipulator robot does not know them and tries to infer them. In addition, because of the difference between human and robot bodies, the body sizes of the learning robot and the model robot are also made different. The method of obtaining behavioral criteria is realized by comparing the efficiencies with which the learning robot learns the model behaviors. Results from the simulation have demonstrated that the proposed method is effective for obtaining behavioral criteria. The proposed method, the details regarding the simulation, and the results are presented in this paper.     

An online collision-free trajectory generation algorithm for human–robot collaboration

The premise of human–robot collaboration is that robots have adaptive trajectory planning strategies in hybrid work cell. The aim of this paper is to propose a new online collision avoidance trajectory planning algorithm for moderate dynamic environments to insure human safety when sharing collaborative tasks. The algorithm contains two parts: trajectory generation and local optimization. Firstly, based on empirical Dirichlet Process Gaussian Mixture Model (DPGMM) distribution learning, a neural network trajectory planner called Collaborative Waypoint Planning network (CWP-net) is proposed to generate all key waypoints required for dynamic obstacle avoidance in joint space according to environmental inputs. These points are used to generate quintic spline smooth motion trajectories with velocity and acceleration constraints. Secondly, we present an improved Stochastic Trajectory Optimization for Motion Planning (STOMP) algorithm which locally optimizes the generated trajectories of CWP-net by constraining the trajectory optimization range and direction through the DPGMM model. Simulations and real experiments from an industrial use case of human–robot collaboration in the field of aircraft assembly testing show that the proposed algorithm can smoothly adjust the nominal path online and effectively avoid collisions during the collaboration.     

A behavior-based mobile robot architecture for Learning from Demonstration

Autonomous mobile robots (AMRs), to be truly flexible, should be equipped with learning capabilities, which allow them to adapt effectively to a dynamic and changing environment. This paper proposes a modular, behavior-based control architecture, which is particularly suited for “Learning from Demonstration” experiments in the spatial domain. The robot learns sensory-motor behaviors online by observing the actions of a person, another robot or another behavior. Offline learning phases are not necessary but might be used to trim the attained representation. First results applying RBF-approximation, growing neural cell structures and probabilistic models for progress estimation, are presented.     

Reinforcement learning for a biped robot to climb sloping surfaces

A neural network mechanism is proposed to modify the gait of a biped robot that walks on sloping surfaces using sensory inputs. The robot climbs a sloping surface from a level surface with no priori knowledge of the inclination of the surface. By training the neural network while the robot is walking, the robot adjusts its gait and finally forms a gait that is as stable as when it walks on the level surface. The neural network is trained by a reinforcement learning mechanism while proportional and integral (PI) control is used for position control of the robot joints. Experiments of static and pseudo dynamic learning are performed to show the validity of the proposed reinforcement learning mechanism. © 1997 John Wiley & Sons, Inc.     

Adaptive behavior navigation of a mobile robot

Describes a neural network model for the reactive behavioral navigation of a mobile robot. From the information received through the sensors the robot can elicit one of several behaviors (e.g., stop, avoid, stroll, wall following), through a competitive neural network. The robot is able to develop a control strategy depending on sensor information and learning operation. Reinforcement learning improves the navigation of the robot by adapting the eligibility of the behaviors and determining the linear and angular robot velocities     

Gradual learning for behavior acquisition by evolving artificial neural network

This paper describes a behavior acquisition of virtual robots by evolving artificial neural network (EANN) with a gradual learning. The gradual learning is a method in which initial states of simulation for evaluation is changing as optimization progresses. Motion of virtual robot is calculated by the physical engine PhysX, and it is controlled by an ANN. Parameters of an ANN are optimized by particle swarm optimization (PSO) so that a virtual robot follows the given target. Experimental results show that the gradual learning is better than a common learning method, realizing the standing behaviors which are not acquired by a common learning at all. It is also shown that random initialization of solutions in the middle of optimization leads to better behaviors.     

自组织映射神经网络量化机器人强化学习方法研究

强化学习一词来自于行为心理学，这门学科把行为学习看成反复试验的过程，从而把环境状态映射成相应的动作。在设计智能机器人过程中，如何来实现行为主义的思想，在与环境的交互中学习行为动作？文中把机器人在未知环境中为躲避障碍所采取的动作看作一种行为，采用强化学习方法来实现智能机器人避碰行为学习。为了提高机器人学习速度，在机器人局部路径规划中的状态空量化就显得十分重要。本文采用自组织映射网络的方法来进行空间的量化。由于自组织映射网络本身所具有的自组织特性，使得它在进行空间量化时就能够较好地解决适应性灵活性问题，本文在对状态空间进行自组织量化的基础方法上，采用强化学习。解决了机器人避碰行为的学习问题，取得了满意的学习结果。     

基于改进型自主发育网络的机器人场景识别方法

场景识别是移动机器人在陌生动态环境中完成任务的前提. 考虑到现有方法的不足, 本文提出了一种基于改进型自主发育网络的场景识别方法, 它通过引入基于多优胜神经元的Top-k竞争机制、基于负向学习的权值更新、基于连续性样本的加强型学习等步骤实现对场景的快速识别, 并使该方法具有更好的适应能力. 对于这种基于改进型自主发育网络的场景识别方法, 通过实验进行了对比测试. 结果表明, 这种改进型自主发育神经网络节点利用率高, 场景识别准确可靠, 可以较好地满足机器人作业的实际需求.     

Computational intelligence for structured learning of a partner robot based on imitation

Imitation is a powerful tool for gestural interaction between children and for teaching behaviors to children by parent. Furthermore, others’ action can be a hint for acquiring a new behavior that might not be the same as the original action. The importance is how to map or represent others’ action into new one in the internal state space. A good instructor can teach an action to a learner by understanding the mapping or imitating method of the learner. This indicates a robot also can acquire various behaviors using interactive learning based on imitation. This paper proposes structured learning for a partner robot based on the interactive teaching mechanism. The proposed method is composed of a spiking neural network, self-organizing map, steady-state genetic algorithm, and softmax action selection. Furthermore, we discuss the interactive learning of a human and a partner robot based on the proposed method through experiment results.     

Developmental word acquisition and grammar learning by humanoid robots through a self-organizing incremental neural network.

We present a new approach for online incremental word acquisition and grammar learning by humanoid robots. Using no data set provided in advance, the proposed system grounds language in a physical context, as mediated by its perceptual capacities. It is carried out using show-and-tell procedures, interacting with its human partner. Moreover, this procedure is open-ended for new words and multiword utterances. These facilities are supported by a self-organizing incremental neural network, which can execute online unsupervised classification and topology learning. Embodied with a mental imagery, the system also learns by both top-down and bottom-up processes, which are the syntactic structures that are contained in utterances. Thereby, it performs simple grammar learning. Under such a multimodal scheme, the robot is able to describe online a given physical context (both static and dynamic) through natural language expressions. It can also perform actions through verbal interactions with its human partner.     

An approach to learning mobile robot navigation

This paper describes an approach to learning an indoor robot navigation task through trial-and-error. A mobile robot, equipped with visual, ultrasonic and laser sensors, learns to servo to a designated target object. In less than ten minutes of operation time, the robot is able to navigate to a marked target object in an office environment. The central learning mechanism is the explanation-based neural network learning algorithm (EBNN). EBNN initially learns function purely inductively using neural network representations. With increasing experience, EBNN employs domain knowledge to explain and to analyze training data in order to generalize in a more knowledgeable way. Here EBNN is applied in the context of reinforcement learning, which allows the robot to learn control using dynamic programming.     

Q-学习及其在智能机器人局部路径规划中的应用研究

强化学习一词来自于行为心理学,这门学科把行为学习看成反复试验的过程,从而把环境状态映射成相应的动作．在设计智能机器人过程中,如何来实现行为主义的思想、在与环境的交互中学习行为动作？ 文中把机器人在未知环境中为躲避障碍所采取的动作看作一种行为,采用强化学习方法来实现智能机器人避碰行为学习．Ｑ－学习算法是类似于动态规划的一种强化学习方法,文中在介绍了Ｑ－学习的基本算法之后,提出了具有竞争思想和自组织机制的Ｑ－学习神经网络学习算法;然后研究了该算法在智能机器人局部路径规划中的应用,在文中的最后给出了详细的仿真结果     

Deep learning-based human action recognition to leverage context awareness in collaborative assembly

Human–Robot Collaboration is a critical component of Industry 4.0, contributing to a transition towards more flexible production systems that are quickly adjustable to changing production requirements. This paper aims to increase the natural collaboration level of a robotic engine assembly station by proposing a cognitive system powered by computer vision and deep learning to interpret implicit communication cues of the operator. The proposed system, which is based on a residual convolutional neural network with 34 layers and a long-short term memory recurrent neural network (ResNet-34 + LSTM), obtains assembly context through action recognition of the tasks performed by the operator. The assembly context was then integrated in a collaborative assembly plan capable of autonomously commanding the robot tasks. The proposed model showed a great performance, achieving an accuracy of 96.65% and a temporal mean intersection over union (mIoU) of 94.11% for the action recognition of the considered assembly. Moreover, a task-oriented evaluation showed that the proposed cognitive system was able to leverage the performed human action recognition to command the adequate robot actions with near-perfect accuracy. As such, the proposed system was considered as successful at increasing the natural collaboration level of the considered assembly station.     

Incremental acquisition of behaviors and signs based on a reinforcement learning schemata model and a spike timing-dependent plasticity network

A novel integrative learning architecture based on a reinforcement learning schemata model (RLSM) with a spike timing-dependent plasticity (STDP) network is described. This architecture models operant conditioning with discriminative stimuli in an autonomous agent engaged in multiple reinforcement learning tasks. The architecture consists of two constitutional learning architectures: RLSM and STDP. RLSM is an incremental modular reinforcement learning architecture, and it makes an autonomous agent acquire several behavioral concepts incrementally through continuous interactions with its environment and/or caregivers. STDP is a learning rule of neuronal plasticity found in cerebral cortices and the hippocampus of the human brain. STDP is a temporally asymmetric learning rule that contrasts with the Hebbian learning rule. We found that STDP enabled an autonomous robot to associate auditory input with its acquired behaviors and to select reinforcement learning modules more effectively. Auditory signals interpreted based on the acquired behaviors were revealed to correspond to 'signs' of required behaviors and incoming situations. This integrative learning architecture was evaluated in the context of on-line modular learning.