Artificial neural networks for robotics coordinate transformation

Artificial neural networks with such characteristics as learning, graceful degradation, and speed inherent to parallel distributed architectures might provide a flexible and cost solution to the real time control of robotics systems. In this investigation artificial neural networks are presented for the coordinate transformation mapping of a two-axis robot modeled with Fischertechnik physical modeling components. The results indicate that artificial neural systems could be utilized for practical situations and that extended research in these neural structures could provide adaptive architectures for dynamic robotics control.     

Neural network learning from demonstration and epipolar geometry for visual control of a nonholonomic mobile robot

The control of a robot system using camera information is a challenging task regarding unpredictable conditions, such as feature point mismatch and changing scene illumination. This paper presents a solution for the visual control of a nonholonomic mobile robot in demanding real world circumstances based on machine learning techniques. A novel intelligent approach for mobile robots using neural networks (NNs), learning from demonstration (LfD) framework, and epipolar geometry between two views is proposed and evaluated in a series of experiments. A direct mapping from the image space to the actuator command is conducted using two phases. In an offline phase, NN–LfD approach is employed in order to relate the feature position in the image plane with the angular velocity for lateral motion correction. An online phase refers to a switching vision based scheme between the epipole based linear velocity controller and NN–LfD based angular velocity controller, which selection depends on the feature distance from the pre-defined interest area in the image. In total, 18 architectures and 6 learning algorithms are tested in order to find optimal solution for robot control. The best training outcomes for each learning algorithms are then employed in real time so as to discover optimal NN configuration for robot orientation correction. Experiments conducted on a nonholonomic mobile robot in a structured indoor environment confirm an excellent performance with respect to the system robustness and positioning accuracy in the desired location.     

基于神经网络的机器人学习与控制：回顾与展望

机器人因其高效的感知、决策和执行能力,在人工智能、信息技术和智能制造等领域中具有巨大的应用价值。目前,机器人学习与控制已成为机器人研究领域的重要前沿技术之一。各种基于神经网络的智能算法被设计,从而为机器人系统提供同步学习与控制的规划框架。首先从神经动力学(ND)算法、前馈神经网络(FNNs)、递归神经网络(RNNs)和强化学习(RL)四个方面介绍了基于神经网络的机器人学习与控制的研究现状,回顾了近30年来面向机器人学习与控制的智能算法和相关应用技术。最后展望了该领域存在的问题和发展趋势,以期促进机器人学习与控制理论的推广及应用场景的拓展。     

机器人动态神经网络导航算法的研究和实现

针对Pioneer3-DX 移动机器人, 提出了基于强化学习的自主导航策略, 完成了基于动态神经网络的移动机器人导航算法设计. 动态神经网络可以根据机器人环境状态的复杂程度自动地调整其结构, 实时地实现机器人的状态与其导航动作之间的映射关系, 有效地解决了强化学习中状态变量表的维数爆炸问题. 通过对Pioneer3-DX移动机器人导航进行仿真和实物实验, 证明该方法的有效性, 且导航效果明显优于人工势场法.     

极坐标下基于迭代学习的移动机器人轨迹跟踪控制

为提高自主移动机器人对一类特殊轨迹的重复跟踪能力,在极坐标下建立了3轮全向移动机器人的运动学模型,结合离散时域下对轨迹跟踪问题的描述方法,采用开闭环P型迭代学习控制算法,并在给定条件下证明了其收敛性,随着迭代次数的增加,该算法能够有效改善动态不确定环境中系统的稳定性与收敛的快速性。通过将仿真结果作用于实际动态系统的初始控制输入,从而在实际环境下能以较少的迭代过程来获取控制律。实验结果表明,在仿真环境下机器人可以较好地跟踪玫瑰曲线,在实际机器人测试中,机器人能够较好地跟踪期望轨迹,从而证实了该方法对提高自主移动机器人轨迹跟踪能力的可行性与有效性。     

Reinforcement learning for a biped robot to climb sloping surfaces

A neural network mechanism is proposed to modify the gait of a biped robot that walks on sloping surfaces using sensory inputs. The robot climbs a sloping surface from a level surface with no priori knowledge of the inclination of the surface. By training the neural network while the robot is walking, the robot adjusts its gait and finally forms a gait that is as stable as when it walks on the level surface. The neural network is trained by a reinforcement learning mechanism while proportional and integral (PI) control is used for position control of the robot joints. Experiments of static and pseudo dynamic learning are performed to show the validity of the proposed reinforcement learning mechanism. © 1997 John Wiley & Sons, Inc.     

An approach to enlarge learning space coverage for robot learningcontrol

In robot learning control, the learning space for executing general motions of multijoint robot manipulators is quite large. Consequently, for most learning schemes, the learning controllers are used as subordinates to conventional controllers or the learning process needs to be repeated each time a new trajectory is encountered, although learning controllers are considered to be capable of generalization. In this paper, we propose an approach for larger learning space coverage in robot learning control. In this approach, a new structure for learning control is proposed to organize information storage via effective memory management. The proposed structure is motivated by the concept of human motor program and consists mainly of a fuzzy system and a cerebellar model articulation controller (CMAC)-type neural network. The fuzzy system is used for governing a number of sampled motions in a class of motions. The CMAC-type neural network is used to generalize the parameters of the fuzzy system, which are appropriate for the governing of the sampled motions, to deal with the whole class of motions. Under this design, in some sense the qualitative fuzzy rules in the fuzzy system are generalized by the CMAC-type neural network and then a larger learning space can be covered. Therefore, the learning effort is dramatically reduced in dealing with a wide range of robot motions, while the learning process is performed only once. Simulations emulating ball carrying under various conditions are presented to demonstrate the effectiveness of the proposed approach     

Robot Control Optimization Using Reinforcement Learning

Conventional robot control schemes are basically model-based methods. However, exact modeling of robot dynamics poses considerable problems and faces various uncertainties in task execution. This paper proposes a reinforcement learning control approach for overcoming such drawbacks. An artificial neural network (ANN) serves as the learning structure, and an applied stochastic real-valued (SRV) unit as the learning method. Initially, force tracking control of a two-link robot arm is simulated to verify the control design. The simulation results confirm that even without information related to the robot dynamic model and environment states, operation rules for simultaneous controlling force and velocity are achievable by repetitive exploration. Hitherto, however, an acceptable performance has demanded many learning iterations and the learning speed proved too slow for practical applications. The approach herein, therefore, improves the tracking performance by combining a conventional controller with a reinforcement learning strategy. Experimental results demonstrate improved trajectory tracking performance of a two-link direct-drive robot manipulator using the proposed method.     

Adaptive Impedance Control for Upper-Limb Rehabilitation Robot Using Evolutionary Dynamic Recurrent Fuzzy Neural Network

Control system implementation is one of the major difficulties in rehabilitation robot design. A newly developed adaptive impedance controller based on evolutionary dynamic fuzzy neural network (EDRFNN) is presented, where the desired impedance between robot and impaired limb can be regulated in real time according to the impaired limb??s physical recovery condition. Firstly, the impaired limb??s damping and stiffness parameters for evaluating its physical recovery condition are online estimated by using a slide average least squares (SALS)identification algorithm. Then, hybrid learning algorithms for EDRFNN impedance controller are proposed, which comprise genetic algorithm (GA), hybrid evolutionary programming (HEP) and dynamic back-propagation (BP) learning algorithm. GA and HEP are used to off-line optimize DRFNN parameters so as to get suboptimal impedance control parameters. Dynamic BP learning algorithm is further online fine-tuned based on the error gradient descent method. Moreover, the convergence of a closed loop system is proven using the discrete-type Lyapunov function to guarantee the global convergence of tracking error. Finally, simulation results show that the proposed controller provides good dynamic control performance and robustness with regard to the change of the impaired limb??s physical condition.     

Tracking control of a nonholonomic mobile robot using compound cosine function neural networks

The purpose of this paper is to propose a compound cosine function neural network with continuous learning algorithm for the velocity and orientation angle tracking control of a nonholonomic mobile robot with nonlinear disturbances. Herein, two neural network (NN) controllers embedded in the closed-loop control system have the simple continuous learning and rapid convergence capability without the dynamics information of the mobile robot to realize the adaptive control of the mobile robot. The neuron function of the hidden layer in the three-layer feed-forward network structure is on the basis of combining a cosine function with a unipolar sigmoid function. The developed neural network controllers have simple algorithm and fast learning convergence because the weight values are only adjusted between the nodes in hidden layer and the output nodes, while the weight values between the input layer and the hidden layer are one, i.e. constant, without the weight adjustment. Therefore, the main advantages of this control system are the real-time control capability and the robustness by use of the proposed neural network controllers for a nonholonomic mobile robot with nonlinear disturbances. Through simulation experiments applied to the nonholonomic mobile robot with the nonlinear disturbances which are considered as dynamics uncertainty and external disturbances, the simulation results show that the proposed NN control system of nonholonomic mobile robots has real-time control capability, better robustness and higher control precision. The compound cosine function neural network provides us with a new way to solve tracking control problems for mobile robots.     

基于机器视觉的目标定位与机器人规划系统研究

为完成机械臂在非特定复杂背景环境下的自主抓取,通过设计RGB-D相机对场景内的物体进行实时检测,采用基于深度学习的目标检测定位方法,并对相机-机械臂-目标物体的三维标定模型进行研究。将物体的三维坐标信息通过ROS话题机制发送给机械臂,并通过moveIT编程规划抓取规划。 通过设计一套基于ROS的视觉检测和机械臂抓取系统,将计算机视觉检测技术以及机械臂运动规划抓取应用在机器人操作系统ROS平台上。实验结果表明,该系统可以实时高效地操作机器人来完成指定的控制作业,提高了系统对环境的适应能力,该系统具有抓取准确、物体识别准确率高的特点,解决了传统机械臂操控中的不足。     

Learning model-free robot control by a Monte Carlo EM algorithm

We address the problem of learning robot control by model-free reinforcement learning (RL). We adopt the probabilistic model for model-free RL of Vlassis and Toussaint (Proceedings of the international conference on machine learning, Montreal, Canada, 2009), and we propose a Monte Carlo EM algorithm (MCEM) for control learning that searches directly in the space of controller parameters using information obtained from randomly generated robot trajectories. MCEM is related to, and generalizes, the PoWER algorithm of Kober and Peters (Proceedings of the neural information processing systems, 2009). In the finite-horizon case MCEM reduces precisely to PoWER, but MCEM can also handle the discounted infinite-horizon case. An interesting result is that the infinite-horizon case can be viewed as a ‘randomized’ version of the finite-horizon case, in the sense that the length of each sampled trajectory is a random draw from an appropriately constructed geometric distribution. We provide some preliminary experiments demonstrating the effects of fixed (PoWER) vs randomized (MCEM) horizon length in two simulated and one real robot control tasks.     

Task-space neuro-sliding mode control of robot manipulators under Jacobian uncertainties

Cartesian robot control is an appealing scheme because it avoids the computation of inverse kinematics, in contrast to joint robot control approach. For tracking, high computational load is typically required to obtain Cartesian robot dynamics. In this paper, an alternative approach for Cartesian tracking is proposed under assumption that robot dynamics is unknown and the Jacobian are uncertain. A neuro-sliding second order mode controller delivers a low dimensional neural network, which roughly estimates inverse robot dynamics, and an inner smooth control loop guarantees exponential tracking. Experimental results are presented to confirm the performance in a real time environment.     

Optimal unsupervised motor learning for dimensionality reduction ofnonlinear control systems

In this paper, optimal unsupervised motor learning is defined to be a technique for finding the coordinate system of minimum dimensionality which can adequately describe a particular motor task. An explicit method is provided for learning a stable controller that translates commands within the new coordinate system into motor variables appropriate for plant control. The method makes use of previously described neural network algorithms including the generalized Hebbian algorithm, basis-function trees, and trajectory extension learning. Examples of applications to a real direct-drive two joint planar robot arm and a simulated three joint robot arm with visual sensing are given.     

The Application of Connectionist Structures to Learning Impedance Control in Robotic Contact Tasks

The goal of this paper is to consider the synthesis of learning impedance control using recurrent connectionist structures for on-line learning of robot dynamic uncertainties in the case of robot contact tasks. The connectionist structures are integrated in non-learning impedance control laws that are intended to improve the transient dynamic response immediately after the contact. The recurrent neural network as a part of hybrid learning control algorithms uses fast learning rules and available sensor information in order to improve the robotic performance progressively for a minimum possible number of learning epochs. Some simulation results of deburring process with the MANUTEC r3 robot are presented here in order to verify the effectiveness of the proposed control learning algorithms.     

Policy gradient learning for quadruped soccer robots

In real-world robotic applications, many factors, both at low level (e.g., vision, motion control and behaviors) and at high level (e.g., plans and strategies) determine the quality of the robot performance. Consequently, fine tuning of the parameters, in the implementation of the basic functionalities, as well as in the strategic decisions, is a key issue in robot software development. In recent years, machine learning techniques have been successfully used to find optimal parameters for typical robotic functionalities. However, one major drawback of learning techniques is time consumption: in practical applications, methods designed for physical robots must be effective with small amounts of data. In this paper, we present a method for concurrent learning of best strategy and optimal parameters using policy gradient reinforcement learning algorithm. The results of our experimental work in a simulated environment and on a real robot show a very high convergence rate.     

基于改进型BP网络的并联机器人自适应力控制

并联机器人力控制是并联机器人研究的一个热点和难点,引起了许多学者的关注,并取得了一定的成果。多数使用了传统的力控制研究方法。该文中,作者将神经网络引入并联机器人的力控制中,并介绍了一种改进型BP神经网络,以及其学习算法和网络的训练过程,并结合实际并联机器人6-SPS并联机器人,设计出基于改进型BP神经网络的并联机器人自适应力控制器,并进行了仿真和实验研究,通过研究表明所设计的控制器是可行和有效的。     

Stable neurovisual servoing for robot manipulators

In this paper, we propose a stable neurovisual servoing algorithm for set-point control of planar robot manipulators in a fixed-camera configuration an show that all the closed-loop signals are uniformly ultimately bounded (UUB) and converge exponentially to a small compact set. We assume that the gravity term and Jacobian matrix are unknown. Radial basis function neural networks (RBFNNs) with online real-time learning are proposed for compensating both gravitational forces and errors in the robot Jacobian matrix. The learning rule for updating the neural network weights, similar to a back propagation algorithm, is obtained from a Lyapunov stability analysis. Experimental results on a two degrees of freedom manipulator are presented to evaluate the proposed controller.     

Open-end human–robot interaction from the dynamical systems perspective: mutual adaptation and incremental learning

In this paper, we experimentally investigated the open-end interaction generated by the mutual adaptation between humans and robot. Its essential characteristic, incremental learning, is examined using the dynamical systems approach. Our research concentrated on the navigation system of a specially developed humanoid robot called Robovie and seven human subjects whose eyes were covered, making them dependent on the robot for directions. We used the usual feed-forward neural network (FFNN) without recursive connections and the recurrent neural network (RNN) for the robot control. Although the performances obtained with both the RNN and the FFNN improved in the early stages of learning, as the subject changed the operation by learning on its own, all performances gradually became unstable and failed. Next, we used a 'consolidation-learning algorithm' as a model of the hippocampus in the brain. In this method, the RNN was trained by both new data and the rehearsal outputs of the RNN not to damage the contents of current memory. The proposed method enabled the robot to improve performance even when learning continued for a long time (open-end). The dynamical systems analysis of RNNs supports these differences and also showed that the collaboration scheme was developed dynamically along with succeeding phase transitions.     

自律个体的一种遗传强化模型研究

自律个体的遗传强化模型是模拟实际生物进化机制的计算模型。本文利用进化算法和人工神经网络的研究方法,设计一种自律个体的遗传强化模型。该模型强调多层次学习,实现了先天的遗传学习进化和后天的个体神经系统学习进化的有机结合。本文同时将该模型应用于模拟机器人的生存控制,观察它在环境中的行为表现及经能力,取得了满意的实验结果。