Obstacle avoidance of redundant manipulators using neural networks based reinforcement learning

This paper proposes a new approach for solving the problem of obstacle avoidance during manipulation tasks performed by redundant manipulators. The developed solution is based on a double neural network that uses Q-learning reinforcement technique. Q-learning has been applied in robotics for attaining obstacle free navigation or computing path planning problems. Most studies solve inverse kinematics and obstacle avoidance problems using variations of the classical Jacobian matrix approach, or by minimizing redundancy resolution of manipulators operating in known environments. Researchers who tried to use neural networks for solving inverse kinematics often dealt with only one obstacle present in the working field. This paper focuses on calculating inverse kinematics and obstacle avoidance for complex unknown environments, with multiple obstacles in the working field. Q-learning is used together with neural networks in order to plan and execute arm movements at each time instant. The algorithm developed for general redundant kinematic link chains has been tested on the particular case of PowerCube manipulator. Before implementing the solution on the real robot, the simulation was integrated in an immersive virtual environment for better movement analysis and safer testing. The study results show that the proposed approach has a good average speed and a satisfying target reaching success rate.     

A fast approach for the robust trajectory planning of redunant robot manipulators

In this article, a fast approach for robust trajectory planning, in the task space, of redundant robot manipulators is presented. The approach is based on combining an original method for obstacle avoidance by the manipulator configuration with the traditional potential field approach for the motion planning of the end-effector. This novel method is based on formulating an inverse kinematics problem under an inexact context. This procedure permits dealing with the avoidance of obstacles with an appropriate and easy to compute null space vector; whereas the avoidance of singularities is attained by the proper pseudoinverse perturbation. Furthermore, it is also shown that this formulation allows one to deal effectively with the local minimum problem frequently associated with the potential field approaches. The computation of the inverse kinematics problem is accomplished by numerically solving a linear system, which includes the vector for obstacle avoidance and a scheme for the proper pseudoinverse perturbation to deal with the singularities and/or the potential function local minima. These properties make the proposed approach suitable for redundant robots operating in real time in a sensor-based environment. The developed algorithm is tested on the simulation of a planar redundant manipulator. From the results obtained it is observed that the proposed approach compares favorably with the other approaches that have recently been proposed. © 1995 John Wiley & Sons, Inc.     

A dual neural network for redundancy resolution of kinematically redundant manipulators subject to joint limits and joint velocity limits

In this paper, a recurrent neural network called the dual neural network is proposed for online redundancy resolution of kinematically redundant manipulators. Physical constraints such as joint limits and joint velocity limits, together with the drift-free criterion as a secondary task, are incorporated into the problem formulation of redundancy resolution. Compared to other recurrent neural networks, the dual neural network is piecewise linear and has much simpler architecture with only one layer of neurons. The dual neural network is shown to be globally (exponentially) convergent to optimal solutions. The dual neural network is simulated to control the PA10 robot manipulator with effectiveness demonstrated.     

An efficient local approach for the path generation of robot manipulators

In this article an efficient local approach for the path generation of robot manipulators is presented. The approach is based on formulating a simple nonlinear programming problem. This problem is considered as a minimization of energy with given robot kinematics and subject to the robot requirements and a singularities avoidance constraint. From this formulation a closed form solution is derived which has the properties that allows to pursue both singularities and obstacle avoidance simultaneously; and that it can incorporate global information. These properties enable the accomplishment of the important task that while a specified trajectory in the operational space can be closely followed, also a desired joint configuration can be attained accurately at a given time. Although the proposed approach is primarily developed for redundant manipulators, its application to nonredundant manipulators is examplified by considering a particular commercial manipulator.     

A primal-dual neural network for online resolving constrained kinematic redundancy in robot motion control.

This paper proposes a primal-dual neural network with a one-layer structure for online resolution of constrained kinematic redundancy in robot motion control. Unlike the Lagrangian network, the proposed neural network can handle physical constraints, such as joint limits and joint velocity limits. Compared with the existing primal-dual neural network, the proposed neural network has a low complexity for implementation. Compared with the existing dual neural network, the proposed neural network has no computation of matrix inversion. More importantly, the proposed neural network is theoretically proved to have not only a finite time convergence, but also an exponential convergence rate without any additional assumption. Simulation results show that the proposed neural network has a faster convergence rate than the dual neural network in effectively tracking for the motion control of kinematically redundant manipulators.     

The SDRE control of mobile base cooperative manipulators: Collision free path planning and moving obstacle avoidance

This work proposes application of a state-dependent Riccati equation (SDRE) controller for wheeled mobile cooperative manipulators. Implementation of the SDRE on a wheeled mobile manipulator (WMM) considering holonomic and non-holonomic constraints is difficult and leads to instability of the system. The present study introduces a method of controlling the WMMs including: a general formulation, state-dependent coefficient parameterization, and control structure of the SDRE. Overcoming the problem of instability of the WMM resulted in control design for a system of cooperative manipulators mounted on a wheeled mobile platform. Optimal load distribution (OLD) was employed to distribute the load between the cooperative arms. The presence of obstacles and the probability of a collision between multiple robots in a workspace are the motivations behind employment of the artificial potential field (APF) approach. Two cooperative manipulators mounted on a mobile platform retrieved from Scout robot were modeled and simulated for situations such as controlling multiple mobile bases (collision avoidance), a cooperative system of manipulators, and moving obstacle avoidance. The OLD improved the load capacity, precision, and stability in motion of the cooperative system. Compatibility of the APF within the structure of the SDRE controller is another promising aspect of this research.     

A Lagrangian network for kinematic control of redundant robotmanipulators

A recurrent neural network, called the Lagrangian network, is presented for the kinematic control of redundant robot manipulators. The optimal redundancy resolution is determined by the Lagrangian network through real-time solution to the inverse kinematics problem formulated as a quadratic optimization problem. While the signal for a desired velocity of the end-effector is fed into the inputs of the Lagrangian network, it generates the joint velocity vector of the manipulator in its outputs along with the associated Lagrange multipliers. The proposed Lagrangian network is shown to be capable of asymptotic tracking for the motion control of kinematically redundant manipulators.     

基于BP神经网络的移动机器人避障设计及仿真

为了解决移动机器人在复杂环境中如何高效精确地躲避障碍物的问题,提出了一种基于BP神经网络的避障方法。建立了机器人的避障运动模型并设计了神经网络避障控制系统;分析了机器人在运动过程中与障碍物的位置关系,使用超声波传感器采集距离信息,进行BP神经网络输入、输出训练并采用Matlab工具进行仿真试验。结果表明,该方法可以高效精确地实现移动机器人的自主避障,运行相对稳定、轨迹连续平滑,达到了较为理想的避障效果。验证了方法的可行性和有效性,为移动机器人自主避障提供了一种新的控制方法。     

基于解析法和遗传算法的机械手运动学逆解

研究优化机械手轨迹规划问题,机械手运动时要具有稳定性避障性能。针对平面3自由度冗余机械手优化控制问题,建立机械手的结构模型。提出用解析法和遗传算法相结合满足具有计算量小和适应性强的特点。在给定机械手末端执行器的运动轨迹,按着机械手冗余自由度,运动轨迹上每个点对应的关节角有无穷多个解。而通过算法可以找到一组最优的关节角,可得到优化机械手运动过程中柔顺性和避障点。仿真结果表明,该算法可以快速收敛到全局最优解,可用于计算冗余机械手运动学逆解,并可实现机器人的轨迹规划和避障优化控制。     

A Neural Network Adaptive Controller for End-effector Tracking of Redundant Robot Manipulators

In this paper we propose a neural network adaptive controller to achieve end-effector tracking of redundant robot manipulators. The controller is designed in Cartesian space to overcome the problem of motion planning which is closely related to the inverse kinematics problem. The unknown model of the system is approximated by a decomposed structure neural network. Each neural network approximates a separate element of the dynamical model. These approximations are used to derive an adaptive stable control law. The parameter adaptation algorithm is derived from the stability study of the closed loop system using Lyapunov approach with intrinsic properties of robot manipulators. Two control strategies are considered. First, the aim of the controller is to achieve good tracking of the end-effector regardless the robot configurations. Second, the controller is improved using augmented space strategy to ensure minimum displacements of the joint positions of the robot. Simulation examples are also presented to verify the effectiveness of the proposed approach.     

冗余度卫星搭载机械臂神经网络优化控制

研究了节点变换函数参数可调的多层神经网络,并将这种神经网络用于控制冗余度卫星搭载机器人系统;提出了卫星位姿最小摄动的冗余度机器人臂神经网络控制方案,在这种方案中将系统的正运动学方程作为神经网络学习的对象,由神经网络学习拟合系统运动学,实现了机器人臂运动和卫星运动的解耦,在机器人臂的运动和卫星本体解耦的基础上,基于带动量项的BP解法,实现了系统的最小动能优化控制,利用平面4关节星载机器人系统进行了仿真,取得比较理想的效果。     

基于简单模糊逻辑的冗余度机器人动力学路径规划新方法

论文提出了基于模糊逻辑的冗余度机器人避障算法,算法中利用模糊规则自动调整避障控制参数,使机器人在最短的时间内做出快速回避反应,提高了系统的智能性;另一方面,算法中利用动力学控制代替了位置控制,减少了避障过程中,由于速度突变引起的关节冲击力,提高了系统使用寿命。最后通过一平面三连杆三自由度机器人进行仿真研究,结果证实了该算法的有效性。     

基于二次型规划的平面冗余机械臂的自运动

提出一种基于二次型性能指标的方法,用于规划平面冗余机械臂的自运动轨迹．鉴于实际的机械臂都存在关节物理约束,该自运动规划方案考虑了关节极限和关节速度极限的躲避．提出了基于线性变分不等式的原对偶神经网络,并将其作为所对应的二次型规划方案的实时求解器．仿真结果证实了该基于神经网络的自运动规划方案的有效性．     

A novel formulation for determining joint constraint loads during optimal dynamic motion of redundant manipulators in DH representation

The kinematic representations of general open-loop chains in many robotic applications are based on the Denavit–Hartenberg (DH) notation. However, when the DH representation is used for kinematic modeling, the relative joint constraints cannot be described explicitly using the common formulation methods. In this paper, we propose a new formulation of solving a system of differential-algebraic equations (DAEs) where the method of Lagrange multipliers is incorporated into the optimization problem for optimal motion planning of redundant manipulators. In particular, a set of fictitious joints is modeled to solve for the joint constraint forces and moments, as well as the optimal dynamic motion and the required actuator torques of redundant manipulators described in DH representation. The proposed method is formulated within the framework of our earlier study on the generation of load-effective optimal dynamic motions of redundant manipulators that guarantee successful execution of given tasks in which the Lagrangian dynamics for general external loads are incorporated. Some example tasks of a simple planar manipulator and a high-degree-of-freedom digital human model are illustrated, and the results show accurate calculation of joint constraint loads without altering the original planned motion. The proposed optimization formulation satisfies the equivalent DAEs.     

A global approach for the optimal path generation of redundant robot manipulators

In this article the optimal path generation of redundant robot manipulators is considered as an optimization problem, with given kinematics and subject to the robot requirements and a singularities avoidance constraint. This problem is formulated as a constrained continuous optimal control problem, which allows to consider joints and velocities constraints and/or manipulator dynamics. This approach is exemplified for a planar redundant manipulator and the resultant state constrained problem is solved by an efficient iterative numerical technique.     

Dynamic optimal payload path planning of mobile manipulators among moving obstacles

This paper is dealt with dynamic analysis of the wheeled mobile manipulators in the presence of moving obstacles considering optimal payload criterion. General dynamic formulation of the system was derived, and the moving obstacle avoidance strategy was proposed in terms of dynamic potential functions. The problem of dynamic motion planning and payload maximization was formulated using open-loop optimal control theory. Then, the indirect solution based on Pontryagin’s minimum principle was employed to solve the problem. Using the proposed method, complete nonlinear states and control constraints were treated without any simplifications such as linearizing the dynamics equations, discretizing the robot’s workspace, or parameterizing the solution. The proposed method will be useful for the system design and in the situation where the trajectories of obstacles are predefined. Finally, capability and applicability of the proposed method were investigated by the number of simulations on a two-link mobile manipulator.     

Repetitive motion of redundant robots planned by three kinds of recurrent neural networks and illustrated with a four-link planar manipulator’s straight-line example

In this paper, a dual neural network, LVI (linear variational inequalities)-based primal-dual neural network and simplified LVI-based primal-dual neural network are presented for online repetitive motion planning (RMP) of redundant robot manipulators (with a four-link planar manipulator as an example). To do this, a drift-free criterion is exploited in the form of a quadratic performance index. In addition, the repetitive-motion-planning scheme could incorporate the joint physical limits such as joint limits and joint velocity limits simultaneously. Such a scheme is finally reformulated as a quadratic program (QP). As QP real-time solvers, the aforementioned three kinds of neural networks all have piecewise-linear dynamics and could globally exponentially converge to the optimal solution of strictly-convex quadratic-programs. Furthermore, the neural-network based RMP scheme is simulated based on a four-link planar robot manipulator. Computer-simulation results substantiate the theoretical analysis and also show the effective remedy of the joint angle drift problem of robot manipulators.     

Bi-criteria Velocity Minimization of Robot Manipulators Using a Linear Variational Inequalities-Based Primal-Dual Neural Network and PUMA560 Example

In this paper, to diminish discontinuity points arising in the infinity-norm velocity minimization scheme, a bi-criteria velocity minimization scheme is presented based on a new neural network solver (i.e., a primal-dual neural network based on linear variational inequalities (LVI)). Such a kinematic control scheme of redundant manipulators can incorporate joint physical limits such as joint limits and joint velocity limits simultaneously. Moreover, the presented kinematic control scheme can be formulated as a quadratic programming (QP) problem. As a real-time QP solver, the LVI-based primal-dual neural network is established with a simple piecewise linear structure and higher computational efficiency. Computer simulations performed based on a PUMA560 manipulator are presented to illustrate the validity and advantages of such a bi-criteria neural control scheme for redundant robots.     

Dynamic design,numerical solution and effective verification of acceleration-level obstacle-avoidance scheme for robot manipulators

For avoiding obstacles and joint physical constraints of robot manipulators, this paper proposes and investigates a novel obstacle avoidance scheme (termed the acceleration-level obstacle-avoidance scheme). The scheme is based on a new obstacle-avoidance criterion that is designed by using the gradient neural network approach for the first time. In addition, joint physical constraints such as joint-angle limits, joint-velocity limits and joint-acceleration limits are incorporated into such a scheme, which is further reformulated as a quadratic programming (QP). Two important ‘bridge’ theorems are established so that such a QP can be converted equivalently to a linear variational inequality and then equivalently to a piecewise-linear projection equation (PLPE). A numerical algorithm based on a PLPE is thus developed and applied for an online solution of the resultant QP. Four path-tracking tasks based on the PA10 robot in the presence of point and window-shaped obstacles demonstrate and verify the effectiveness and accuracy of the acceleration-level obstacle-avoidance scheme. Besides, the comparisons between the non-obstacle-avoidance and obstacle-avoidance results further validate the superiority of the proposed scheme.     

Motion planning for mobile manipulators with a non-holonomic constraint using the FSP (full space parameterization) method

The efficient utilization of the motion capabilities of mobile manipulators, i.e., manipulators mounted on mobile platforms, requires the resolution of the kinematically redundant system formed by the addition of the degrees of freedom (DOF) of the platform to those of the manipulator. At the velocity level, the linearized Jacobian equation for such a redundant system represents an underspecified system of algebraic equations, which can be subject to a varying set of contraints such as a non-holonomic constraint on the platform motion, obstacles in the workspace, and various limits on the joint motions. A method, which we named the Full Space Parameterization (FSP), has recently been developed to resolve such underspecified systems with constraints that may vary in time and in number during a single trajectory. In this article, we first review the principles of the FSP and give analytical solutions for constrained motion cases with a general optimization criterion for resolving the redundancy. We then focus on the solutions to (1) the problem introduced by the combined use of prismatic and revolute joints (a common occurrence in practical mobile manipulators), which makes the dimensions of the joint displacement vector components non-homogeneous, and (2) the treatment of a non-holonomic constraint on the platform motion. Sample implementations on several large-payload mobile manipulators with up to 11 DOF are discussed. Comparative trajectories involving combined motions of the platform and manipulator for problems with obstacle and joint limit constraints, and with non-holonomic contraints on the platform motions, are presented to illustrate the use and efficiency of the FSP approach in complex motion planning problems. © 1996 John Wiley & Sons, Inc.