Neural Control for Driving a Mobile Robot Integrating Stereo Vision Feedback

This paper proposes a neural control integrating stereo vision feedback for driving a mobile robot. The proposed approach consists in synthesizing a suitable inverse optimal control to avoid solving the Hamilton Jacobi Bellman equation associated to nonlinear system optimal control. The mobile robot dynamics is approximated by an identifier using a discrete-time recurrent high order neural network, trained with an extended Kalman filter algorithm. The desired trajectory of the robot is computed during navigation using a stereo camera sensor. Simulation and experimental result are presented to illustrate the effectiveness of the proposed control scheme.     

Neural network learning from demonstration and epipolar geometry for visual control of a nonholonomic mobile robot

The control of a robot system using camera information is a challenging task regarding unpredictable conditions, such as feature point mismatch and changing scene illumination. This paper presents a solution for the visual control of a nonholonomic mobile robot in demanding real world circumstances based on machine learning techniques. A novel intelligent approach for mobile robots using neural networks (NNs), learning from demonstration (LfD) framework, and epipolar geometry between two views is proposed and evaluated in a series of experiments. A direct mapping from the image space to the actuator command is conducted using two phases. In an offline phase, NN–LfD approach is employed in order to relate the feature position in the image plane with the angular velocity for lateral motion correction. An online phase refers to a switching vision based scheme between the epipole based linear velocity controller and NN–LfD based angular velocity controller, which selection depends on the feature distance from the pre-defined interest area in the image. In total, 18 architectures and 6 learning algorithms are tested in order to find optimal solution for robot control. The best training outcomes for each learning algorithms are then employed in real time so as to discover optimal NN configuration for robot orientation correction. Experiments conducted on a nonholonomic mobile robot in a structured indoor environment confirm an excellent performance with respect to the system robustness and positioning accuracy in the desired location.     

Design of a sliding mode controller for an automatic guided vehicle and its implementation

In this paper, a control scheme that combines a kinematic controller and a sliding mode dynamic controller with external disturbances is proposed for an automatic guided vehicle to track a desired trajectory with a specified constant velocity. It provides a method of taking into account specific mobile robot dynamics to convert desired velocity control inputs into torques for the actual mobile robot. First, velocity control inputs are designed for the kinematic controller to make the tracking error vector asymptotically stable. Then, a sliding mode dynamic controller is designed such that the mobile robot’s velocities converge to the velocity control inputs. The control law is obtained based on the backstepping technique. System stability is proved using the Lyapunov stability theory. In addition, a scheme for measuring the errors using a USB camera is described. The simulation and experimental results are presented to illustrate the effectiveness of the proposed controller.     

PD control with on-line gravity compensation for robots with elastic joints: Theory and experiments

A proportional-derivative (PD) control with on-line gravity compensation is proposed for regulation tasks of robot manipulators with elastic joints. The work extends a previous PD control with constant gravity compensation at the desired configuration. The control law requires measuring only position and velocity on the motor side of the elastic joints, while the on-line gravity compensation torque uses a biased measure of the motor position. It is proved via a Lyapunov argument that the control law globally asymptotically stabilizes the desired robot configuration. A simulation study on a two-joint arm reveals the better performance that can be obtained with the new scheme as compared to the case of constant gravity compensation. Moreover, the proposed controller is experimentally tested on an eight-joint cable-driven robot manipulator, in combination with a point-to-point interpolating trajectory, showing the practical advantages of the on-line compensation.     

Self-calibration of environmental camera for mobile robot navigation

An environmental camera is a camera embedded in a working environment to provide vision guidance to a mobile robot. In the setup of such robot systems, the relative position and orientation between the mobile robot and the environmental camera are parameters that must unavoidably be calibrated. Traditionally, because the configuration of the robot system is task-driven, these kinds of external parameters of the camera are measured separately and should be measured each time a task is to be performed. In this paper, a method is proposed for the robot system in which calibration of the environmental camera is rendered by the robot system itself on the spot after a system is set up. Specific kinds of motion patterns of the mobile robot, which are called test motions, have been explored for calibration. The calibration approach is based upon executing certain selected test motions on the mobile robot and then using the camera to observe the robot. According to a comparison of odometry and sensing data, the external parameters of the camera can be calibrated. Furthermore, an evaluation index (virtual sensing error) has been developed for the selection and optimization of test motions to obtain good calibration performance. All the test motion patterns are computed offline in advance and saved in a database, which greatly shorten the calibration time. Simulations and experiments verified the effectiveness of the proposed method.     

软体机器人手眼视觉/形状混合控制

当末端带有相机的连续型软体机器人进行作业时,由于避障、安全性等多方面因素,既需要末端相机－机器人系统的视觉伺服,也需要机器人的整体形状控制．针对这个问题,本文提出了一种软体机器人手眼视觉/形状混合控制方法．该方法无需知道空间特征点的3维坐标,只需给定特征点在末端相机像平面的期望像素坐标和软体机器人的期望形状就可达到控制目的．建立了软体机器人的运动学模型,利用该模型,结合深度无关交互矩阵自适应手眼视觉控制和软体机器人形状控制,提出了一种混合控制律,并用李亚普诺夫稳定性理论对该控制律进行证明．仿真和实验的结果均表明,末端相机特征点像素坐标和形状可以收敛到期望值．     

A single visual-servo controller of mobile robots with super-twisting control

This paper presents a novel approach for image-based visual servoing, extending the existing works that use the trifocal tensor (TT) as source for image measurements. In the proposed approach, singularities typically encountered in this kind of methods are avoided. A formulation of the TT-based control problem with a virtual target resulting from the vertical translation of the real target allows us to design a single controller, able to regulate the robot pose towards the desired configuration, without local minima. In this context, we introduce a super-twisting control scheme guaranteeing continuous control inputs, while exhibiting strong robustness properties. Our approach is valid for perspective cameras as well as catadioptric systems obeying the central camera model. All these contributions are supported by convincing numerical simulations and experiments under a popular dynamic robot simulator.     

Adaptive position and orientation regulation for the camera‐in‐hand problem

This paper considers the camera‐space position and orientation regulation problem for the camera‐in‐hand problem via visual serving in the presence of parametric uncertainty associated with the robot dynamics and the camera system. Specifically, an adaptive robot controller is developed that forces the end‐effector of a robot manipulator to move such that the position and orientation of an object are regulated to a desired position and orientation in the camera‐space, despite parametric uncertainty throughout the entire robot‐camera system. An extension is also provided that illustrates how slight modifications can be made to the camera‐in‐hand control law to achieve adaptive position and orientation tracking of the end‐effector in the camera‐space for a fixed‐camera configuration. Simulation results are provided to illustrate the performance of the adaptive, camera‐in‐hand controller. © 2005 Wiley Periodicals, Inc.     

Vision-based interception of a moving target with a nonholonomic mobile robot

A novel vision-based scheme is presented for driving a nonholonomic mobile robot to intercept a moving target. The proposed method has a two-level structure. On the lower level, the pan–tilt platform carrying the on-board camera is controlled so as to keep the target as close as possible to the center of the image plane. On the higher level, the relative position of the target is retrieved from its image coordinates and the camera pan–tilt angles through simple geometry, and used to compute a control law which drives the robot to the target. Various possible choices are discussed for the high-level robot controller, and the associated stability properties are rigorously analysed. The proposed visual interception method is validated through simulations as well as experiments on the mobile robot MagellanPro.     

Obstacle avoidance for redundant robots using configuration control

The article presents a new and simple solution to the obstacle avoidance problem for redundant robots. In the proposed approach, called configuration control, the redundancy is utilized to configure the robot so as to satisfy a set of kinematic inequality constraints representing obstacle avoidance, while the end-effector is tracking a desired trajectory. The robot control scheme is very simple, and uses on-line adaptation to eliminate the need for the complex dynamic model and parameter values of the robot. Several simulation results for a four-link planar robot are presented to illustrate the versatility of the approach. These include reaching around a stationary obstacle, simultaneous avoidance of two obstacles, robot reconfiguration to avoid a moving obstacle, and avoidance of rectangular obstacles. The simplicity and computational efficiency of the proposed scheme allows on-line implementation with a high sampling rate for real-time obstacle avoidance in a dynamically varying environment.     

Reaching for redundant arms with human-like motion and compliance properties

This work proposes a redundant arm torque controller for reaching, guaranteeing desired completion time and accuracy requirements without the need for trajectory planning and prior knowledge of robot dynamics. The proposed controller is designed based on the prescribed performance control methodology and it is a reaching regulator in which the target pose for the hand acts as an attractor for the arm. It provides configuration consistency in return motions and hand and joint velocity smoothness. Its use in an admittance control scheme given measurements or estimates of external forces is also proposed providing active compliance capabilities in robot–environment interactions. Simulation studies for a 5dof human arm-like robot and experiments with a 7dof arm are performed to verify the approach and demonstrate the proposed controller’s performance.     

具有深度自适应估计的视觉伺服优化

在手眼机器人视觉伺服中,如何确定机器人末端摄像机移动的速度和对物体的深度进行有效的估计还没有较好的解决方法.本文采用一般模型法,通过求解最优化控制问题来设计摄像机的速度,同时,利用物体初始及期望位置的深度估计值,提出了一种自适应估计的算法对物体的深度进行估计,给出了深度变化趋势,实现了基于图像的定位控制.该方法能够使机器人在工作空间范围内从任一初始位置出发到达期望位置,实现了系统的全局渐近稳定且不需要物体的几何模型及深度的精确值.最后给出的仿真实例表明了本方法的有效性.     

机器人手眼协调的图象直接反馈方法*

本文从视觉与控制集成观点出发，提出了一种机械人手眼协调的新方法，利用手部机于期望抓取（操作）位置获取待待抓（加工）物体图象，以此作为期望图象，而后可以对任意位置的同类物体进行实时抓取，首先将期望图象与实时采样图象进行比较生成差图象，从而依据差图象进行实时伺服反馈，最终达到消除图象差的目的。     

Visual servo walking control for humanoids with finite-time convergence and smooth robot velocities

In this paper, we address the problem of humanoid locomotion guided from information of a monocular camera. The goal of the robot is to reach a desired location defined in terms of a target image, i.e., a positioning task. The proposed approach allows us to introduce a desired time to complete the positioning task, which is advantageous in contrast to the classical exponential convergence. In particular, finite-time convergence is achieved while generating smooth robot velocities and considering the omnidirectional waking capability of the robot. In addition, we propose a hierarchical task-based control scheme, which can simultaneously handle the visual positioning and the obstacle avoidance tasks without affecting the desired time of convergence. The controller is able to activate or inactivate the obstacle avoidance task without generating discontinuous velocity references while the humanoid is walking. Stability of the closed loop for the two task-based control is demonstrated theoretically even during the transitions between the tasks. The proposed approach is generic in the sense that different visual control schemes are supported. We evaluate a homography-based visual servoing for position-based and image-based modalities, as well as for eye-in-hand and eye-to-hand configurations. The experimental evaluation is performed with the humanoid robot NAO.     

Reactive navigation in dynamic environment using a multisensorpredictor

A reactive navigation system for an autonomous mobile robot in unstructured dynamic environments is presented. The motion of moving obstacles is estimated for robot motion planning and obstacle avoidance. A multisensor-based obstacle predictor is utilized to obtain obstacle-motion information. Sensory data from a CCD camera and multiple ultrasonic range finders are combined to predict obstacle positions at the next sampling instant. A neural network, which is trained off-line, provides the desired prediction on-line in real time. The predicted obstacle configuration is employed by the proposed virtual force based navigation method to prevent collision with moving obstacles. Simulation results are presented to verify the effectiveness of the proposed navigation system in an environment with multiple mobile robots or moving objects. This system was implemented and tested on an experimental mobile robot at our laboratory. Navigation results in real environment are presented and analyzed.     

Visual tracking control for an uncalibrated robot system with unknown camera parameters

The robust trajectory tracking problem for an eye-in-hand system is addressed in this paper. A novel visual feedback control model is proposed. It considers not only the uncertainties and disturbances in the robot model, but also the unknown camera parameters. By using sliding mode control, filter method and adaptive technique, the controller is designed such that the robot can track the desired trajectory well by using information provided by camera. Finally, stability and robustness are rigorously proved by using Lyapunov method. Computer simulations are presented to show the effectiveness of the proposed visual feedback controller.     

Visual control through the trifocal tensor for nonholonomic robots

We present a new vision-based control approach which drives autonomously a nonholonomic vehicle to a target location. The vision system is a camera fixed on the vehicle and the target location is defined by an image taken previously in that location. The control scheme is based on the trifocal tensor model, which is computed from feature correspondences in calibrated retina across three views: initial, current and target images. The contribution is a trifocal-based control law defined by an exact input–output linearization of the trifocal tensor model. The desired evolution of the system towards the target is directly defined in terms of the trifocal tensor elements by means of sinusoidal functions without needing metric or additional information from the environment. The trifocal tensor presents important advantages for visual control purposes, because it is more robust than two-view geometry as it includes the information of a third view and, contrary to the epipolar geometry, short baseline is not a problem. Simulations show the performance of the approach, which has been tested with image noise and calibration errors.     

Adaptive control of free-floating space robots in Cartesian coordinates

An adaptive control scheme is proposed for the end-effector trajectory tracking control of free-floating space robots. In order to cope with the nonlinear parameterization problem of the dynamic model of the free-floating space robot system, the system is modeled as an extended robot which is composed of a pseudo-arm representing the base motions and a real robot arm. An on-line estimation of the unknown parameters along with a computed-torque controller is used to track the desired trajectory. The proposed control scheme does not require measurement of the accelerations of the base and the real robot arm. A two-link planar space robot system is simulated to illustrate the validity and effectiveness of the proposed control scheme.     

Redundant robot control using task based performance measures

The joint velocities required to move the end-effector of a redundant robot with a desired linear and angular velocity depend on its configuration. Similarly, the joint torques produced due to the force and moment at the end-effector also depend on its configuration. When the robot is near a singular configuration, the joint velocities required to attain the end-effector velocity in certain directions are extremely high. Similarly, in some configurations the joint torque produced at certain joints may be high for a relatively small magnitude of external force. An infinite number of trajectories in the joint space can be used to achieve a desired end-effector trajectory for redundant robots. However, a joint trajectory resulting in robot configurations requiring lower joint velocities or joint torques is desired. This may be achieved through a proper utilization of redundancy. Local performance measures for redundant robots are defined in this article as indicators of their ability to follow a desired end-effector trajectory and their ability to apply desired forces at the end-effector. Thus, these performance measures depend on the task to be performed. Control algorithms which can be efficiently applied to redundant robots to improve these performance measures are presented. These control algorithms are based on the gradient projection method. Gradients of the performance measures used in the control schemes result in simple symbolic expressions for “real world” robots'. Feasibility and effectiveness of these control schemes is demonstrated through the simulation of a seven-degree-of-freedom redundant robot derived from the PUMA geometry.     

无奇异间接迭代学习控制及其在机器人运动模仿中的应用

针对相当广泛的一类非线性系统有限时间轨迹跟踪问题,提出了间接迭代学习方案. 采用最小二乘算法,根据重复跟踪历史辨识非线性系统的线性化模型.利用一个分段学习方案 可保证学习控制总在有效线性近似区域内进行.探讨了如何在学习过程中避免控制奇异问题, 提出了一种高效的参数修正方法,保证输入耦合矩阵的估计行列式不为零.本文将这一控制方 案应用于未知机器人及摄像机模型下的机器人运动模仿中,而不面临任何奇异问题.这是一个 采用摄像机替代传统程序编写的新的机器人编程方法.