Homography-based visual servo tracking control of a wheeled mobile robot

A visual servo tracking controller is developed in this paper for a monocular camera system mounted on an underactuated wheeled mobile robot (WMR) subject to nonholonomic motion constraints (i.e., the camera-in-hand problem). A prerecorded image sequence (e.g., a video) of three target points is used to define a desired trajectory for the WMR. By comparing the target points from a stationary reference image with the corresponding target points in the live image and the prerecorded sequence of images, projective geometric relationships are exploited to construct Euclidean homographies. The information obtained by decomposing the Euclidean homography is used to develop a kinematic controller. A Lyapunov-based analysis is used to develop an adaptive update law to actively compensate for the lack of depth information required for the translation error system. Experimental results are provided to demonstrate the control design.     

基于图像的双臂模糊自适应轨迹跟踪控制

在固定相机的监视下,随机位姿的目标物体在双臂系统的控制下执行轨迹跟踪控制任务时,既要考虑目标随机位姿引起的运动学和动力学不确定性问题,还要考虑运动学和动力学的协同问题.针对上述问题,分别采用自适应方法估计目标质心和特征点的位置信息,利用模糊逻辑系统逼近系统的动力学模型,使用分散控制策略处理双臂的协同问题,最后基于位置/力混合控制方法设计基于图像的双臂模糊自适应轨迹跟踪控制器,并采用李亚普诺夫方法证明系统的稳定性.仿真实验验证了所设计控制器的有效性.     

Homography-based visual servo regulation of mobile robots.

A monocular camera-based vision system attached to a mobile robot (i.e., the camera-in-hand configuration) is considered in this paper. By comparing corresponding target points of an object from two different camera images, geometric relationships are exploited to derive a transformation that relates the actual position and orientation of the mobile robot to a reference position and orientation. This transformation is used to synthesize a rotation and translation error system from the current position and orientation to the fixed reference position and orientation. Lyapunov-based techniques are used to construct an adaptive estimate to compensate for a constant, unmeasurable depth parameter, and to prove asymptotic regulation of the mobile robot. The contribution of this paper is that Lyapunov techniques are exploited to craft an adaptive controller that enables mobile robot position and orientation regulation despite the lack of an object model and the lack of depth information. Experimental results are provided to illustrate the performance of the controller.     

移动机器人自适应视觉伺服镇定控制

对有单目视觉的移动机器人系统,提出了一种自适应视觉伺服镇定控制算法;在缺乏深度信息传感器并且摄像机外参数未知的情况下,该算法利用视觉反馈实现了移动机器人位置和姿态的渐近稳定.由于机器人坐标系与摄像机坐标系之间的平移外参数(手眼参数)是未知的,本文利用静态特征点的位姿变化特性,建立移动机器人在摄像机坐标系下的运动学模型.然后,利用单应矩阵分解的方法得到了可测的角度误差信号,并结合2维图像误差信号,通过一组坐标变换,得到了系统的开环误差方程.在此基础之上,基于Lyapunov稳定性理论设计了一种自适应镇定控制算法.理论分析、仿真与实验结果均证明了本文所设计的单目视觉控制器在摄像机外参数未知的情况下,可以使移动机器人渐近稳定到期望的位姿.     

Evolutionary algorithm based offline/online path planner for UAV navigation

An evolutionary algorithm based framework, a combination of modified breeder genetic algorithms incorporating characteristics of classic genetic algorithms, is utilized to design an offline/online path planner for unmanned aerial vehicles (UAVs) autonomous navigation. The path planner calculates a curved path line with desired characteristics in a three-dimensional (3-D) rough terrain environment, represented using B-spline curves, with the coordinates of its control points being the evolutionary algorithm artificial chromosome genes. Given a 3-D rough environment and assuming flight envelope restrictions, two problems are solved: i) UAV navigation using an offline planner in a known environment, and, ii) UAV navigation using an online planner in a completely unknown environment. The offline planner produces a single B-Spline curve that connects the starting and target points with a predefined initial direction. The online planner, based on the offline one, is given on-board radar readings which gradually produces a smooth 3-D trajectory aiming at reaching a predetermined target in an unknown environment; the produced trajectory consists of smaller B-spline curves smoothly connected with each other. Both planners have been tested under different scenarios, and they have been proven effective in guiding an UAV to its final destination, providing near-optimal curved paths quickly and efficiently.     

Visual Servoing Path Planning via Homogeneous Forms and LMI Optimizations

Path planning is a useful technique for visual servoing as it allows one to take into account system constraints and achieve desired performances during the camera motion. In this paper, we propose a new framework for path planning based on the use of homogeneous forms and linear matrix inequalities (LMIs). Specifically, we introduce a general parametrization of the trajectories from the initial to the desired location based on homogeneous forms and a parameter-dependent version of the Rodrigues formula. This allows us to impose typical constraints (field of view, workspace, joint, avoidance of collision, and occlusion) via positivity conditions on suitable homogeneous forms. Then, we reformulate the problem of finding a trajectory in the 3-D space satisfying all these constraints as an LMI optimization that can handle the maximization of typical performances (e.g., visibility margin, similarity to a straight line). The planned camera path is tracked by using an image-based controller. The proposed approach is illustrated and validated through simulations and experiments.     

Adaptive position and orientation regulation for the camera‐in‐hand problem

This paper considers the camera‐space position and orientation regulation problem for the camera‐in‐hand problem via visual serving in the presence of parametric uncertainty associated with the robot dynamics and the camera system. Specifically, an adaptive robot controller is developed that forces the end‐effector of a robot manipulator to move such that the position and orientation of an object are regulated to a desired position and orientation in the camera‐space, despite parametric uncertainty throughout the entire robot‐camera system. An extension is also provided that illustrates how slight modifications can be made to the camera‐in‐hand control law to achieve adaptive position and orientation tracking of the end‐effector in the camera‐space for a fixed‐camera configuration. Simulation results are provided to illustrate the performance of the adaptive, camera‐in‐hand controller. © 2005 Wiley Periodicals, Inc.     

Aggressive Longitudinal Aircraft Path Tracking Using Nonlinear Control

We study the problem of converting a trajectory tracking controller to a path tracking controller for a nonlinear non-minimum phase longitudinal aircraft model. The solution of the trajectory tracking problem is based on the requirement that the aircraft follows a given time parameterized trajectory in inertial frame. In this paper we introduce an alternative nonlinear control design approach called path tracking control. The path tracking approach is based on designing a nonlinear state feedback controller that maintains a desired speed along a desired path with closed loop stability. This design approach is different from the trajectory tracking approach where aircraft speed and position are regulated along the desired path. The path tracking controller regulates the position errors transverse to the desired path but it does not regulate the position error along the desired path. First, a trajectory tracking controller, consisting of feedforward and static state feedback, is designed to guarantee uniform asymptotic trajectory tracking. The feedforward is determined by solving a stable noncausal inversion problem. Constant feedback gains are determined based on LQR with singular perturbation approach. A path tracking controller is then obtained from the trajectory tracking controller by introducing a suitable state projection.     

Dynamic Visual Tracking for Manipulators Using an Uncalibrated Fixed Camera

This paper presents a new controller for controlling a number of feature points on a robot manipulator to trace desired trajectories specified on the image plane of a fixed camera. It is assumed that the intrinsic and extrinsic parameters of the camera are not calibrated. A new adaptive algorithm is developed to estimate the unknown parameters online, based on three original ideas. First, we use the pseudoinverse of the depth-independent interaction matrix to map the image errors onto the joint space of the manipulator. By eliminating the depths in the interaction matrix, we can linearly parameterize the closed-loop dynamics of the manipulator. Second, to guarantee the existence of the pseudoinverse, the adaptive algorithm introduces a potential force to drive the estimated parameters away from the values that result in a singular Jacobian matrix. Third, to ensure that the estimated parameters are convergent to their true values up to a scale, we combine the Slotine-Li method with an online algorithm for minimizing the error between the estimated projections and real image coordinates of the feature points. We have proved asymptotic convergence of the image errors to zero by the Lyapunov theory based on the nonlinear robot dynamics. Experiments have been carried out to verify the performance of the proposed controller.     

基于图像误差的机器人运动目标自适应跟踪

讨论了机械手运动目标跟踪问题，直接利用图像误差达到注视目的。利用手部摄像机于期望位置获取待跟踪物体图像，以此作为期望图像，而后可以对同为运动物体进行注视跟踪，使实时采样图像收敛于期望图像。     

Indirect iterative learning control for a discrete visual servo without a camera-robot model.

This paper presents a discrete learning controller for vision-guided robot trajectory imitation with no prior knowledge of the camera-robot model. A teacher demonstrates a desired movement in front of a camera, and then, the robot is tasked to replay it by repetitive tracking. The imitation procedure is considered as a discrete tracking control problem in the image plane, with an unknown and time-varying image Jacobian matrix. Instead of updating the control signal directly, as is usually done in iterative learning control (ILC), a series of neural networks are used to approximate the unknown Jacobian matrix around every sample point in the demonstrated trajectory, and the time-varying weights of local neural networks are identified through repetitive tracking, i.e., indirect ILC. This makes repetitive segmented training possible, and a segmented training strategy is presented to retain the training trajectories solely within the effective region for neural network approximation. However, a singularity problem may occur if an unmodified neural-network-based Jacobian estimation is used to calculate the robot end-effector velocity. A new weight modification algorithm is proposed which ensures invertibility of the estimation, thus circumventing the problem. Stability is further discussed, and the relationship between the approximation capability of the neural network and the tracking accuracy is obtained. Simulations and experiments are carried out to illustrate the validity of the proposed controller for trajectory imitation of robot manipulators with unknown time-varying Jacobian matrices.     

Experiments on manipulator gross motion using self-tuning controller and visual information

A picture of a scene is used to extract information for an adaptive control algorithm. The object of interest is first located by means of a classifier. The position and orientation of the object are determined from a binary picture. The desired path that the gripper of the manipulator is to follow is specified by discrete points, first in the Cartesian and then in the joint coordinate system. The adaptive self-tuning controller is outlined. The controller design is based on a time series difference equation model in which the parameters are estimated on-line. The gains of the controller are tuned so that a quadratic performance criterion is minimized. The performance of the system designed is then tested experimentally, and the results are presented.     

具有深度自适应估计的视觉伺服优化

在手眼机器人视觉伺服中,如何确定机器人末端摄像机移动的速度和对物体的深度进行有效的估计还没有较好的解决方法.本文采用一般模型法,通过求解最优化控制问题来设计摄像机的速度,同时,利用物体初始及期望位置的深度估计值,提出了一种自适应估计的算法对物体的深度进行估计,给出了深度变化趋势,实现了基于图像的定位控制.该方法能够使机器人在工作空间范围内从任一初始位置出发到达期望位置,实现了系统的全局渐近稳定且不需要物体的几何模型及深度的精确值.最后给出的仿真实例表明了本方法的有效性.     

A self-organizing video neuro-tracker

We investigate a radial basis function neural network for the control of a video camera for tracking a target moving at high speed. The adjusting of the weights and parameters of the network minimizes the performance measureP that is the sum of the squared errors between the target position on the screen and the center of the screen over an entire trajectory of the target. Every adjustment of a single weight requires the evaluation of the performance measure over the fixed target trajectory. A very general target trajectory is required for generalized training of the network to appropriately control any camera and target situation. The results show considerable promise for this and similar cases where a trainable controller is desired. This work was presented, in part, at the International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1996.     

Automated camera planning to film robot operations

Automatic 3D animation generation techniques are becoming increasingly popular in different areas related to computer graphics such as video games and animated movies. They help automate the filmmaking process even by non professionals without or with minimal intervention of animators and computer graphics programmers. Based on specified cinematographic principles and filming rules, they plan the sequence of virtual cameras that the best render a 3D scene. In this paper, we present an approach for automatic movie generation using linear temporal logic to express these filming and cinematography rules. We consider the filming of a 3D scene as a sequence of shots satisfying given filming rules, conveying constraints on the desirable configuration (position, orientation, and zoom) of virtual cameras. The selection of camera configurations at different points in time is understood as a camera plan, which is computed using a temporal-logic based planning system (TLPlan) to obtain a 3D movie. The camera planner is used within an automated planning application for generating 3D tasks demonstrations involving a teleoperated robot arm on the the International Space Station (ISS). A typical task demonstration involves moving the robot arm from one configuration to another. The main challenge is to automatically plan the configurations of virtual cameras to film the arm in a manner that conveys the best awareness of the robot trajectory to the user. The robot trajectory is generated using a path-planner. The camera planner is then invoked to find a sequence of configurations of virtual cameras to film the trajectory.     

Uncalibrated visual servoing of robots using a depth-independent interaction matrix

This paper presents a new adaptive controller for image-based dynamic control of a robot manipulator using a fixed camera whose intrinsic and extrinsic parameters are not known. To map the visual signals onto the joints of the robot manipulator, this paper proposes a depth-independent interaction matrix, which differs from the traditional interaction matrix in that it does not depend on the depths of the feature points. Using the depth-independent interaction matrix makes the unknown camera parameters appear linearly in the closed-loop dynamics so that a new algorithm is developed to estimate their values on-line. This adaptive algorithm combines the Slotine-Li method with on-line minimization of the errors between the real and estimated projections of the feature points on the image plane. Based on the nonlinear robot dynamics, we prove asymptotic convergence of the image errors to zero by the Lyapunov theory. Experiments have been conducted to verify the performance of the proposed controller. The results demonstrated good convergence of the image errors.     

机器人多指手抓取中的规划问题

在机器人抓取系统中，一般认为需要４种规划器：即策略规划器，触觉规划器，轨迹规划抓取规划器，抓取规划器对成功抓取来说是非常重要的，在抓取规划器中，视觉模块用来把图象变换成物体的描述，接着用抓取模式选择模块把物体的描述换成一系列的控制信号，本文从最优抓取规划和基于专家系统的抓取规划这两个方面，着重从基于专家系统的抓取规划方面对当前机器人多指手抓取规划的研究现状及主要问题进行了深入地剖析。     

微小无人直升机自适应视觉伺服

针对摄像机未标定和特征点坐标未知的情况, 本文提出一种新颖的基于图像的无人直升机自适应视觉伺服方法. 控制器是基于反推法设计的, 但是和已有的基于反推法的视觉伺服不同的是, 它利用与深度无关矩阵将图像误差映射到执行器空间, 从而可以避免估计特征点的深度. 这种设计方法可以线性化未知的摄像机参数和特征点坐标, 所以能方便地设计自适应算法来在线估计这些未知参数, 同时为了保证图像误差收敛和避免估计参数收敛至零解而引入了两个势函数. 利用Lyapunov方法证明了基于非线性动力学的控制器的稳定性, 并给出了仿真验证.     

Visual Servoing Path Planning for Cameras Obeying the Unified Model

Abstract

This paper proposes a path planning visual servoing strategy for a class of cameras that includes conventional perspective cameras, fisheye cameras and catadioptric cameras as special cases. Specifically, these cameras are modeled by adopting a unified model recently proposed in the literature and the strategy consists of designing image trajectories for eye-in-hand robotic systems that allow the robot to reach a desired location while satisfying typical visual servoing constraints. To this end, the proposed strategy introduces the projection of the available image features onto a virtual plane and the computation of a feasible image trajectory through polynomial programming. Then, the computed image trajectory is tracked by using an image-based visual servoing controller. Experimental results with a fisheye camera mounted on a 6-d.o.f. robot arm are presented in order to illustrate the proposed strategy.     

Visual tracking control for an uncalibrated robot system with unknown camera parameters

The robust trajectory tracking problem for an eye-in-hand system is addressed in this paper. A novel visual feedback control model is proposed. It considers not only the uncertainties and disturbances in the robot model, but also the unknown camera parameters. By using sliding mode control, filter method and adaptive technique, the controller is designed such that the robot can track the desired trajectory well by using information provided by camera. Finally, stability and robustness are rigorously proved by using Lyapunov method. Computer simulations are presented to show the effectiveness of the proposed visual feedback controller.