Visual Homing: Surfing on the Epipoles

We introduce a novel method for visual homing. Using this method a robot can be sent to desired positions and orientations in 3D space specified by single images taken from these positions. Our method is based on recovering the epipolar geometry relating the current image taken by the robot and the target image. Using the epipolar geometry, most of the parameters which specify the differences in position and orientation of the camera between the two images are recovered. However, since not all of the parameters can be recovered from two images, we have developed specific methods to bypass these missing parameters and resolve the ambiguities that exist. We present two homing algorithms for two standard projection models, weak and full perspective.Our method determines the path of the robot on-line, the starting position of the robot is relatively not constrained, and a 3D model of the environment is not required. The method is almost entirely memoryless, in the sense that at every step the path to the target position is determined independently of the previous path taken by the robot. Because of this property the robot may be able, while moving toward the target, to perform auxiliary tasks or to avoid obstacles, without this impairing its ability to eventually reach the target position. We have performed simulations and real experiments which demonstrate the robustness of the method and that the algorithms always converge to the target pose.     

Visual servoing in the task-function framework: A contour following task

We describe an approach to contour following unknown objects using a handeye robotic system. Relevant and sufficient feature points providing optical flow data are extracted from the edges of the target object. The desired motion of the end-effector is computed with the objective of keeping the visual features always at the same target location in the image plane. A cartesian PD controller is used to perform the desired motion by the robot's end-effector. To address thecontrol issues, we take advantage of the unifying robot control theory stated in the literature as thetask-function approach [21]. To validate our approach, we restricted our experiments to motionless objects positioned in a plane parallel to the image plane: three degrees of freedom (two of translation, one of rotation) are thus controlled.     

利用平面单应分解实现服务机器人视觉伺服

智能空间中家庭服务机器人所需完成的主要任务是协助人完成物品的搜寻、定位与传递。而视觉伺服则是完成上述任务的有效手段。搭建了由移动机器人、机械臂、摄像头组成的家庭服务机器人视觉伺服系统,建立了此系统的运动学模型并对安装在机械臂末端执行器上的视觉系统进行了内外参数标定,通过分解世界平面的单应来获取目标物品的位姿参数,利用所获取的位姿参数设计了基于位置的视觉伺服控制律。实验结果表明,使用平面单应分解方法来设计控制律可简单有效地完成家庭物品的视觉伺服任务。     

基于点线特征的解耦视觉伺服控制方法

针对机器人的自动对准问题,提出一种基于点线特征的解耦视觉伺服控制方法。所提方法以点和直线作为图像特征,并利用图像特征的交互矩阵解耦姿态控制和位置控制,实现六自由度对准。首先利用直线及其交互矩阵设计姿态控制律,以消除旋转偏差;然后利用点及其交互矩阵设计位置控制律,以消除位置偏差;最后实现机器人末端目标的自动对准。在对准控制过程中,基于执行的相机运动量以及相机运动前后特征的变化量,可实现对深度的在线估计。另外,还设计了监督器对相机的运动速度进行调节,从而确保特征一直处于相机视野当中。在Eye-in-Hand机器人平台上,分别用所提方法和传统的基于图像的视觉伺服方法实现了机器人的六自由度对准。所提方法经过16步实现了机器人的自动对准,对准结束时机器人末端位姿的最大平移误差为3.26 mm,最大旋转误差为0.72°。相较于对比方法,该方法的控制过程更加高效,控制误差收敛更快,对准误差更小。实验结果表明,所提方法可以实现快速高精度的自动对准,能够提高机器人操作的自主性和智能化水平,有望应用于目标跟踪、拾取和定位、自动化装配、焊接、服务机器人等领域。     

Application of Lie Algebras to Visual Servoing

A novel approach to visual servoing is presented, which takes advantage of the structure of the Lie algebra of affine transformations. The aim of this project is to use feedback from a visual sensor to guide a robot arm to a target position. The target position is learned using the principle of teaching by showing in which the supervisor places the robot in the correct target position and the system captures the necessary information to be able to return to that position. The sensor is placed in the end effector of the robot, the camera-in-hand approach, and thus provides direct feedback of the robot motion relative to the target scene via observed transformations of the scene. These scene transformations are obtained by measuring the affine deformations of a target planar contour (under the weak perspective assumption), captured by use of an active contour, or snake. Deformations of the snake are constrained using the Lie groups of affine and projective transformations. Properties of the Lie algebra of affine transformations are exploited to provide a novel method for integrating observed deformations of the target contour. These can be compensated with appropriate robot motion using a non-linear control structure. The local differential representation of contour deformations is extended to allow accurate integration of an extended series of small perturbations. This differs from existing approaches by virtue of the properties of the Lie algebra representation which implicitly embeds knowledge of the three-dimensional world within a two-dimensional image-based system. These techniques have been implemented using a video camera to control a 5 DoF robot arm. Experiments with this implementation are presented, together with a discussion of the results.     

基于Adept机器人的视觉伺服控制系统

机器人视觉伺服控制在理论和应用等方面还有许多问题需要研究,例如特征选择、系统标定和伺服控制算法等.针对Adept机器人,提出了一种简单快速的不需要精确标定摄像机内外部参数的摄像机标定方法,完成了从被观测物体表面所在的视觉平面坐标系到机器人基坐标系的坐标变换.使用图像的全局特征,即图像矩特征进行伺服跟踪;利用所推导的图像雅可比矩阵,设计了由图像反馈与目标运动自适应补偿组成的视觉伺服控制器.将算法对静态目标的定位实验进行了验证,然后又将其应用到移动目标的跟踪上,通过调节和优选控制参数,实现了稳定的伺服跟踪和抓取.实验结果表明采用图像矩作为图像特征能够避免复杂的特征匹配过程,并且能够获得较好的跟踪精度.     

基于双目视差和主动轮廓的机器人手眼协调控制技术研究

提出一种基于双目视差和主动轮廓的机器人手眼协调技术;该方法利用主动轮廓的思想动态地逼近和跟踪机器人及目标物体的外部轮廓,通过控制双目视差趋零来实现机器人靠近目标和抓取物体.首先建立了机器人手指轮廓的几何参数模型和相应的观测概率密度模型,利用CONDENSATION算法对轮廓进行动态逼近和跟踪.然后针对动态轮廓的几何特征,探讨了基于双目视差的手眼协调控制策略.最后利用这种手眼协调控制技术进行了机器人抓取圆球目标的实验.实验表明,这种方法具有对图像噪声不敏感的特点,可以在杂乱的背景环境和复杂的纹理条件下执行视觉引导的跟踪和抓取任务,具有较强的鲁棒性.     

一种无标定视觉伺服控制技术的研究

在视觉伺服控制过程中无法精确地标定摄像机和机器人运动学模型,而当前的无标定视觉伺服控制技术或者只能针对静态的目标,或者针对动态目标但无法摆脱大偏差的影响．针对此问题,提出一种动态无标定的视觉伺服控制方法：基于非线性方差最小化法控制机器人跟踪运动目标,利用动态拟牛顿法估计图像雅克比矩阵,采用迭代最小二乘法提高系统的稳定性并提出大偏差条件下的无标定控制策略．仿真实验证明了该方法的正确性和有效性．     

Target Reaching by Using Visual Information and Q-learning Controllers

This paper presents a solution to the problem of manipulation control: target identification and grasping. The proposed controller is designed for a real platform in combination with a monocular vision system. The objective of the controller is to learn an optimal policy to reach and to grasp a spherical object of known size, randomly placed in the environment. In order to accomplish this, the task has been treated as a reinforcement problem, in which the controller learns by a trial and error approach the situation-action mapping. The optimal policy is found by using the Q-Learning algorithm, a model free reinforcement learning technique, that rewards actions that move the arm closer to the target.The vision system uses geometrical computation to simplify the segmentation of the moving target (a spherical object) and determines an estimate of the target parameters. To speed-up the learning time, the simulated knowledge has been ported on the real platform, an industrial robot manipulator PUMA 560. Experimental results demonstrate the effectiveness of the adaptive controller that does not require an explicit global target position using direct perception of the environment.     

An efficient technique for finding the desired global optimum of robotic joint displacement

For an industrial robot on a daily operation basis such as pick and place, it is desired to minimize the robotic joint displacements when moving the robot from one location to another. The objective of the optimization here is to simultaneously minimize a robot end effector's positional error and the robotic joint displacements. By modifying the searching algorithm in the existing complex optimization method, this article presents a technique for finding the desired global optimum solution more efficiently. To compare the optimum searching capability between the proposed and existing searching algorithms, a modified Himmelblau's function is used as an objective function. The presented technique is then applied to a spatial three-link robot manipulator for global minimization of the joint displacements. © 1992 John Wiley & Sons, Inc.     

Real-Time Dynamic Visual Tracking Using PSD Sensors and Extended Trapezoidal Motion Planning

A real-time visual servo tracking system for an industrial robot has been implemented using PSD (Position Sensitive Detector) cameras, neural networks, and an extended trapezoidal motion planning method. PSD and directly transduces the light's projected position on its sensor plane into an analog current and lends itself to fast real-time tracking. A neural network, after proper training, transforms the PSD sensor reading into a 3D position of the target, which is then input to an extended trapezoidal motion planning algorithm. This algorithm implements a continuous motion update strategy in response to an ever-changing sensor information from the moving target, while greatly reducing the tracking delay. This planning method is found to be very useful for sensor-based control such as moving target tracking or weld-seam tracking in which the robot needs to change its motion in real time in response to incoming sensor information. Further, for real-time usage of the neural net, a new architecture called LANN (Locally Activated Neural Network) has been developed based on the concept of CMAC input partitioning and local learning. Experimental evidence shows that an industrial robot can smoothly track a moving target of unknown motion with speeds of up to 1 m/s and with oscillation frequency up to 5 Hz.     

一种新的双环结构机器人无标定自抗扰视觉伺服控制方法

在研究基于自抗扰控制器的机器人无标定视觉伺服方法的基础上,提出了一种新的双环结构机器人无标定自抗扰视觉伺服控制方法．内环采用Kalman滤波算法进行图像雅可比矩阵的在线辨识,可较好地逼近真实模型;外环采用自抗扰控制器,利用非线性观测器实时估计系统相对于当前估计模型的总扰动,并在控制中加以动态补偿．针对六自由度工业机器人进行了二维运动目标的跟踪实验,实验结果表明了该方法的可行性和有效性．     

Real-time method for tip following navigation of continuum snake arm robots

This paper presents a novel technique for the navigation of a snake arm robot, for real-time inspections in complex and constrained environments. These kinds of manipulators rely on redundancy, making the inverse kinematics very difficult. Therefore, a tip following method is proposed using the sequential quadratic programming optimization approach to navigate the robot. This optimization is used to minimize a set of changes to the arrangement of the snake arm that lets the algorithm follow the desired trajectory with minimal error. The information of the Snake Arm pose is used to limit deviations from the path taken. Therefore, the main objective is to find an efficient objective function that allows uninterrupted movements in real-time. The method proposed is validated through an extensive set of simulations of common arrangements and poses for the snake arm robot. For a 24 DoF robot, the average computation time is 0.4 s, achieving a speed of 4.5 mm/s, with deviation of no more than 25 mm from the ideal path.     

GA-pattern matching-based manipulator control system for real-time visual servoing

_—In robotic applications, tasks of picking and placing are the most fundamental ones. Also, for a robot manipulator, the recognition of its working environment is one of the most important issues to do intelligent tasks, since this aptitude enables it to work in a variable environment. This paper presents a new control strategy for robot manipulators, which utilizes visual information to direct the manipulator in its working space, to pick up an object of known shape, but with arbitrary position and orientation. During the search for an object to be picked up, vision-based control by closed-loop feedback, referred to as visual servoing, is performed to obtain the motion control of the manipulator hand. The system employs a genetic algorithm (GA) and a pattern matching technique to explore the search space and exploit the best solutions by this search technique. The control strategy utilizes the found results of GA-pattern matching in every step of GA evolution to direct the manipulator towards the target object. We named this control strategy step-GA-evnlution. This control method can be applied for manipulator real-time visual servoing and solve its path planning problem in real-time, i.e. in order for the manipulator to adapt the execution of the task by visual information during the process execution. Simulations have been performed, using a two-link planar manipulator and three image models, in order to find which one is the best for real-time visual servoing and the results show the effectiveness of the control method.     

基于图像雅可比矩阵的智能机器人视觉跟踪

根据智能机器人视觉伺服的要求,建立了固定眼结构的MOTOMAN-SV3XL型工业机器人无标定视觉伺服系统.研究了目前在机器人视觉伺服研究领域中使用最为广泛的一类方法,即基于图像雅可比矩阵的方法.针对图像雅可比矩阵的特点,设计了一种基于简化Sage-Husa自适应滤波的图像雅可比矩阵在线估计算法,并将其应用于机器人视觉反馈控制任务中,实现了对二维平面上运动目标的跟踪,实验结果表明了该算法的有效性.     

Robust auto tool change for industrial robots using visual servoing

This work presents an automated solution for tool changing in industrial robots using visual servoing and sliding mode control. The robustness of the proposed method is due to the control law of the visual servoing, which uses the information acquired by a vision system to close a feedback control loop. Furthermore, sliding mode control is simultaneously used in a prioritised level to satisfy the constraints typically present in a robot system: joint range limits, maximum joint speeds and allowed workspace. Thus, the global control accurately places the tool in the warehouse, but satisfying the robot constraints. The feasibility and effectiveness of the proposed approach is substantiated by simulation results for a complex 3D case study. Moreover, real experimentation with a 6R industrial manipulator is also presented to demonstrate the applicability of the method for tool changing.     

基于图像视觉伺服的二维运动优化控制

基于图像的视觉伺服方法,图像的变化直接解释为摄像机的运动,而不是直接对机械手末端实现笛卡尔速度控制,导致机械手的运动轨迹迂回,产生摄像机回退现象.针对这一问题,提出了将旋转和平移分离并先实现旋转的视觉伺服方案.该方案计算量小,系统响应时间短,解决了图像旋转和平移间的干扰,克服了传统基于图像视觉伺服产生的摄像机回退现象,实现了时间和路径的最优控制.并用传统IBVS的控制律和摄像机成像模型解释了回退现象的产生原因.二维运动仿真说明了提出方案的有效性.     

基于图像矩与神经网络的机器人四自由度视觉伺服

针对传统的视觉伺服方法中图像几何特征的标记、提取与匹配过程复杂且通用性差等问题,本文提出了一种基于图像矩的机器人四自由度(4DOF)视觉伺服方法.首先建立了眼在手系统中图像矩与机器人位姿之间的非线性增量变换关系,为利用图像矩进行机器人视觉伺服控制提供了理论基础,然后在未对摄像机与手眼关系进行标定的情况下,利用反向传播(BP)神经网络的非线性映射特性设计了基于图像矩的机器人视觉伺服控制方案,最后用训练好的神经刚络进行了视觉伺服跟踪控制.实验结果表明基于本文算法可实现0.5 mm的位置与0.5°的姿态跟踪精度,验证了算法的的有效性与较好的伺服性能.     

模型无关的无定标视觉伺服控制

本文首先介绍了视觉伺服的一般原理．然后提出了一种模型无关的无定标视觉伺服控制方法，在这种方法中不需要机器人模型和摄像机模型，应用方差最小化的原理推导出了模型无关的无定标视觉伺服控制律．此外还给出了图像雅可比矩阵的递推公式．文章最后通过一个轨线跟踪的仿真实验验证了算法的正确性和有效性．     

不需要标定系统模型的"眼在手上"视觉伺服控制技术

工业实践中不可能精确地标定摄像机和机器人模型，但当前的视觉伺服控制都需要标定系统模型．针对这一现象，提出一种新颖的、能应用于“眼在手上”视觉伺服控制结构的动态无标定的视觉伺服控制算法，无需标定摄像机和机器人运动学模型即可跟踪运动物体，通过将非线性目标函数最小化，以视觉信息跟踪动态图像．针对目前视觉伺服控制系统中“眼在手上”系统的复合雅克比矩阵随每个时间增量的变化无法计算的现象，提出了对每一时间增量时刻的图像雅克比矩阵的变化做出估计的方法，仿真实验证明了上述方法的正确性和有效性．