Frequency response experiments of eye-vergence visual servoing in lateral motion with 3D evolutionary pose tracking

Visual servoing towards moving target with hand-eye cameras fixed at hand is inevitably affected by hand dynamical oscillations. Therefore, it is difficult to make target position keep always at the center of camera’s view, as nonlinear dynamical effects of whole manipulator stand against tracking ability. To overcome this defect of the hand-eye fixed camera system, an eye-vergence system has been put forward, where the cameras could rotate to observe the target object. The visual servoing controllers of hand and eye-vergence are installed independently, so that it can observe the target object at the center of camera images through eye-vergence function. The dynamical superiorities of eye-vergence system are verified through frequency response experiments, comparing with hand tracking performances and the proposed eye-vergence tracking performances. This paper analyzes the performance of 3D-object position and orientation tracking, in which orientation representation method is based on quaternion, and the orientation tracking results are shown with more comprehensive analysis of system performance.     

Eye-vergence visual servoing enhancing Lyapunov-stable trackability

Visual servoing methods for hand–eye configuration are vulnerable for hand’s dynamical oscillation, since nonlinear dynamical effects of whole manipulator stand against the stable tracking ability (trackability). Our proposal to solve this problem is that the controller for visual servoing of the hand and the one for eye-vergence should be separated independently based on decoupling each other, where the trackability is verified by Lyapunov analysis. Then the effectiveness of the decoupled hand and eye-vergence visual servoing method is evaluated through simulations incorporated with actual dynamics of 7-DoF robot with additional 3-DoF for eye-vergence mechanism by amplitude and phase frequency analysis.     

Position-Based Visual Servoing in Industrial Multirobot Cells Using a Hybrid Camera Configuration

This paper deals with the problem of position-based visual servoing in a multiarm robotic cell equipped with a hybrid eye-in-hand/eye-to-hand multicamera system. The proposed approach is based on the real-time estimation of the pose of a target object by using the extended Kalman filter. The data provided by all the cameras are selected by a suitable algorithm on the basis of the prediction of the object self-occlusions, as well as of the mutual occlusions caused by the robot links and tools. Only an optimal subset of image features is considered for feature extraction, thus ensuring high estimation accuracy with a computational cost independent of the number of cameras. A salient feature of the paper is the implementation of the proposed approach to the case of a robotic cell composed of two industrial robot manipulators. Two different case studies are presented to test the effectiveness of the hybrid camera configuration and the robustness of the visual servoing algorithm with respect to the occurrence of occlusions     

利用平面单应分解实现服务机器人视觉伺服

智能空间中家庭服务机器人所需完成的主要任务是协助人完成物品的搜寻、定位与传递。而视觉伺服则是完成上述任务的有效手段。搭建了由移动机器人、机械臂、摄像头组成的家庭服务机器人视觉伺服系统,建立了此系统的运动学模型并对安装在机械臂末端执行器上的视觉系统进行了内外参数标定,通过分解世界平面的单应来获取目标物品的位姿参数,利用所获取的位姿参数设计了基于位置的视觉伺服控制律。实验结果表明,使用平面单应分解方法来设计控制律可简单有效地完成家庭物品的视觉伺服任务。     

基于遗传算法的运动物体实时识别

文章提出了一种应用遗传算法对一个已知几何模型的运动目标物体的位置与方向进行识别的方法,这种对运动目标的识别可以用于机器人手臂实时视觉伺服控制中。文章以一个三角形的物体为目标,对所提出的方法进行了仿真试验,结果证实了这种识别方法的有效性。     

Stereo Tracking and Servoing for Space Applications

This paper proposes a real-time, robust and efficient three-dimensional (3-D) model-based tracking algorithm for visual servoing. A virtual visual servoing approach is used for 3-D tracking. This method is similar to more classical nonlinear pose computation techniques. Robustness is obtained by integrating an M-estimator into the virtual visual control law via an iteratively reweighted least-squares implementation. The presented approach is also extended to the use of multiple cameras. Results show the method to be robust to occlusion, changes in illumination and mistracking.     

单目视觉智能车的伺服规律研究

通过分析单目视觉智能车的运动特征,建立其空间约束模型以及与运动控制密切相关的图像坐标系和运动平面坐标系,在此基础上根据摄像头相对位姿的不同推导出2种不同的坐标映射关系,讨论将其应用在单目标跟踪和连续目标跟踪时的车体运动控制规律,形成一个有较强针对性和实用价值的手眼机器人视觉伺服框架。实验验证了单目视觉智能车伺服规律的有效性。     

基于Adept机器人的视觉伺服控制系统

机器人视觉伺服控制在理论和应用等方面还有许多问题需要研究,例如特征选择、系统标定和伺服控制算法等.针对Adept机器人,提出了一种简单快速的不需要精确标定摄像机内外部参数的摄像机标定方法,完成了从被观测物体表面所在的视觉平面坐标系到机器人基坐标系的坐标变换.使用图像的全局特征,即图像矩特征进行伺服跟踪;利用所推导的图像雅可比矩阵,设计了由图像反馈与目标运动自适应补偿组成的视觉伺服控制器.将算法对静态目标的定位实验进行了验证,然后又将其应用到移动目标的跟踪上,通过调节和优选控制参数,实现了稳定的伺服跟踪和抓取.实验结果表明采用图像矩作为图像特征能够避免复杂的特征匹配过程,并且能够获得较好的跟踪精度.     

基于图像矩与神经网络的机器人四自由度视觉伺服

针对传统的视觉伺服方法中图像几何特征的标记、提取与匹配过程复杂且通用性差等问题,本文提出了一种基于图像矩的机器人四自由度(4DOF)视觉伺服方法.首先建立了眼在手系统中图像矩与机器人位姿之间的非线性增量变换关系,为利用图像矩进行机器人视觉伺服控制提供了理论基础,然后在未对摄像机与手眼关系进行标定的情况下,利用反向传播(BP)神经网络的非线性映射特性设计了基于图像矩的机器人视觉伺服控制方案,最后用训练好的神经刚络进行了视觉伺服跟踪控制.实验结果表明基于本文算法可实现0.5 mm的位置与0.5°的姿态跟踪精度,验证了算法的的有效性与较好的伺服性能.     

Photometric visual servoing for omnidirectional cameras

2D visual servoing consists in using data provided by a vision sensor for controlling the motions of a dynamic system. Most of visual servoing approaches has relied on the geometric features that have to be tracked and matched in the image acquired by the camera. Recent works have highlighted the interest of taking into account the photometric information of the entire image. This approach was tackled with images of perspective cameras. We propose, in this paper, to extend this technique to central cameras. This generalization allows to apply this kind of method to catadioptric cameras and wide field of view cameras. Several experiments have been successfully done with a fisheye camera in order to control a 6 degrees of freedom robot and with a catadioptric camera for a mobile robot navigation task.     

一种无标定视觉伺服控制技术的研究

在视觉伺服控制过程中无法精确地标定摄像机和机器人运动学模型,而当前的无标定视觉伺服控制技术或者只能针对静态的目标,或者针对动态目标但无法摆脱大偏差的影响．针对此问题,提出一种动态无标定的视觉伺服控制方法：基于非线性方差最小化法控制机器人跟踪运动目标,利用动态拟牛顿法估计图像雅克比矩阵,采用迭代最小二乘法提高系统的稳定性并提出大偏差条件下的无标定控制策略．仿真实验证明了该方法的正确性和有效性．     

不需要标定系统模型的"眼在手上"视觉伺服控制技术

工业实践中不可能精确地标定摄像机和机器人模型，但当前的视觉伺服控制都需要标定系统模型．针对这一现象，提出一种新颖的、能应用于“眼在手上”视觉伺服控制结构的动态无标定的视觉伺服控制算法，无需标定摄像机和机器人运动学模型即可跟踪运动物体，通过将非线性目标函数最小化，以视觉信息跟踪动态图像．针对目前视觉伺服控制系统中“眼在手上”系统的复合雅克比矩阵随每个时间增量的变化无法计算的现象，提出了对每一时间增量时刻的图像雅克比矩阵的变化做出估计的方法，仿真实验证明了上述方法的正确性和有效性．     

An efficient continuous tracking system in real-time surveillance application

A tracking object must present a proper field of view (FOV) in a multiple active camera surveillance system; its clarity can facilitate smooth processing by the surveillance system before further processing, such as face recognition. However, when pan–tilt–zoom (PTZ) cameras are used, the tracking object can be brought into the FOV by adjusting its intrinsic parameters; consequently, selection of the best-performing camera is critical. Performance is determined by the relative positions of the camera and the tracking objects, image quality, lighting and how much of the front side of the object faces the camera. In a multi-camera surveillance system, both camera hand-off and camera assignment play an important role in automated and persistent tracking, which are typical surveillance requirements. This study investigates the use of automatic methods for tracking an object across cameras in a surveillance network using PTZ cameras. An automatic, efficient continuous tracking scheme is developed. The goal is to determine the decision criteria for hand-off using Sight Quality Indication (SQI) (which includes information on the position of the object and the proportion of the front of object faces the camera), and to perform the camera handoff task in a manner that optimizes the vision effect associated with monitoring. Experimental results reveal that the proposed algorithm can be efficiently executed, and the handoff method for feasible and continuously tracking active objects under real-time surveillance.     

Pose Estimation using Point and Line Correspondences

The problem of a real-time pose estimation between a 3D scene and a single camera is a fundamental task in most 3D computer vision and robotics applications such as object tracking, visual servoing, and virtual reality. In this paper we present two fast methods for estimating the 3D pose using 2D to 3D point and line correspondences. The first method is based on the iterative use of a weak perspective camera model and forms a generalization of DeMenthon's method (1995) which consists of determining the pose from point correspondences. In this method the pose is iteratively improved with a weak perspective camera model and at convergence the computed pose corresponds to the perspective camera model. The second method is based on the iterative use of a paraperspective camera model which is a first order approximation of perspective. We describe in detail these two methods for both non-planar and planar objects. Experiments involving synthetic data as well as real range data indicate the feasibility and robustness of these two methods. We analyse the convergence of these methods and we conclude that the iterative paraperspective method has better convergence properties than the iterative weak perspective method. We also introduce a non-linear optimization method for solving the pose problem.     

Visual Object Tracking and Servoing Control of a Nano-Scale Quadrotor: System,Algorithms, and Experiments

There are two main trends in the development of unmanned aerial vehicle(UAV)technologies:miniaturization and intellectualization,in which realizing object tracking capabilities for a nano-scale UAV is one of the most challenging problems.In this paper,we present a visual object tracking and servoing control system utilizing a tailor-made 38 g nano-scale quadrotor.A lightweight visual module is integrated to enable object tracking capabilities,and a micro positioning deck is mounted to provide accurate pose estimation.In order to be robust against object appearance variations,a novel object tracking algorithm,denoted by RMCTer,is proposed,which integrates a powerful short-term tracking module and an efficient long-term processing module.In particular,the long-term processing module can provide additional object information and modify the short-term tracking model in a timely manner.Furthermore,a positionbased visual servoing control method is proposed for the quadrotor,where an adaptive tracking controller is designed by leveraging backstepping and adaptive techniques.Stable and accurate object tracking is achieved even under disturbances.Experimental results are presented to demonstrate the high accuracy and stability of the whole tracking system.     

Finite-time tracking control for multiple non-holonomic mobile robots based on visual servoing

This paper investigates finite-time tracking control problem of multiple non-holonomic mobile robots via visual servoing. It is assumed that the pinhole camera is fixed to the ceiling, and camera parameters are unknown. The desired reference trajectory is represented by a virtual leader whose states are available to only a subset of the followers, and the followers have only interaction. First, the camera-objective visual kinematic model is introduced by utilising the pinhole camera model for each mobile robot. Second, a unified tracking error system between camera-objective visual servoing model and desired reference trajectory is introduced. Third, based on the neighbour rule and by using finite-time control method, continuous distributed cooperative finite-time tracking control laws are designed for each mobile robot with unknown camera parameters, where the communication topology among the multiple mobile robots is assumed to be a directed graph. Rigorous proof shows that the group of mobile robots converges to the desired reference trajectory in finite time. Simulation example illustrates the effectiveness of our method.     

Robust Visual‐Servo Control of Robot Manipulators in the Presence of Uncertainty

This paper considers the problem of position control of planar robot manipulators via visual servoing in the presence of uncertainty associated with the robot mechanical dynamics and the camera system for both fixed‐camera and camera‐in‐hand configurations. Specifically, we first design a robust controller that compensates for uncertainty throughout the whole robot‐camera system and ensures global uniformly ultimately bounded position tracking for the fixed‐camera configuration. Under the same class of uncertainty, we then develop a setpoint controller for the camera‐in‐hand configuration that achieves global uniformly ultimately bounded regulation. Experimental results illustrating the performance of both controllers are also included. © 2003 Wiley Periodicals, Inc.     

Efficient model-based tracking for robot vision

This paper proposes a real-time, robust and efficient three-dimensional (3D) model-based tracking algorithm. A virtual visual servoing approach is used for monocular 3D tracking. This method is similar to more classical non-linear pose computation techniques. A concise method for derivation of efficient distance-to-contour interaction matrices is described. An oriented edge detector is used in order to provide real-time tracking of points normal to the object contours. Robustness is obtained by integrating a M-estimator into the virtual visual control law via an iteratively re-weighted least-squares implementation. The method presented in this paper has been validated on several visual servoing experiments considering various objects. Results show the method to be robust to occlusion, changes in illumination and mis-tracking.     

Autonomous selection and modification of camera configurations using visibility and manipulability measures

In this article, we present a camera control method in which the selection of an optimal camera position and the modification of camera configurations are accomplished according to changes in the surroundings. For the autonomous selection and modification of camera configurations during tasks, we consider the camera's visibility and the manipulator's manipulability. The visibility constraint guarantees that the whole of a target object can be “viewed” with no occlusions by the surroundings, and the manipulability constraint guarantees avoidance of the singular position of the manipulator and rapid modification of the camera position. By considering visibility and manipulability constraints simultaneously, we determine the optimal camera position and modify the camera configuration such that visual information for the target object can be obtained continuously during tasks. The results of simulations and experiments show that the active camera system with an eye‐in‐hand configuration can modify its configuration autonomously according to the motion of the surroundings by applying the proposed camera control method. © 2002 Wiley Periodicals, Inc.     

基于图像雅可比矩阵的智能机器人视觉跟踪

根据智能机器人视觉伺服的要求,建立了固定眼结构的MOTOMAN-SV3XL型工业机器人无标定视觉伺服系统.研究了目前在机器人视觉伺服研究领域中使用最为广泛的一类方法,即基于图像雅可比矩阵的方法.针对图像雅可比矩阵的特点,设计了一种基于简化Sage-Husa自适应滤波的图像雅可比矩阵在线估计算法,并将其应用于机器人视觉反馈控制任务中,实现了对二维平面上运动目标的跟踪,实验结果表明了该算法的有效性.