Visual servoing of redundant manipulator with Jacobian matrix estimation using self-organizing map

Vision based redundant manipulator control with a neural network based learning strategy is discussed in this paper. The manipulator is visually controlled with stereo vision in an eye-to-hand configuration. A novel Kohonen’s self-organizing map (KSOM) based visual servoing scheme has been proposed for a redundant manipulator with 7 degrees of freedom (DOF). The inverse kinematic relationship of the manipulator is learned using a Kohonen’s self-organizing map. This learned map is shown to be an approximate estimate of the inverse Jacobian, which can then be used in conjunction with the proportional controller to achieve closed loop servoing in real-time. It is shown through Lyapunov stability analysis that the proposed learning based servoing scheme ensures global stability. A generalized weight update law is proposed for KSOM based inverse kinematic control, to resolve the redundancy during the learning phase. Unlike the existing visual servoing schemes, the proposed KSOM based scheme eliminates the computation of the pseudo-inverse of the Jacobian matrix in real-time. This makes the proposed algorithm computationally more efficient. The proposed scheme has been implemented on a 7 DOF PowerCube? robot manipulator with visual feedback from two cameras.     

利用平面单应分解实现服务机器人视觉伺服

智能空间中家庭服务机器人所需完成的主要任务是协助人完成物品的搜寻、定位与传递。而视觉伺服则是完成上述任务的有效手段。搭建了由移动机器人、机械臂、摄像头组成的家庭服务机器人视觉伺服系统,建立了此系统的运动学模型并对安装在机械臂末端执行器上的视觉系统进行了内外参数标定,通过分解世界平面的单应来获取目标物品的位姿参数,利用所获取的位姿参数设计了基于位置的视觉伺服控制律。实验结果表明,使用平面单应分解方法来设计控制律可简单有效地完成家庭物品的视觉伺服任务。     

视觉伺服机器人对运动目标操作的研究

机器人视觉伺服系统是机器人领域一重要的研究方向，它的研究对于开发手眼协调的机器人在工业生产、航空航天等方面的应用有着极其重要的意义．本文着眼于视觉伺服机器人操作运动目标这一问题，分析了建立此类系统的控制结构并指明其特点；同时，详细地阐述了三个组成环节：视觉图像处理、预测及滤波、视觉控制器的研究方法和现状．最后，分析了今后的研究趋势．     

Vision resolvability for visually servoed manipulation

This article introduces a sensor placement measure called vision resolvability. The measure provides a technique for estimating the relative ability of various visual sensors, including monocular systems, stereo pairs, multi-baseline stereo systems, and 3D rangefinders, to accurately control visually manipulated objects. The resolvability ellipsoid illustrates the directional nature of resolvability, and can be used to direct camera motion and adjust camera intrinsic parameters in real-time so that the servoing accuracy of the visual servoing system improves with camera-lens motion. The Jacobian mapping from task space to sensor space is derived for a monocular system, a stereo pair with parallel optical axes, and a stereo pair with perpendicular optical axes. Resolvability ellipsoids based on these mappings for various sensor configurations are presented. Visual servoing experiments demonstrate that vision resolvability can be used to direct camera-lens motion to increase the ability of a visually servoed manipulator to precisely servo objects. © 1996 John Wiley & Sons, Inc.     

基于遗传算法模式匹配的机器人实时视觉伺服

对于机器人手臂来讲 ,对工作环境的识别是完成一个智能任务的最重要的问题之一 .因为这种智能可以使它工作在一个变化的环境中 .本文提出了一种新的机器人手臂的控制策略 ,可以利用视觉信息来指导机器人的手臂在它的工作空间中捡起一个已知形状但任意位置和方向的物体 .在对物体的搜索过程中 ,利用基于视觉闭环的视觉伺服来完成对机器人手臂的运动控制 .本系统利用遗传算法 (Genetic Algorithm ,GA)和模式匹配技术完成对搜索空间的搜索并获得了良好的结果 .本文完成了对带有两连杆手臂的视觉伺服系统的仿真 ,仿真结果证明了算法的有效性     

GA-pattern matching-based manipulator control system for real-time visual servoing

_—In robotic applications, tasks of picking and placing are the most fundamental ones. Also, for a robot manipulator, the recognition of its working environment is one of the most important issues to do intelligent tasks, since this aptitude enables it to work in a variable environment. This paper presents a new control strategy for robot manipulators, which utilizes visual information to direct the manipulator in its working space, to pick up an object of known shape, but with arbitrary position and orientation. During the search for an object to be picked up, vision-based control by closed-loop feedback, referred to as visual servoing, is performed to obtain the motion control of the manipulator hand. The system employs a genetic algorithm (GA) and a pattern matching technique to explore the search space and exploit the best solutions by this search technique. The control strategy utilizes the found results of GA-pattern matching in every step of GA evolution to direct the manipulator towards the target object. We named this control strategy step-GA-evnlution. This control method can be applied for manipulator real-time visual servoing and solve its path planning problem in real-time, i.e. in order for the manipulator to adapt the execution of the task by visual information during the process execution. Simulations have been performed, using a two-link planar manipulator and three image models, in order to find which one is the best for real-time visual servoing and the results show the effectiveness of the control method.     

Precise Positioning of Binocular Eye-to-Hand Robotic Manipulators

This article addresses the visual servoing of a rigid robotic manipulator equipped with a binocular vision system in eye-to-hand configuration. The control goal is to move the robot end-effector to a visually determined target position precisely without knowing the precise camera model. Many vision-based robotic positioning systems have been successfully implemented and validated by supporting experimental results. Nevertheless, this research aims at providing stability analysis for a class of robotic set-point control systems employing image-based feedback laws. Specifically, by exploring epipolar geometry of the binocular vision system, a binocular visual constraint is found to assist in establishing stability property of the feedback system. Any three-degree-of-freedom positioning task, if satisfying appropriate conditions with the image-based encoding approach, can be encoded in such a way that the encoded error, when driven to zero, implies that the original task has been accomplished with precision. The corresponding image-based control law is proposed to drive the encoded error to zero. The overall closed-loop system is exponentially stable provided that the binocular model imprecision is small.     

机器人视觉伺服系统的研究

机器人伺觉伺服系统及到多学科内容。针对机器人视觉伺服系统主要的三方面内容;系统的结构方式,图象处理,控制方法,介绍了该领域的研究现状及所取得的成就。最后分析了今后的发展趋势。     

Predictive control architecture for real-time image moments based servoing of robot manipulators

This paper presents a real-time architecture for visual servoing of robot manipulators using nonlinear based predictive control. In order to increase the robustness of the control algorithm, image moments were chosen to be the visual features which describe the objects from the image. A visual predictive architecture is designed to solve tasks addressed to robot manipulators with an eye-in-hand configuration. An implementation of the proposed architecture is done so that the capabilities of a 6 d.o.f robot manipulator are extended. The results of different experiments conducted with two types of image moments based controllers (proportional and predictive with reference trajectory) are presented and discussed.     

Particle Filtering for Industrial 6DOF Visual Servoing

Visual servoing allows the introduction of robotic manipulation in dynamic and uncontrolled environments. This paper presents a position-based visual servoing algorithm using particle filtering. The objective is the grasping of objects using the 6 degrees of freedom of the robot manipulator in non-automated industrial environments using monocular vision. A particle filter has been added to the position-based visual servoing algorithm to deal with the different noise sources of those industrial environments. Experiments performed in the real industrial scenario of ROBOFOOT (http://www.robofoot.eu/) project showed accurate grasping and high level of stability in the visual servoing process.     

Remote Visual Servoing of a Robot Manipulator via Internet2

Visual servoing is a powerful approach to enlarge the applications of robotic systems by incorporating visual information into the control system. On the other hand, teleoperation – the use of machines in a remote way – is increasing the number of applications in many domains. This paper presents a remote visual servoing system using only partial camera calibration and exploiting the high bandwidth of Internet2 to stream video information. The underlying control scheme is based on the image-based philosophy for direct visual servoing – computing the applied torque inputs to the robot based in error signals defined in the image plane – and evoking a velocity field strategy for guidance. The novelty of this paper is a remote visual servoing with the following features: (1) full camera calibration is unnecessary, (2) direct visual servoing does not neglect the robot nonlinear dynamics, and (3) the novel velocity field control approach is utilized. Experiments carried out between two laboratories demonstrated the effectiveness of the application. Work partially supported by CONACyT grant 45826 and CUDI.     

基于图像矩与神经网络的机器人四自由度视觉伺服

针对传统的视觉伺服方法中图像几何特征的标记、提取与匹配过程复杂且通用性差等问题,本文提出了一种基于图像矩的机器人四自由度(4DOF)视觉伺服方法.首先建立了眼在手系统中图像矩与机器人位姿之间的非线性增量变换关系,为利用图像矩进行机器人视觉伺服控制提供了理论基础,然后在未对摄像机与手眼关系进行标定的情况下,利用反向传播(BP)神经网络的非线性映射特性设计了基于图像矩的机器人视觉伺服控制方案,最后用训练好的神经刚络进行了视觉伺服跟踪控制.实验结果表明基于本文算法可实现0.5 mm的位置与0.5°的姿态跟踪精度,验证了算法的的有效性与较好的伺服性能.     

Navigation system including associative memory

A new adaptive linear robot control system for a robot work cell that can visually track and intercept stationary and moving objects undergoing arbitrary motion anywhere along its predicted trajectory within the robot's workspace is presented in this paper. The proposed system was designed by integrating a stationary monocular CCD camera with off-the-shelf frame grabber and an industrial robot operation into a single application on the MATLAB platform. A combination of the model based object recognition technique and a learning vector quantization network is used for classifying stationary objects without overlapping. The optical flow technique and the MADALINE network are used for determining the target trajectory and generating the predicted robot trajectory based on visual servoing, respectively. The necessity of determining a model of the robot, camera, all the stationary and moving objects, and environment is eliminated. The location and image features of these objects need not be preprogrammed, marked and known before, and any change in a task is possible without changing the robot program. After the learning process on the robot, it is shown that the KUKA robot is capable of tracking and intercepting both stationary and moving objects at an optimal rendezvous point on the conveyor accurately in real-time.     

An Experiment with Reactive Data-flow Tasking in Active Robot Vision

This paper presents an experiment with the synchronous approach to reactive systems programming, and particularly the Signal language, applied to a significant problem in robot vision: active visual reconstruction. This application consists of the specification of a system dealing with various domains such as robot control, computer vision and transitions between different modes of control. It illustrates the adequacy in such domains of Signal, a data flow programming language and environment. The programming environment features tools for formal specification, analysis, consistency checking and code generation. Signal and its language-level extension for task preemption SignalGTi are used at the different levels of the application: data-flow function for the camera motion control (visual servoing), reconstruction method (in parallel to visual servoing, involving the dynamical processes), and reconstruction of complex scenes (with transitions between several robotics tasks). The combination of these levels constitutes a hybrid behavior with (sampled) continuous control and discrete transitions. These techniques are validated experimentally by an implementation on a robotic cell. © 1997 by John Wiley & Sons, Ltd.     

Cooperative passers-by tracking with a mobile robot and external cameras

In this paper, we present a cooperative passers-by tracking system between fixed view wall mounted cameras and a mobile robot. The proposed system fuses visual detections from wall mounted cameras and detections from a mobile robot–in a centralized manner–employing a “tracking-by-detection” approach within a Particle Filtering strategy. This tracking information is then used to endow the robot with passers-by avoidance ability to facilitate its navigation in crowds during the execution of a person following mission. The multi-person tracker’s ability to track passers-by near the robot distinctively is demonstrated through qualitative and quantitative off-line experiments. Finally, the designed perceptual modalities are deployed on our robotic platform, controlling its actuators via visual servoing techniques and free space diagrams in the vicinity of the robot, to illustrate the robot’s ability to follow a given target person in human crowded areas.     

Designing and testing a calibrating procedure for combining the coordination systems of a handling robot and a stationed video camera

The paper discusses some of the results of designing and testing a calibrating procedure with important implications in contemporary robotics. This procedure is also mentioned in the literature as the “procedure for calibrating by matching the coordination systems of a robot and a stationed video camera”. The procedure is tested by a training robotechnic system, which consists of an anthropomorphic robot with five stages of freedom and a stationed video camera that can identify solid stationary objects. The results of this experimental testing are used to create and adopt some new methods for laboratory exams in robotics for students of FA in a Technical University and for developing further research on calibrating procedures in robotics needed for the industry.     

Robust auto tool change for industrial robots using visual servoing

This work presents an automated solution for tool changing in industrial robots using visual servoing and sliding mode control. The robustness of the proposed method is due to the control law of the visual servoing, which uses the information acquired by a vision system to close a feedback control loop. Furthermore, sliding mode control is simultaneously used in a prioritised level to satisfy the constraints typically present in a robot system: joint range limits, maximum joint speeds and allowed workspace. Thus, the global control accurately places the tool in the warehouse, but satisfying the robot constraints. The feasibility and effectiveness of the proposed approach is substantiated by simulation results for a complex 3D case study. Moreover, real experimentation with a 6R industrial manipulator is also presented to demonstrate the applicability of the method for tool changing.     

Uncalibrated Eye-to-Hand Visual Servoing Using Inverse Fuzzy Models

A new uncalibrated eye-to-hand visual servoing based on inverse fuzzy modeling is proposed in this paper. In classical visual servoing, the Jacobian plays a decisive role in the convergence of the controller, as its analytical model depends on the selected image features. This Jacobian must also be inverted online. Fuzzy modeling is applied to obtain an inverse model of the mapping between image feature variations and joint velocities. This approach is independent from the robot's kinematic model or camera calibration and also avoids the necessity of inverting the Jacobian online. An inverse model is identified for the robot workspace, using measurement data of a robotic manipulator. This inverse model is directly used as a controller. The inverse fuzzy control scheme is applied to a robotic manipulator performing visual servoing for random positioning in the robot workspace. The obtained experimental results show the effectiveness of the proposed control scheme. The fuzzy controller can position the robotic manipulator at any point in the workspace with better accuracy than the classic visual servoing approach.     

Intelligent control of a robot manipulator by visual feedback

We present a new approach to visual feedback control using image-based visual servoing with stereo vision. In order to control the position and orientation of a robot with respect to an object, a new technique is proposed using binocular stereo vision. The stereo vision enables us to calculate an exact image Jacobian not only at around a desired location, but also at other locations. The suggested technique can guide a robot manipulator to the desired location without needing such a priori knowledge as the relative distance to the desired location or a model of the object, even if the initial positioning error is large. We describes a model of stereo vision and how to generate feedback commands. The performance of the proposed visual servoing system is illustrated by a simulation and by experimental results, and compared with the conventional method for an assembling robot. This work was presented in part at the Fourth International Symposium on Artificial Life and Robotics, Oita, Japan, January 19–22, 1999     

机器人视觉伺服综述

系统论述了机器人视觉伺服发展的历史和现状。从不同角度对机器人视觉控制系统进行分类,重点介绍了基位置的视觉伺服系统和基于图像的视觉伺服系统。对人工神经网络在机器人视觉伺服方面的应用情况作了介绍。讨论了视觉伺服中图像特征的选择问题。对机器人视觉所涉及的前沿问题进行阐述,并指出了目前研究中所存在的问题及今后发展方向。