Robot guidance using computer vision

This paper describes a procedure for locating an autonomous robot vehicle in a three-dimensional space. The method is an extension of a two-dimensional technique described by Fukui. The method is simple, requires little computation and provides the unique three-dimensional position of the robot relative to a standard mark.     

Intelligent robotic assembly

It is normal when programming a robotic manipulator to provide the end effector's orientation and position at the pick up and drop off locations. Additional sensory information and intelligence is needed, however, to detect the presence of a part as well as its location if the assembly site cannot be controlled precisely by employing expensive jigs or fixtures. This paper investigates, for this purpose, the application of solely an inexpensive laser sensor mounted unobtrusively to the end effector of a CRS robot having customized hardware and open software. Data from the sensor is converted into a single “Feature Value Vector” to recognize a part and accurately determine its location by using a neural network and back propagation training. The procedure's viability is tested by assembling a set of tightly meshing gears under poor ambient lighting.     

An expert system for design for robotic assembly

In recent years, due to the various advantages associated with automation and robotics, much work has been done in developing robotic systems for assembly operations. Since part design plays a major role in assembly, this paper deals with the design of parts for ease of robotic assembly. Considerable knowledge is available in the form of design for robotic assembly rules. In addition, a large amount of data is required for decisions regarding suitability of parts for robotic assembly. The implementation of design for robotic assembly rules would be much easier with the help of an expert system, which would guide the designer toward choosing the design alternative that can best facilitate ease of assembly from a robotic point of view.To this end, a prototype expert system for design for robotic assembly is developed and presented in this paper. The expert system was implemented as a production system, which consists of rules and Object-Attribute-Value (O-A-V) triplets to represent domain knowledge. In order to best utilize the domain specific knowledge, a state space search-based inference mechanism was employed. The implementation of the prototype system is illustrated with examples.     

Sensory robotic assembly of apparel at Hull University

Research on sensory robotic assembly has been carried out at Hull University for over 12 years. Applied research is carried out in collaboration with industry and with additional support from govement research agencies. This is enhanced by longer-term, more speculative work providing the foundations for future applied research. This approach is exemplified in this paper by consideration of the automation of apparel manufacture. Some solutions to the limp material handling problem are outlined with emphasis on the ways in which sensors are sued to cope with uncertainties in the materials and handling processes.     

Adaptive robotic assembly of compliant aero-structure components

The application of robotics to the assembly of large aero-structures has been limited by the large size and inherent compliance of the components involved. The compliance of the components is significant and simple ‘pick and place’ approaches cannot be used due to the inherent dimensional variability between mating parts. The research described in this paper aims to solve this problem by using a non-contact sensing system to measure part deformation and misalignment in real time. The acquired data can then be processed through a mathematical algorithm to calculate the relative component positions required for optimal assembly. The data can also be used to check gross distortion of components and to reject those outside the specification limits. Existing part-to-part holes were used to provide alignment for individual components within the structure. A series of experiments were performed to investigate the feasibility of the method. Results are presented along with a discussion of the problems that may be encountered during robotic assembly. The experimental results show that robots when combined with non-contact metrology can be used for the assembly of compliant aero-structure components within required tolerance limits.     

A fast directional filter set for pyramidal image-based robotic systems

In seeking to develop a real-time, pyramidal imaging system for tracking and robotic visual servoing applications, inadequacies in the standard, commonly available kernels for directional filtering have led to the implementation of an alternative set of simple filters that combine to act as oriented edge-contrast operators fulfilling the familiar requirements for both speed and a highly localized response pattern. Although numerous edge operators, kernels and directional filters have been proposed, and low-level filtering is a well-established and mature area of computer vision, within the application described here, this operator set is proving superior to more established methods in efficiently preserving the response localization requirement while propagating the oriented gradient energies through multichannel resolution pyramids for further recursive processing. This paper provides a detailed discussion of the technique, the context of its development, and supporting empirical tests and statistical results. The evidence presented shows that the scheme provides a unified response to both step and line edge types, with acceptable performance in the presence of anticipated noise levels. The approach is pragmatic, of low complexity, and of general utility.     

The role of computer vision in prosthetic vision

The cost of vision loss worldwide has been estimated at nearly $3 trillion (http://www.amdalliance.org/cost-of-blindness.html). Non-preventable diseases cause a significant proportion of blindness in developed nations and will become more prevalent as people live longer. Prosthetic vision technologies including retinal implants will play an important therapeutic role. Retinal implants convert an input image stream to visual percepts via stimulation of the retina. This paper highlights some barriers to restoring functional human vision for current generation visual prosthetic devices that computer vision can help overcome. Such computer vision is interactive, aiming to restore function including visuo-motor tasks and recognition.     

Detection and tracking for robotic visual servoing systems

Robot manipulators require knowledge about their environment in order to perform their desired actions. In several robotic tasks, vision sensors play a critical role by providing the necessary quantity and quality of information regarding the robot's environment. For example, “visual servoing” algorithms may control a robot manipulator in order to track moving objects that are being imaged by a camera. Current visual servoing systems often lack the ability to detect automatically objects that appear within the camera's field of view. In this research, we present a robust “figureiground” framework for visually detecting objects of interest. An important contribution of this research is a collection of optimization schemes that allow the detection framework to operate within the real-time limits of visual servoing systems. The most significant of these schemes involves the use of “spontaneous” and “continuous” domains. The number and location of continuous domains are. allowed to change over time, adjusting to the dynamic conditions of the detection process. We have developed actual servoing systems in order to test the framework's feasibility and to demonstrate its usefulness for visually controlling a robot manipulator.     

A neural network compensator for uncertainties in robotic assembly

It is difficult to represent the nonlinear characteristics in the dynamics of robot manipulators by means of a mathematical model. An alternative approach of using a neural network to learn the parametric and unstructured uncertainties in robot manipulators is proposed. It is then embedded in the structure of a joint torque perturbation observer to compensate for uncertainties in the robot dynamic model. As the result, an accurate estimate of the joint reaction torque against the environment can be deduced. The approach is applied to monitor the insertion force during electronic components assembly using a SCARA robot. A true teaching signal of neural network for learning the model uncertainties is obtained. Furthermore, a special motion test is conducted to generate the required training data set. After learning, the neural network is capable of reproducing the training data. The generalizing ability of the network enables it to output the correct compensation signal for a trajectory which it has not been trained. With the proposed technique, it is possible to verify the success of component insertion in real time and avoid causing damages to the electronic components.     

MinGPU: a minimum GPU library for computer vision

In the field of computer vision, it is becoming increasingly popular to implement algorithms, in sections or in their entirety, on a graphics processing unit (GPU). This is due to the superior speed GPUs offer compared to CPUs. In this paper, we present a GPU library, MinGPU, which contains all of the necessary functions to convert an existing CPU code to GPU. We have created GPU implementations of several well known computer vision algorithms, including the homography transformation between two 3D views. We provide timing charts and show that our MinGPU implementation of homography transformations performs approximately 600 times faster than its C++ CPU implementation.

A formal framework for design and verification of robotic agents

Intelligent robots are autonomous and are used in environments where human interaction is hazardous or impossible. Verification of software for intelligent robots is mandatory because in situations where intelligent robots are employed online, error recovery is almost impossible. In this paper, we provide a formal framework for offline verification of software used in robotic applications. The specification enables one to design a robotic agent which represents a class of real-life robots. Forward and inverse kinematic operations of the robotic agent are specified using the specification for rigid solids and their primitive operations. An object-oriented design of the robotic agent derived from the specifications is given. We use the specification technique VDM for our purpose.This work was partially supported by FCAR, Quebec and NSERC, Canada.     

A model-based computer vision system for recognizing handwritten ZIP codes

This paper describes a recognition system for handwritten ZIP Codes currently under development at the Environmental Research Institute of Michigan (ERIM). Included within this system are techniques for preprocessing address block images, locating ZIP Codes, splitting touching characters, and identifying handwritten numerals. These techniques rely on mathematical morphology-based image processing and on hierarchical matching of object models to symbolic image representations. The image processing uses adaptive filtering, thresholding, and skeletonizing to create binary and state-labeled images. The matching process uses these images and extensively developed handwritten digit models to identify ZIP Codes. The end-to-end system has been tested on 500 randomly selected address block images. The system correctly recognized a large portion of the ZIP Codes in the test images (45.0%), and incorrectly classified a very low percentage of isolated handwritten digits (0.9%). Overall performance continues to be improved through incremental digit model refinement.This work was funded by the Office of Advanced Technology, United States Postal Service under contract 104230-86-H-0042.     

Image segmentation schema for low-level computer vision

This paper describes two distinct approaches to image-segmentation, both of which take the form of so-called region-growing algorithms. This kind of algorithms begin with a seed location and attempt to join neighboring pixels to this growing seed until no neighbors can be joined to it.

In the first approach, each pixel is linked to the nearest neighbor in the sense of the gray level. The second approach is based on a binary relation named relative similarity relation which reflects relative properties in an image. The combined segmentation scheme of these approaches is also presented.     

Toward an object-oriented and automated approach for achieving robotic assembly

This paper elaborates an object-oriented and automated approach for generating assembly sequences and achieving robotic assembly. This system, built with database and artificial intelligence (AI) techniques, provides the robot with the assembly sequence layout for the automatic handling rule. The critical set of problems is derived from various schemes such as model-based object recognition, features information, geometric and physical constraints between components, knowledge interpretation and robot task sequencing. Pattern recognition regarding shapes and features, along with a knowledge-based assembly, are the key issues of the authors' work. The methodology is shown through two illustrative examples involving different designs of parts in an assembly environment.     

计算机视觉技术在作物形态测量中的应用

为了测量作物形态信息,制定了玉米植株照片拍摄标准;利用对玉米图像细化后叶片和茎杆间存在的交点数目来检测玉米叶片的数量,采用噪声和毛刺去除算法提高了系统检测的精度;改进了freeman链码的长度测量方法,大大提高了曲线长度的测量精度.用Microsoft Visual C＋＋.NET程序开发工具实现了作物形态无损检测系统,通过实验表明,该系统可以对作物叶长、叶倾角等形态特征进行自动和手动的测量,精度符合设计要求.     

塔式起重机机器视觉监控系统设计

为解决各类施工现场以塔机运行过程中的安全问题,设计了一套基于视觉传感技术的智能视频监控系统,根据实际需要,在嵌入式平台上配合图像传感器和视觉跟踪TLD算法,实现了对塔机吊臂运动轨迹的追踪定位、判断和预警.测试结果表明:系统的检测率和正确率均能达到95％以上.     

Machine vision based path planning for a robotic golf club head welding system

A path planning method based on machine vision techniques is constructed for a golf-club head robotic welding system. This system uses 3D machine vision techniques to recognize the weldseam and generates a welding path for the robot. The location of the weldseam is discovered by applying a Sobel mask to the captured data. A Laplace mask is also useful to filter out the noise points due to the scatter light refraction of tack-welding spots. The weldseam is then replenished and smoothed out by a B-spline curve fitting. The task frame of the weldseam is computed by finding the tangent, normal, and bi-normal of the curve. The robotic welding path is obtained by further rotations and translation along the axes of the task frame according to the requirement of the welding attitude. The developed machine vision technique and the mathematic framework pertaining to the generation of a welding task frame can readily be used for various three-dimensional welding tasks.     

Improvement of vision guided robotic accuracy using Kalman filter

This study addresses one of the most prevalent, yet difficult problems in vision calibration, namely, the accuracy of remote, vision-guided robotic control in tune with the notion of e-quality for manufacture (EQM). In all areas of robot work space, image distortion occurs due to imperfect lens curvatures, which in turn induces an inaccurate vision guidance. Non-uniform nature of image distortion is effectively rectified, using the Kalman Filtering technique. Consequently, the robotic positioning accuracy is significantly improved. In recent years, stringent quality standards and intense competition compelled many companies to adopt new, advanced technologies to further enhance their strategic competitiveness. EQM is an emergent technology better suited for today’s fast-changing, zero-defect production environment. The proposed methodology has great potential for improving the product quality and operational efficiency of networked robotic production system, which has been vindicated by the statistical analysis.     

A model for output evaluation of an intelligent robotic assembly station

The evolution of intelligent robots has brought widespread changes in the concept of automatic assembly. An intelligent robot can successfully accomplish assembly in a multiple end product processing and unstructured component part(s) handling environment. This advanced capability of the robot dictates a major change in their basic output characteristics. For example, task service time of an intelligent assembly robot is no longer completely deterministic like that of an ordinary pick-and-place type robot. In this paper, a typical component insertion type assembly task is analyzed to show that the task service time is a random variable consisting of two (2) parts, a deterministic component and a stochastic component. Thus, with the assumption of exponential part input an assembly station served by a single intelligent robot is modeled as an M/G/1 queueing model. A robot availability factor is also introduced into the service time random variable. Closed form expressions for important system performance parameters are obtained under the assumption of three (3) different probability distributions for the stochastic part of the service time random variable.     

一种用于三维物体定位的视觉方法

本文讨论了一种利用视觉信息对限定性物体在三维空间定位的方法．这项工作是一个在脱机的高级机器人语言控制下验证视觉系统的一部分．本文对这一视觉系统的整体结构及其与高级机器人语言系统的关系做了概要的介绍,并且着重讨论了其中的低级视觉信息处理模块的工作原理,以及利用视觉信息对三维空间中的物体进行定位的精度问题．