Feedback strategies for visual search in airframe structural inspection

Feedback of information has consistently shown positive results in human inspection, provided it is given in a timely and appropriate manner. Feedback serves as the basis of most training schemes; traditionally this has been performance feedback. Other forms of feedback which provide strategy information rather than performance information may have a role in improving inspection. This study compared performance feedback and cognitive feedback in a realistic simulation of an aircraft structural inspection task. Performance (time, errors) feedback showed the greatest improvements in performance measures. Cognitive feedback enhanced efficiency measures of search strategy. When cognitive feedback consisted of visual representations of the path and the coverage of the search sequence, subjects also were able to use this task information to improve their search performance.     

Active visual localization for multiple inspection robots

In the routine inspection of industrial or other areas, teams of robots with various sensors could operate together to great effect, but require reliable, accurate and flexible localization capabilities to be able to move around safely. We demonstrate accurate localization for an inspection team consisting of a robot with stereo active vision and its blind companion with an active lighting system, and show that in this case a single sensor can be used for measuring the position of known or unknown scene features, measuring the relative location of the two robots and actually carrying out an inspection task.     

Computer-aided visual inspection for integrated quality control

This paper first briefly introduces the fundamentals of automated inspection techniques, followed by a categorization of typical applications. At its focus the paper elaborates on issues relating to the need for, the approach to and the benefits of integrating automatic inspection systems within the context of computer integrated PCB manufacturing. It argues that automatic inspection systems should assume the role of not only quality inspection systems but also important quality information generators on the shop floor. In particular, integrated quality control will be realized through an integration of inspection systems with process/quality control systems such as statistical process control (SPC).     

Optimized sensor placement for active visual inspection

This article presents an optimized sensor planning system for active visual inspection of three‐dimensional manufacturing computer‐aided design (CAD) models. Quantization errors and displacement errors are inevitable in active visual inspection. To obtain high accuracy for dimensioning the entities of three‐dimensional CAD models, minimization of these errors is essential. Spatial quantization errors result in digitization. The errors are serious when the size of the pixel is significant compared to the allowable tolerance in the object dimension on the image. In placing the active sensor to perform inspection, displacement of the sensors in orientation and location is common. The difference between observed dimensions obtained by the displaced sensor and the actual dimensions is defined as displacement errors. The density functions of quantization errors and displacement errors depend on camera resolution and camera locations and orientations. The sensor constraints, such as resolution, focus, field‐of‐view, and visibility constraints, restrict sensor placement. To obtain a satisfactory view of the targeted entities of the CAD models, these constraints have to be satisfied. In this article, we focus on the edge line segments as the inspected entities. We use a genetic algorithm to minimize the probabilistic magnitude of the errors subject to the sensor constraints. Since the objective functions and constraint functions are both complicated and nonlinear, traditional nonlinear programming may not be efficient and it may trap at a local minimum. Using crossover operations, mutation operations, and the stochastic selection in the genetic algorithm, trapping can be avoided. Experiments are conducted and the performance of the genetic algorithm is presented. Given the CAD model and the entities to be inspected, the active visual inspection planning system obtains the sensor setting that maximizes the probabilities of a required accuracy for each entity. © 2001 John Wiley & Sons, Inc.     

Opportunistic sampling-based active visual SLAM for underwater inspection

This paper reports on an active SLAM framework for performing large-scale inspections with an underwater robot. We propose a path planning algorithm integrated with visual SLAM that plans loop-closure paths in order to decrease navigation uncertainty. While loop-closing revisit actions bound the robot’s uncertainty, they also lead to redundant area coverage and increased path length. Our proposed opportunistic framework leverages sampling-based techniques and information filtering to plan revisit paths that are coverage efficient. We employ Gaussian process regression for modeling the prediction of camera registrations and use a two-step optimization procedure for selecting revisit actions. We show that the proposed method offers many benefits over existing solutions and good performance for bounding navigation uncertainty in long-term autonomous operations with hybrid simulation experiments and real-world field trials performed by an underwater inspection robot.     

A rule based approach for visual pattern inspection

The authors investigate the use of a priori knowledge about a scene to coordinate and control bilevel image segmentation, interpretation, and shape inspection of different objects in the scene. The approach is composed of two main steps. The first step consists of proper segmentation and labeling of individual regions in the image for subsequent ease in interpretation. General as well as scene-specific knowledge is used to improve the segmentation and interpretation processes. Once every region in the image has been identified, the second step proceeds by testing different regions to ensure they meet the design requirements, which are formalized by a set of rules. Morphological techniques are used to extract certain features from the previously processed image for rule verification purposes. As a specific example, results for detecting defects in printed circuit boards are presented     

Aspects of image fusion for automated visual inspection

In automated visual inspection, the evaluation of only one image is often not sufficient to obtain the desired information on a scene. Shortcomings originate at the image acquisition and are mainly due to the projection of the world by the optical system and the sampling and quantization in spatial and temporal dimensions that are necessary to obtain a digitized image. In consequence, image acquisition is always a noninjective mapping. However, by acquiring image series where at least one acquisition parameter has been varied systematically, the useful information about the scene can often be transformed such that it is conserved in some way in the image series. In this context, image fusion is the technique of reconstructing useful information about a scene by means of sophisticated processing of an image series. In this way, information can be gained that is not directly conceivable in a single image. The text was submitted by the author in English. Michael Heizmann (born 1971) received the M.S. degree in mechanical engineering in 1998 and the Ph.D. degree in automated visual inspection in 2004, both from the University of Karlsruhe, Germany. Since 2004, he is a Postdoctoral Research Assistant at the Fraunhofer Institute for Information and Data Processing IITB, Karlsruhe, where he heads the research group for Variable Image Acquisition and Processing. His research interests include image fusion, automated visual inspection, pattern recognition, and their applications in industrial quality inspection. He is author or coauthor of about 30 national and international publications, head of the technical committee on Image Processing in Measurement and Automation Engineering of the VDI (Association of German Engineers), and organizer of several workshops and conferences on image processing and fusion.     

A software laboratory for visual inspection and recognition

We present the design criteria and the basic structure of GYPSY, a software laboratory for visual inspection and recognition in industrial automation tasks.GYPSY analyzes images of objects on the basis of their two-dimensional shape. It is fully portable and easily programmable. The user can write application programs in a Pascal-like language to give the system the ability to take over various tasks.GYPSY has been developed for a multi-microprocessor architecture designed in Italy especially for industrial automation.     

Use of saccadic latency for visual inspection system

In some industrial inspection processes, multiprocess‐handling workers are required to enter the results of visual inspection tests without using their hands because of efficiency or hygiene reasons. We have developed a hands‐free visual inspection system by using saccadic latency, a temporal characteristic of saccadic eye movements. The proposed system is free from the Midas touch problem (i.e., the difficulty in developing an eye‐typing interface owing to the difficulty in differentiating between intentional blinks and gazes and natural ones). For verifying the system, an experiment was conducted in which ten subjects performed a visual inspection task. The average defect detection rates were 99.4%, and no Midas touch–related errors were observed. Results of the error analysis showed that redesigning the system interface would lead to a further enhancement of the system performance. © 2011 Wiley Periodicals, Inc.     

Factors affecting visual inspection accuracy

A four-fold classification of factors which might influence the extent of inspector accuracy is provided. It is emphasized that the influence of these factors has been mainly evaluated in the context of laboratory tasks which simulate real inspection tasks to a varying degree. For each of the four classes of factors one has been selected to illustrate its contribution to reducing inspector error. These are the inspector's visual acuity, the work-place lighting conditions, the time available for inspection and the provision of feedback or knowledge of results to the inspector. There is now sufficient evidence to confirm the advantages that can follow from careful consideration being given to these factors when designing or modifying industrial inspection tasks.     

Semiautomatic calibration of robot manipulator for visual inspection task

A semiautomatic procedure to calibrate a robot manipulator for improving positioning accuracy is proposed. An error-correcting transformation is estimated using the point-pairs in the theoretical and actual coordinate frames. The theoretical points are obtained using a 3D model of the inspected part, whereas the actual points are derived with image processing and analytic geometry techniques. The estimation of the error-correcting transformation is based on a leastsquare method. Experimental results are described to illustrate the calibration procedure.     

Specifying and optimizing robotic motion for visual quality inspection

Installation or even just modification of robot-supported production and quality inspection is a tedious process that usually requires full-time human expert engagement. The resulting parameters, e.g. robot velocities specified by an expert, are often subjective and produce suboptimal results. In this paper, we propose a new approach for specifying visual inspection trajectories based on CAD models of workpieces to be inspected. The expert involvement is required only to select – in a CAD system – the desired points on the inspection path along which the robot should move the camera. The rest of the approach is fully automatic. From the selected path data, the system computes temporal parametrization of the path, which ensures smoothness of the resulting robot trajectory for visual inspection. We then apply a new learning method for the optimization of robot speed along the specified path. The proposed approach combines iterative learning control and reinforcement learning. It takes a numerical estimate of image quality as input and produces the fastest possible motion that does not result in the degradation of image quality as output. In our experiments, the algorithm achieved up to 53% cycle time reduction from an initial, manually specified motion, without degrading the image quality. We show experimentally that the proposed algorithm achieves better results compared to some other policy learning approaches. The described approach is general and can be used with different types of learning and feedback signals.     

硅太阳能电池视觉检测方法研究

硅太阳能电池生产过程中的检测是一个重要的环节,针对硅太阳能电池片的特点,提出了一种基于视觉检测的硅太阳能电池检测方法。通过拟合直线和圆进行外形尺寸的测量,并在此基础上通过计算边缘点到拟合直线或圆弧的距离进行外形破损检测。针对丝印后电池表面纹理结构比较复杂的特点,采用多模板匹配并结合差影的方法进行丝印缺陷检测。实验结果表明：提出的方法具有稳定、可靠和精度高等特点,达到了快速、有效地实现硅太阳能电池缺陷检测的目的。     

A multilevel information fusion approach for visual quality inspection

For visual quality inspection systems to be applicable in industrial settings, it is mandatory that they are highly flexible, robust and accurate. In order to improve these characteristics a multilevel information fusion approach is presented. A first fusion step at the feature-level enables the system to learn from an undefined number of potential defects which might be segmented from the images. This allows for the quality control operators to label the data at the image-level and the sub-image-level, and use this information during the learning process. Additionally, the operators are allowed to provide a confidence measure for their labelling. The additional information obtained from the increased flexibility of the operator inputs allows to build more accurate classifiers. A second fusion step at the decision-level combines the classifications of different classifiers, making the system more accurate and more robust with respect to the classification method chosen. The experimental results, using various artificial and real-world visual quality inspection data sets, show that each of these fusion approaches can significantly improve the classification accuracy. If both information fusion approaches are combined the accuracy increases even further, significantly outperforming each of the fusion approaches on their own.     

Real-time image segmentation for visual inspection of pharmaceutical tablets

Visual appearance is an important quality factor of pharmaceutical tablets. Due to the vast quantities of produced tablets and high-quality requirements, pharmaceutical companies are interested in employing automated systems for real-time visual tablet inspection with the speeds of up to 100 tablets per second. Such systems require reliable tablet manipulation, illumination, image acquisition, tablet image analysis, classification, and sorting system. Tablet image segmentation, in which each tablet image is partitioned into the tablet region and background, is the first and very important step in tablet image analysis. It this paper, we propose a novel real-time segmentation method for grey-level images that is based on border tracking. The proposed method was designed to be accurate, robust, and computationally undemanding. The performances of the method were objectively assessed on a large number of simulated and real tablet images. The obtained results indicated high reliability, accuracy, and speed. The 100% reliability was obtained for segmentation of real images of pharmaceutical tablets, while the segmentation times were no more than 1.5 ms or 15% of the whole time available for tablet image analysis. As such, the proposed method proved feasible for real-time visual quality inspection of pharmaceutical tablets. Based on just a few assumptions that are usually fulfilled, the method may be a valuable segmentation tool for many other visual quality inspection applications.     

Image registration for visual inspection of imprinted pharmaceutical tablets

Image registration plays a vital role in visual quality inspection of tablets with imprints. In this paper, three registration methods, based on direct matching of pixel intensities, principal axes matching and circular profile matching, were proposed and objectively evaluated on real, large and representative image database (1,634 images) of various types of non-defective and defective tablets. Comparing against a devised “gold standard” registration and considering execution times and different registration tasks, circular profile matching method proved to be a powerful image processing tool for improving visual quality inspection of imprinted tablets in real time.     

Automated visual inspection: a survey

This paper surveys publications, reports, and articles dealing with automated visual inspection for industry. The references are organized according to their contents: overview and discussions, rationales, components and design considerations, commercially available systems, applications. A number of applications and their inspection methodologies are discussed in detail: the inspection of printed circuit boards, photomasks, integrated circuit chips. Other inspection applications are listed as a bibliography. A list of selectively annotated references in commercially available visual inspection tools is also included.