Automated Refinement of Automated Visual Inspection Algorithms

One of the challenges faced by the users of automated visual inspection (AVI) systems is how to efficiently upgrade the legacy systems to inspect the new components introduced into the assembly lines. If the AVI systems are not flexible enough to accommodate new components, they will be rendered obsolete even by small changes in the product being inspected. The overall objective of the research presented in this paper is to produce the methodological basis that will result in the development of highly reconfigurable AVI systems. In this paper, we focus on part of this overall development, the adaptation of preexisting inspection algorithms to inspect similar components introduced into the assembly line. While this paper bases its development and discussion on the inspection of surface mounted devices (SMDs), the proposed methodology is general enough to be applicable to a broad range of inspection problems.      

Automated visual inspection: A tutorial

This article introduces the topic of automated visual inspection (AVI) as a subset of machine vision. The components of a generic AVI system will be presented and current applications of AVI systems will be discussed. Specifically, the applications of automated dimension verification, bare printed circuit board inspection, stuffed printed circuit board inspection and component identification using color technology will be covered in some detail.     

Application of statistical modeling of image spatial structures to automated visual inspection of product quality

Automated visual inspection (AVI) attracts increasing interest in product quality control both academic and industrial communities, particularly on mass production processes, because product qualities of most types can be characterized with their corresponding surface visual attributes. However, many product images in AVI systems are comprised of stochastically accumulative fragmentations (particles) of local homogeneity, without distinctive foregrounds and backgrounds, which brings great challenges in computer analysis, e.g., rice images, fabric images, and consequently, in the intelligent identification of the product qualities. A method of Weibull distribution (WD)-based statistical modeling of image spatial structures (ISSs) to inspect automatically the product quality is presented. The ISS, obtained with multi-scale and omnidirectional Gaussian derivative filters (OGDFs), was demonstrated to be subject to a representative WD model of integral form based on the theory of sequential fragmentation in advance. The WD-model parameters (WD-MPs) of the ISS, with essential human perceptual significance, were extracted as the visual features for product quality identification. The classification performance of the proposed product quality inspection method, namely, the proposed WD-MP features integrated with an introduced spline regression (SR) classifier in this study, was verified on two case studies in the field of the AVI of product quality, namely, automated rice quality classification, and intelligent fabric quality assessment in the corresponding assembly lines of industrial scale. Experimental results indicate that the proposed WD-MP features can effectively characterize the statistical distribution profiles of ISS of complex grain images, piled with a large number of stochastically accumulative fragmentations. The proposed method provides an effective tool for grain image modeling and analysis and consequently lays a foundation for the intelligent perception of product qualities on assembly lines.     

SMD LED chips defect detection using a YOLOv3-dense model

This research develops a defect detector on the basis of the you only look once version 3, YOLOv3, and applies a process to inspect the chips of surface-mounted device light-emitting diodes (SMD LED). In addition to an inspection process that identifies defect-free products, the model also identifies missing components, incorrect placement, inverse polarity, missing wires and defective surface. To improve the performance of the defect detector, the research substitutes densely connected convolutional networks (DenseNet) for the Darknet-53 backbone, producing a model that is called YOLOv3-dense. The research introduces the Taguchi method to evaluate the sensitivity of YOLOv3-dense’s hyper-parameters in a fewer number of experiments. The result shows that the testing mean average precision (mAP) measured by YOLOv3-dense is 33.69% higher than the class activation mapping (CAM) localization of convolutional neural network (CNN). The testing mAP is also 14.98% higher than traditional YOLOv3.     

基于二值投影的PCB元件安装缺陷检测算法研究

研究分析了适用于AOI设备的PCB表面安装元件的缺陷检测算法.使用二值投影分析方法对2种元件类型的缺陷检测方法进行了研究,包括针对贴片电阻电容类型的chip元件和集成电路芯片类型的IC元件的缺陷检测方法.使用VC++6.0编写MFC程序实现算法,并制作了各种元件图像进行实验测试.实验结果表明,提出的方法能够快速有效的对两种类型的元件安装缺陷进行检测.     

Model-based inspection system for component boards

A computer vision system has been developed to perform the complex inspection of many components on a flat surface or board. It has a hierarchical structure which allows it to determine the location of components with one camera resolution and then to inspect for small details with a higher resolution. The system has generality in that the same program can be used to inspect boards with completely different components by forming a new model. The MODEL-FORMER program carries out an interactive session with an operator who specifies the layout of components, showing the system each different component and selecting inspection tests which apply to specific regions of the component. These tests allow the INSPECTOR system to differentiate between good and bad components. During inspection, the system (using gray-scale rather than the more limited binary techniques) automatically inspects all components on the board for: (1) presence of correct component; (2) correct position and orientation; (3) any other important features which the operator has designated.     

PCAAD—an object-oriented expert system for assembly of printed circuit boards

Abstract: Electronics manufacturing is the fastest growing segment of the manufacturing domain today. The rapid change in printed circuit board manufacturing has been possible with the induction of surface-mounted components and robotic assembly. The high density packaging capability of surface-mounted technology has contributed enormously to the development of new trends in printed circuit board manufacturing.
PCAAD—Printed Circuit Board Assembly Advisor—is an expert system developed to capture this new technology and its interaction with the existing through-hole-mounting technology. In building the system, rules and guidelines pertaining to assembly are formulated and incorporated into the knowledge-base. The system is built on an object-oriented paradigm. The object-oriented approach, Smalltalk-80, the object-oriented language and the hierarchical rule structure paradigm are briefly discussed. A rating system is also introduced to rank various assembly designs. PCAAD can advise whether or not to proceed with assembly with a particular assembly design, and suggests assembly modification at the early stage of design to eliminate the assembly problems in the later phase of board production.     

Subminiature binary camera design for inspection and robotic assembly applications

The article describes the design and utilization of a small, lightweight digital camera intended to assist in robot assembly tasks and to provide a low cost visual inspection tool. The main operational features of the design are outlined together with some sample applications involving the identification of integrated circuit devices, and the location of assembly sites on a printed wiring board.     

Design and application of industrial machine vision systems

In this paper, the role and importance of the machine vision systems in the industrial applications are described. First understanding of the vision in terms of a universal concept is explained. System design methodology is discussed and a generic machine vision model is reported. Such a machine includes systems and sub-systems, which of course depend on the type of applications and required tasks. In general, expected functions from a vision machine are the exploitation and imposition of the environmental constraint of a scene, the capturing of the images, analysis of those captured images, recognition of certain objects and features within each image, and the initiation of subsequent actions in order to accept or reject the corresponding objects. After a vision system performs all these stages, the task in hand is almost completed. Here, the sequence and proper functioning of each system and sub-systems in terms of high-quality images is explained. In operation, there is a scene with some constraint, first step for the machine is the image acquisition, pre-processing of image, segmentation, feature extraction, classification, inspection, and finally actuation, which is an interaction with the scene under study. At the end of this report, industrial image vision applications are explained in detail. Such applications include the area of automated visual inspection (AVI), process control, parts identification, and important role in the robotic guidance and control. Vision developments in manufacturing that can result in improvements in the reliability, in the product quality, and enabling technology for a new production process are presented. The key points in design and applications of a machine vision system are also presented. Such considerations can be generally classified into the six different categories such as the scene constraints, image acquisition, image pre-processing, image processing, machine vision justification, and finally the systematic considerations. Each aspect of such processes is described here and the proper condition for an optimal design is reported.     

基于便携设备的螺钉状态视觉检测方法研究

生产现场中,尤其是小批量大规模设备的装配和调试过程中,螺钉需求量大,螺钉连接状态对装配质量有着重大影响,目前对于该过程的检验仍需要工人目视检验并手动记录,这一过程主观性强,自动化水平低,效率低下。针对装配过程中的螺钉状态进行检测,基于视觉检测的方式,识别螺钉的拧紧程度,提高检测准确率和效率。首先,通过对线性特征的有效匹配来对螺钉进行定位,之后,对螺钉区域进行有效分割,提取螺钉区域的特征,最后对提取的拧紧螺钉和未拧紧螺钉的特征进行分类识别,识别出设备上的未拧紧螺钉,精度可达到2mm以内,并采用便携式检测设备,方便部署,可用于小批量大规模设备的装配质量检测中。     

Automated visual inspection in the semiconductor industry: A survey

Automated visual inspection is an image-processing technique for quality control and production line automation. This paper reviews various optical inspection approaches in the semiconductor industry and categorize the previous literatures by the inspection algorithm and inspected products. The vision-based algorithms that had been adopted in the visual inspection systems include projection methods, filtering-based approaches, learning-based approaches, and hybrid methods. To discuss about the practical applications, the semiconductor industry covers the manufacturing and production of wafer, thin-film transistor liquid crystal displays, and light-emitting diodes. To improve the yield rate and reduce manufacturing costs, the inspection devices are widely installed in the design, layout, fabrication, assembly, and testing processes of production lines. To achieve a high robustness and computational efficiency of automated visual inspection, interdisciplinary knowledge between precision manufacturing and advanced image-processing techniques is required in the novel system design. This paper reviews multiple defect types of various inspected products which can be referenced for further implementations and improvements.     

基于多信息算法融合的电子元件精准装配研究

机器人是现代化工业制造与生产的重要装备之一。随着市场需求向小批量、多品种和柔性化方向快速发展,基于多信息融合的机器人协作系统将为高端精密制造产业赋能。该研究着眼于精密电子元件装配领域,聚焦手眼系统的精准对位和精密插装技术,通过建立待插元件与非均质薄板的接触状态模型,分析其双重位移融合的力位运动特性,并结合视觉检测与跟踪技术,提出一种融合视觉、力觉和编码器信息的复合型控制算法。基于电子元件装配平台,该研究进行了元件插装对比实验和信息融合算法的装配实验,结果表明,对齐阶段的定位精度在 0.185 pixels 以内,装配阶段的接触状态判定和调节算法保障了元件与插槽的安全有效装配。     

Automatic Optical Inspection System with Telecentric Optics and Phase-measuring Profilometry for Highly Accurate Localization of Electronic Packages

Typical visual placement inspection systems in semiconductor or electronics manufacturing industries use only highly-magnified two-dimensional imaging with controlled illumination for high accuracy. However, current semiconductor packaging technology needs more precise manufacturing processes for highly integrated semiconductor packages, in which high-precision alignment between the semiconductor die and the matched substrate is indispensable for successful electronic interconnection. To solve this problem, an advanced visual placement inspection system for 3D stacking of electronic components is proposed; it utilizes 3D profilometry and 2D telecentric optics. In addition to the introduction of the proposed system, the system calibration algorithms and the information processing algorithms for accurate positioning of the semiconductor die, as well as the implemented inspection system are described in detail. To verify the system performance, a series of real experiments on newly developed flip-chip packages for high-performance computing were performed, and the results are analyzed and discussed.

     

Introduction to assembly features: an illustrated synthesis methodology

The article is an introduction to a new concept of assembly features able to support intelligent design and manufacturing of complex products. An assembly feature is defined as a generic solution referring to two groups of parts that need to be related by a relationship so as to solve a design problem. The concept of assembly feature encompasses the notions of design intent, technical function, technological solution and manufacturing process as well as it provides a justification for the use of part features. After a general introduction and a justification of the interest of assembly features, some guidelines are provided to show how assembly features can be characterized in any domain of application concerned with engineering design of assemblies. As an illustration of the proposed methodology, the concept is finally applied to the engineering phase of aeroplane design.     

COMPUTER VISION INSPECTION OF ELLIPTICAL PROFILES

Abstract

Computer vision is a complex and multidisciplinary field that is still in its early stages of development. Many of the current applications of computer vision are in the field of inspection because of the relatively low cost of these systems and the need for complete inspection in certain high-volume automated manufacturing or assembly processes. The techniques used to inspect the geometric shape and size of an object appear to be application-specific. This article is concerned with the development of a practical method to measure and inspect parts with an elliptical profile. The method is based on the minimization of the sum of the squares of the central norm errors of a given set of boundary points. The central norm error is defined as the Euclidean distance between a boundary point and the point in the best-fit ellipse along the extension line from the center to the boundary point.     

Real-Time Automatic Visual Inspection of High-Speed Plane Products by Means of Parallelism

Parallel processing can effectively satisfy the real-time constraints required by on-line machine vision applications. This paper describes a real-time automatic visual inspection (AVI) system for high-speed plane products, which is based on a reconfigurable and scalable coarse-grain distributed memory MIMD architecture, and a unique application programming interface. The code of application algorithms are source level machine independent. As our prototype is organized as a test bed, new algorithms and even new dedicated hardware processing elements can quickly be evaluated on it. The system can easily be tailored for various automatic visual surface inspection applications. An example for inspecting whether separate small mosaics (textures) in a scene are in normal shapes is described. The algorithm combines the connected component labeling, the moment calculation and the pattern recognition. This key task for most applications is well suited for the “divide and conquer” parallel paradigm. The real-time performance has been achieved on a TMS320C40 array.     

An evolutionary factor analysis computation for mining website structures

This paper explores website link structure considering websites as interconnected graphs and analyzing their features as a social network. Two networks have been extracted for representing websites: a domain network containing subdomains or external domains linked through the website and a page network containing webpages browsed from the root domain. Factor analysis provides the statistical methodology to adequately extract the main website profiles in terms of their internal structure. However, due to the large number of indicators, the task of selecting a representative subset of indicators becomes unaffordable. A genetic search of an optimum subset of indicators is proposed in this paper, selecting a multi-objective fitness function based on factor analysis results. The optimum solution provides a coherent and relevant categorization of website profiles, and highlights the possibilities of genetic algorithms as a tool for discovering new knowledge in the field of web mining.     

Formal design and implementation of constraints in software components

This paper deals with the modeling, automatic implementation and runtime verification of constraints in component-based applications. Constraints have been assuming an ever more relevant role in modeling distributed systems as long as business rules implementation, design-by-contract practice, and fault-tolerance requirements are concerned. Nevertheless, component developers are not sufficiently supported by existing tools to model and implement such features.In this paper, we propose a methodology and a set of tools that enable developers both to model component constraints and to generate automatically component skeletons that already implement such constraints. The methodology has been extended to support implementation even in case of legacy components.     

Object oriented concepts and mechanisms for feature-based computer integrated inspection

Computer vision can provide fast, accurate and reliable inspection. In this paper is described a knowledge-based vision inspection planning system for manufactured components based on their CAD design models. An object oriented methodology is developed to capture the knowledge such as the geometric entities, their relationships, sensors and plans. Reasoning mechanisms are developed to automatically determine various attributes of the different features of an object to be inspected, and to generate the alternative strategies for inspection of each attribute. Mechanisms for automated generation of sensor settings and optimization of the inspection plan are also developed. Sensor constraints are considered in determining the camera parameters. This inspection methodology is flexible, and it is easily integrated with the design and process planning aspects of an integrated manufacturing system.     

An assembly decomposition model for subassembly planning considering imperfect inspection to reduce assembly defect rates

The assembly decomposition is to divide the assembly to subassemblies that are to be joined in the final assembly processes. The assembly decomposition decision strongly affects the effectiveness of a product assembly in terms of quality, sequence and supplier selection. This paper presents an assembly-decomposition model to improve product quality. Mixed-integer programming is used to partition the liaison graph of a product assembly. The mixed-integer programming model takes into account the defect rates in components and assembly tasks. The defect rate of the final assembly product is to be minimized considering type II errors in subassembly inspection. A numerical example is presented to demonstrate the methodology, and this numerical study shows that assembly decomposition strongly affects the final assembly defect rate. The developed assembly decomposition method is expected to enhance the decision making in assembly planning.