Automatic surface defect detection for mobile phone screen glass based on machine vision

Defect detection using machine vision technology plays an important role in the manufacturing process of mobile phone screen glass (MPSG). This study proposes an improved detection algorithm for MPSG defect recognition and segmentation. Considering the problem of MPSG image misalignment caused by vibrations in the mobile stages, a contour-based registration (CR) method is used to generate the template image used to align the MPSG images. Based on this registration result, the combination of subtraction and projection (CSP) is used to identify defects on the MPSG image, which can eliminate the influence of fluctuation in ambient illumination. To segment the defects with a fuzzy grey boundary from a noisy MPSG image, an improved fuzzy c-means cluster (IFCM) algorithm is developed in this study. A defect detection system is developed, and the proposed algorithms are validated using a number of experimental tests on MPSG images. The testing results demonstrate that the approach proposed in this study can effectively detect various defects on MPSG and that it has better performance than other methods.     

Robust minimization of lighting variation for real-time defect detection

In machine vision applications that involve comparing two images, it is necessary to match the capture conditions, which can affect their graylevels. Illumination and exposure are two important causes for lighting variation that we should compensate for in the resulting images. A standard technique for this purpose is to map one of the images to achieve the smallest mean square error (MSE) between the two. However, applications in defect detection for manufacturing processes are more challenging, because the existence of defects would affect the mapping significantly. In this paper, we present a robust method that is more tolerant to defects, and discuss its formulation as a linear programming to achieve fast implementations. This algorithm is also flexible and capable of incorporating further constraints, such as ensuring non-negativity of the pixel values.     

Discovering useful and understandable patterns in manufacturing data

Accurate planning of produced quantities is a challenging task in semiconductor industry where the percentage of good parts (measured by yield) is affected by multiple factors. However, conventional data mining methods that are designed and tuned on “well-behaved” data tend to produce a large number of complex and hardly useful patterns when applied to manufacturing databases. This paper presents a novel, perception-based method, called Automated Perceptions Network (APN), for automated construction of compact and interpretable models from highly noisy data sets. We evaluate the method on yield data of two semiconductor products and describe possible directions for the future use of automated perceptions in data mining and knowledge discovery.     

Automatic defect classification for semiconductor manufacturing

Visual defect inspection and classification are important parts of most manufacturing processes in the semiconductor and electronics industries. Defect classification provides relevant information to correct process problems, thereby enhancing the yield and quality of the product. This paper describes an automated defect classification (ADC) system that classifies defects on semiconductor chips at various manufacturing steps. The ADC system uses a golden template method for defect re-detection, and measures several features of the defect, such as size, shape, location and color. A rule-based system classifies the defects into pre-defined categories that are learnt from training samples. The system has been deployed in the IBM Burlington 16 M DRAM manufacturing line for more than a year. The system has examined over 100 000 defects, and has met the design criteria of over 80% classification rate and 80% classification accuracy. Issues involving system design tradeoff, implementation, performance, and deployment are closely examined.     

Automated defect inspection system for metal surfaces based on deep learning and data augmentation

Recent efforts to create a smart factory have inspired research that analyzes process data collected from Internet of Things (IOT) sensors, to predict product quality in real time. This requires an automatic defect inspection system that quantifies product quality data by detecting and classifying defects in real time. In this study, we propose a vision-based defect inspection system to inspect metal surface defects. In recent years, deep convolutional neural networks (DCNNs) have been used in many manufacturing industries and have demonstrated the excellent performance as a defect classification method. A sufficient amount of training data must be acquired, to ensure high performance using a DCNN. However, owing to the nature of the metal manufacturing industry, it is difficult to obtain enough data because some defects occur rarely. Owing to this imbalanced data problem, the generalization performance of the DCNN-based classification algorithm is lowered. In this study, we propose a new convolutional variational autoencoder (CVAE) and deep CNN-based defect classification algorithm to solve this problem. The CVAE-based data generation technology generates sufficient defect data to train the classification model. A conditional CVAE (CCVAE) is proposed to generate images for each defect type in a single CVAE model. We also propose a classifier based on a DCNN with high generalization performance using data generated from the CCVAE. In order to verify the performance of the proposed method, we performed experiments using defect images obtained from an actual metal production line. The results showed that the proposed method exhibited an excellent performance.     

基于局部与非局部线性判别分析和高斯混合模型动态集成的晶圆表面缺陷探测与识别

在复杂的半导体制造过程中,晶圆生产经过薄膜沉积、蚀刻、抛光等多项复杂的工序,制造过程中的异常波动都可能导致晶圆缺陷产生.晶圆表面的缺陷模式通常反映了半导体制造过程的各种异常问题,生产线上通过探测和识别晶圆表面缺陷,可及时判断制造过程故障源并进行在线调整,降低晶圆成品率损失.本文提出了基于一种流形学习算法与高斯混合模型动态集成的晶圆表面缺陷在线探测与识别模型.首先该模型开发了一种新型流形学习算法——局部与非局部线性判别分析法(Local and nonlocal linear discriminant analysis, LNLDA),通过融合数据局部/非局部信息以及局部/非局部惩罚信息,有效地提取高维晶圆特征数据的内在流形结构信息,以最大化数据不同簇样本的低维映射距离,保持特征数据中相同簇的低维几何结构.针对线上晶圆缺陷产生的随机性和复杂性,该模型对每种晶圆缺陷模式构建相应的高斯混合模型(Gaussian mixture model, GMM),提出了基于高斯混合模型动态集成的晶圆缺陷在线探测与识别方法.本文提出的模型成功地应用到实际半导体制造过程的晶圆表面缺陷在线探测与识别,在WM-811K晶圆数据库的实验结果验证了该模型的有效性与实用性.     

基于轻量化重构网络的表面缺陷视觉检测

基于深度学习的方法在某些工业产品的表面缺陷识别和分类方面表现出优异的性能,然而大多数工业产品缺陷样本稀缺,而且特征差异大,导致这类需要大量缺陷样本训练的检测方法难以适用.提出一种基于重构网络的无监督缺陷检测算法,仅使用容易大量获得的无缺陷样本数据实现对异常缺陷的检测.提出的算法包括两个阶段:图像重构网络训练阶段和表面缺陷区域检测阶段.训练阶段通过一种轻量化结构的全卷积自编码器设计重构网络,仅使用少量正常样本进行训练,使得重构网络能够生成无缺陷重构图像,进一步提出一种结合结构性损失和L1损失的函数作为重构网络的损失函数,解决自编码器检测算法对不规则纹理表面缺陷检测效果较差的问题;缺陷检测阶段以重构图像与待测图像的残差作为缺陷的可能区域,通过常规图像操作即可实现缺陷的定位.对所提出的重构网络的无监督缺陷检测算法的网络结构、训练像素块大小、损失函数系数等影响因素进行了详细的实验分析,并在多个缺陷图像样本集上与其他同类算法做了对比,结果表明重构网络的无监督缺陷检测算法有较强的鲁棒性和准确性.由于重构网络的无监督缺陷检测算法的轻量化结构,检测1 024×1 024像素图像仅仅耗时2.82 ms,...     

Efficient Anti-Glare Ceramic Decals Defect Detection by Incorporating Homomorphic Filtering

Nowadays the computer vision technique has widely found applications in industrial manufacturing process to improve their efficiency. However, it is hardly applied in the field of daily ceramic detection due to the following two key reasons: (1) Low detection accuracy as a result of ceramic glare, and (2) Lack of efficient detection algorithms. To tackle these problems, a homomorphic filtering based anti-glare ceramic decals defect detection technique is proposed in this paper. Considering that smooth ceramic surface usually causes glare effects and leads to low detection results, in our approach, the ceramic samples are taken in low light environment and their luminance and details restored by a homomorphic filtering based image enhancement technique. With relatively high quality preprocessed images, an effective ceramic decal defect detection algorithm is then designed to rapidly locate those out-of-bounds defects and further estimate their size. The experimental results show that the proposed scheme could achieve its desired performance.     

Novel method for detection of mixed-type defect patterns in wafer maps based on a single shot detector algorithm

Journal of Intelligent Manufacturing - In semiconductor manufacturing, detecting defect patterns is important because they are directly related to the root causes of failures in the wafer process....     

IC真实缺陷的边界提取和缺陷尺寸与形状的表征

为了对ＩＣ制造中真实多余物缺陷进行分类与识别,ＩＣ多余物缺陷的特征提取是非常重要的一步,文中首先提出一种基于数学形态学的ＩＣ真实多余物缺陷边界的检测方法,其次对边界进行了链码描述,最后对边界所表示的多余物缺陷进行了尺寸测量与形状分析,在预处理阶段,利用彩色ＨＳＶ模型分割原ＩＣ图像,然后用图形学开运算消除背景噪音,对开后的结果图像进行形态膨胀及形态腐蚀运算,消除多余物缺陷中的小洞噪音以获得从复杂背景     

Knowledge-based linguistic equations for defect detection through functional testing of printed circuit boards

Increasing globalization of the economy is imposing tough challenges to manufacturing companies. The ability to produce highly customized products, in order to satisfy market niches, requires the introduction of new features in automation systems. Flexible manufacturing processes must be able to handle unforeseen events, but their complexity makes the supervision and maintenance task difficult to perform by human operators.This paper describes how linguistic equations (LE), an intelligent method derived from Fuzzy Algorithms, has been used in a decision-helping tool for electronic manufacturing. In our case the company involved in the project is mainly producing control cards for the automotive industry. In their business, nearly 70% of the cost of a product is material cost. Detecting defects and repairing the printed circuit boards is therefore a necessity. With an ever increasing complexity of the products, defects are very likely to occur, no matter how much attention is put into their prevention. Therefore, the system described in this paper comes into use only during the final testing of the product and is purely oriented towards the detection and localization of defects. Final control is based on functional testing. Using linguistic equations and expert knowledge, the system is able to analyze that data and successfully detect and trace a defect in a small area of the printed circuit board. If sufficient amount of data is provided, self-tuning and self-learning methods can be used. Diagnosis effectiveness can therefore be improved from detection of a functional area towards component level analysis.     

Low-contrast surface inspection of mura defects in liquid crystal displays using optical flow-based motion analysis

This paper proposes a machine vision scheme for mura defect detection in LCD manufacturing. Mura is a Japanese word for blemish, which typically shows brightness imperfections from its surroundings in the surface. It appears as a low-contrast region without clear edges. Traditional automatic visual inspection algorithms detect mura defects from individual still images. They neglect that a mura defect may not be visually sensed in the image from a stationary system. In this study, the LCD panel is assumed to move along a track. While the panel passes through a fixed camera, the light reflection from different angles can effectively enhance the mura defect in the low-contrast images. The mura detection problem is therefore treated as a motion analysis in image sequences using optical flow techniques. Since a LCD panel moves along a single direction, both two-dimensional and one-dimensional optical flow methods are developed. Three discriminative features based on flow magnitude, mean flow magnitude and flow density in the optical flow field are presented to extract the defective regions. Both real panel images and synthetic surface images are used to evaluate the efficacy of the proposed methods. Experimental results have shown that the proposed 1D optical flow method works as well as the 2D optical flow method to detect very low-contrast mura defects of small size, and achieves a high processing rate around 20 frames per second for images of size 200 × 200.     

Machine learning-based novelty detection for faulty wafer detection in semiconductor manufacturing

Since semiconductor manufacturing consists of hundreds of processes, a faulty wafer detection system, which allows for earlier detection of faulty wafers, is required. statistical process control (SPC) and virtual metrology (VM) have been used to detect faulty wafers. However, there are some limitations in that SPC requires linear, unimodal and single variable data and VM underestimates the deviations of predictors. In this paper, seven different machine learning-based novelty detection methods were employed to detect faulty wafers. The models were trained with Fault Detection and Classification (FDC) data to detect wafers having faulty metrology values. The real world semiconductor manufacturing data collected from a semiconductor fab were tested. Since the real world data have more than 150 input variables, we employed three different dimensionality reduction methods. The experimental results showed a high True Positive Rate (TPR). These results are promising enough to warrant further study.     

Deep-learning-based anomaly detection for lace defect inspection employing videos in production line

Defect inspection plays an essential role in ensuring quality of industrial products. The most widely used human visual inspection method has some drawbacks such as high cost and low efficiency, which bring an eager demand for the application of automatic defect inspection algorithm in actual production. However, few industrial production lines use automatic detection devices due to the gap between data collected in the actual production environment and ready-made datasets. Lace is one of the industrial products which completely depends on manual defect inspection. The complex and fine texture of lace makes it difficult to extract regular patterns using the existing image-based defect inspection methods. In this paper, we propose to collect lace videos in the weaving stage and design a deep-learning-based anomaly detection framework to detect lace defects. The framework contains three stages, namely video pre-processing stage, pixel reconstruction stage and pixel classification stage. In the offline phase, only defect-free lace videos are needed to train the pixel reconstruction model and calculate the detection threshold by our adaptive thresholding method. In the online phase, the proposed framework reconstructs lace videos and performs defect inspection using reconstruction error and the pre-set threshold. As far as we know, this paper the first to detect fabric defects by videos. Experimental results on artificial defect videos demonstrate the effectiveness of the proposed framework.     

Defect cluster recognition system for fabricated semiconductor wafers

The International Technology Roadmap for Semiconductors (ITRS) identifies production test data as an essential element in improving design and technology in the manufacturing process feedback loop. One of the observations made from the high-volume production test data is that dies that fail due to a systematic failure have a tendency to form certain unique patterns that manifest as defect clusters at the wafer level. Identifying and categorising such clusters is a crucial step towards manufacturing yield improvement and implementation of real-time statistical process control. Addressing the semiconductor industry’s needs, this research proposes an automatic defect cluster recognition system for semiconductor wafers that achieves up to 95% accuracy (depending on the product type).     

Defective wafer detection using a denoising autoencoder for semiconductor manufacturing processes

Defective wafer detection is essential to avoid loss of yield due to process abnormalities in semiconductor manufacturing. For most complex processes in semiconductor manufacturing, various sensors are installed on equipment to capture process information and equipment conditions, including pressure, gas flow, temperature, and power. Because defective wafers are rare in current practice, supervised learning methods usually perform poorly as there are not enough defective wafers for fault detection (FD). The existing methods of anomaly detection often rely on linear excursion detection, such as principal component analysis (PCA), k-nearest neighbor (kNN) classifier, or manual inspection of equipment sensor data. However, conventional methods of observing equipment sensor readings directly often cannot identify the critical features or statistics for detection of defective wafers. To bridge the gap between research-based knowledge and semiconductor practice, this paper proposes an anomaly detection method that uses a denoise autoencoder (DAE) to learn a main representation of normal wafers from equipment sensor readings and serve as the one-class classification model. Typically, the maximum reconstruction error (MaxRE) is used as a threshold to differentiate between normal and defective wafers. However, the threshold by MaxRE usually yields a high false positive rate of normal wafers due to the outliers in an imbalanced data set. To resolve this difficulty, the Hampel identifier, a robust method of outlier detection, is adopted to determine a new threshold for detecting defective wafers, called MaxRE without outlier (MaxREwoo). The proposed method is illustrated using an empirical study based on the real data of a wafer fabrication. Based on the experimental results, the proposed DAE shows great promise as a viable solution for on-line FD in semiconductor manufacturing.     

晶圆表面缺陷在线检测研究

针对准确与实时检测晶圆表面缺陷的需求,提出了一种基于主成分分析(Principal Component Analysis, PCA)和贝叶斯概率模型(Bayesian Probability Model, BPM)的在线检测算法。首先,改进双边滤波方法以消除晶圆表面图像中的噪声和突出晶圆表面的模式特征。然后,提取晶圆表面缺陷的Hu不变矩、方向梯度直方图(Histogram of Oriented Gradients, HOG)和尺度不变特征变换特征(Scale Invariant Feature Transform, SIFT)。接着,采用PCA方法对特征进行降维。最后,在离线建模阶段构建各种缺陷模式的BPMs;在在线检测阶段采用胜者全取(Winner-take-all, WTA)法判断缺陷的模式和构建新缺陷模式的BPMs。提出算法在WM-811K晶圆数据库中得到了87.2%的检测准确率。单副图像的平均检测时间为40.5ms。实验结果表明,提出算法具有较高的检测准确性与实时性,可以实际应用到集成电路制造产线的晶圆表面缺陷在线检测中。     

Towards reliable object detection in noisy images

Image noise is a common problem frequently caused by insufficient lighting, low-quality cameras, image compression and other factors. While low image quality is expected to degrade results of visual recognition, most of the current methods and benchmarks for object recognition, such as Pascal Visual Object Classes Challenge and Microsoft Common Objects in Context Challenge, focus on relatively high-quality images. Meanwhile, object recognition in noisy images is a common problem in surveillance and other domains. In this work we address object detection in noisy images and propose a novel low-cost method for image denoising. When combined with the standard Deformable Parts Model and Regions with Convolutional Neural Network object detectors, our method shows improvements of object detection under varying levels of image noise. We present a comprehensive experimental evaluation and compare our method to other denoising techniques as well as to standard detectors re-trained on noisy images. Results are presented for the common Pascal Visual Object Classes benchmark for object detection and KAIST Multispectral Pedestrian Detection Benchmark with the real noise presence in night images.     

An empirical study of design-of-experiment data mining for yield-loss diagnosis for semiconductor manufacturing

To maintain competitive advantages, semiconductor industry has strived for continuous technology migrations and quick response to yield excursion. As wafer fabrication has been increasingly complicated in nano technologies, many factors including recipe, process, tool, and chamber with the multicollinearity affect the yield that are hard to detect and interpret. Although design of experiment (DOE) is a cost effective approach to consider multiple factors simultaneously, it is difficult to follow the design to conduct experiments in real settings. Alternatively, data mining has been widely applied to extract potential useful patterns for manufacturing intelligence. However, because hundreds of factors must be considered simultaneously to accurately characterize the yield performance of newly released technology and tools for diagnosis, data mining requires tremendous time for analysis and often generates too many patterns that are hard to be interpreted by domain experts. To address the needs in real settings, this study aims to develop a retrospective DOE data mining that matches potential designs with a huge amount of data automatically collected in semiconductor manufacturing to enable effective and meaningful knowledge extraction from the data. DOE can detect high-order interactions and show how interconnected factors respond to a wide range of values. To validate the proposed approach, an empirical study was conducted in a semiconductor manufacturing company in Taiwan and the results demonstrated its practical viability.     

基于视觉感知机制的织物疵点轮廓检测

针对传统人工织物疵点检测存在的误检及低效等问题,提出了一种基于视觉感知机制的自适应织物疵点轮廓检测方法.首先,模拟视觉系统中视网膜感受野对视觉信息的处理机制对织物疵点图像进行滤波及疵点增强;其次,依据初级视皮层(V1)区对视觉信息响应的方向选择性机制构建织物疵点图像边缘检测模型,实现对织物疵点图像的边缘检测.最后,采用自适应阈值选择的方法对检测到的边缘进行二次处理,获得织物图像疵点的轮廓.为验证本文方法的有效性和准确性,对4类织物疵点图像进行测试,并定性和定量两方面进行比较分析,结果表明文中提出的方法能够较好地检测出织物疵点轮廓信息,不仅可以得到质量较高的织物疵点轮廓图像,而且在整个检测过程中能够自适应的选择参数,避免受人的主观因素影响,具有实际的应用价值.