Computer-Aided System for Defect Inspection in the PCB Manufacturing Process

INES 2012 IEEE 16th International Conference on Intelligent Engineering Systems June 13 15, 2012, Lisbon, Portugal Computer-Aided System for Defect Inspection in the PCB Manufacturing Process T.J. Mateo Sanguino * and M. Smolčić-Rodríguez ** * Department of Electronic Engineering, Computer Systems and Automatics, University of Huelva, Spain ** Optotechnik & Bildverarbeitung, University of Applied Sciences, Hochschule Darmstadt, Germany tomas.mateo@diesia.uhu.es, r-smolcic@gmx.de Abstract This paper presents a visual inspection system aimed at the automatic detection and classification of bare- PCB manufacturing errors. The interest of this CAE system lies in a twofold approach. On the one hand, we propose a modification of the subtraction method based on reference images that allows higher performance in the process of defect detection. On the other hand, this method is combined with a particle classification algorithm based on two measures of light intensity. As a result of this strategy, a machine vision application has been implemented to assist people in etching, inspection and verification tasks of PCBs. I. INTRODUCTION Automatic optical inspection (AOI) systems are widely present nowadays in manufacturing, inspection and assembling processes of printed circuit boards (PCB). An excellent review of this field which has laid the basis for many later works has been carried out in [1]. This study provides a first definition for the type of defects existing in PCBs and categorizes the automatic inspection algorithms in reference comparison methods, nonreferential comparison methods, and hybrid inspection methods. The reference comparison approach consists in comparing pixel-by-pixel both the image of a PCB and that of an ideal design. For this purpose, ideal images are usually conforming to both CAD/CAM/CAE software and predefined models by standard databases [2]. The main difficulty of these techniques lies in obtaining precise alignments and uniform lighting conditions on images. By the other hand, the non-referential approach is mainly based on the design rule checking (DRC) method for bare PCBs. Although the latter is relatively easier to implement compared to reference comparison methods, the DRC method requires higher processing times and it is limited regarding the type of detected defects [3]. The advantages of reference comparison methods which are mostly used and DRC methods have been combined with other novel techniques resulting in hybrid inspection methods. Thus, this approach includes for example neuronal networks [4], Haar wavelet transform [5] or influence maps [6]. However, these techniques may become more complex to implement. In the field of reference comparison methods to which this paper belongs, there are several proposed techniques. In this context, a simple algorithm for object classification based on the boundary state transition (BST) method is presented [7]. This work contributes a noteworthy application that assists people in recognition tasks of PCB defects. Moreover, a set of image processing algorithms based on image subtraction and simple logical operations is proposed [8]. Despite the implementation is able to detect a wide number of defects on PCBs, the proposed application lacks the ability to classify objects automatically and independently. Based on these arithmetical and logical operations, the use of thresholding and particle analysis techniques to detect defective items is proposed [9]. However, this work consists of a simple script without possibility of implementing an automatic object classifier. Regarding applications used for machine vision purposes, the Títere system has been developed to help students to assimilate certain AOI techniques by PCB inspection [10]. The application includes a great variety of image processing techniques (binarized and noise reduction), segmentation methods, and defect searching based on diverse morphological techniques (e.g. erosion, dilation, opening, closing, hit-ormiss, slimming, or pruning). Nevertheless, users must search and classify manually all those peculiar objects which may be considered defects. Regarding automatic inspection and classification processes, systems have mainly focused on hardware development aimed at the manufacture and assembly of PCBs. As an example, SVP500 by Creasoft is able to inspect both tracks and welding paste on boards. This system uses fiducial points to orientate PCBs and its reference comparison method is based on standard Gerber files [11]. This file format contains useful design information for PCB building by CAD/CAM/CAE systems. Similarly, a high-speed vision system YC- MC726 by Yunco Industrial Co is aimed at the PCB film mask inspection. As an advantage, defects are detected through an algorithm based on DRC and CAM reference comparison methods. Finally, a remarkable example is the OptiCheck system, which is implemented for the continuous inspection of welding paste on production lines [12]. The reference comparison method is based on comparing images from a printer and real SMT (Surface Mount Technology) components. However, they all represent commercial systems which involve a high cost in many cases. This contribution describes a new application aimed at the automatic inspection and classification of PCB defects. The developed tool aims to improve various 978-1-4673-2695-7/12/$31.00 2012 IEEE 151

T. J. M. Sanguino and M. Smolčić-Rodriguez Computer-aided System for Defect Inspection TABLE I. FEATURES OF SOME AOI SYSTEMS FOR MANUFACTURING, INSPECTION AND/OR ASSEMBLING PROCESSES OF PCBS System/Author Programming Language PCB Inspection Image Processing Detected Automatic GUI Year Method Objects Classifier Títere Java Morphologic Noise Reduction & 5 2003 Segmentation Rau et al. C++ Reference Comparison Subtraction 8 BST 2005 Ibrahim et al. C++ & MVTools TM Haar Wavelet Domain Noise Reduction & 14 2005 Subtraction Leta et al. MLC++ Thresholding & Segmentation & 3 2008 Influence Maps Subtraction Khalid et al. Image Processing Toolbox Reference Comparison XOR, NOT, Addition 14 2008 by MATLAB TM & Flood-Fill Singh Vision Assistant by Reference Comparison XOR, Thresholding & 3 2011 Chauhan et al. National Instruments Particle Analysis Scanweiter LabVIEW TM Reference Comparison Subtraction, Kernel 12 Light 2011 Filter & Danielsson Map Intensity YC-MC726 Proprietary System DRC & CAM Reference Comparison Subtraction 9 2012 aspects of the systems mentioned above. In order to assess the contributions that this work makes to the field of AOI systems, Table I shows a comparison of the features and capabilities of our application Scanweiter. Then, this paper is organized as follows. Section II describes the proposed machine vision algorithm, which is the fundamental objective of this work. In particular, it formulates the detection and classification methodology of PCB defects. In Section III the implemented AOI system is presented. Section IV discusses the impact of the inspection method in terms of performance. Finally, the findings from the developed work are presented. II. AUTOMATIC INSPECTION METHOD This work presents a new approach to improve the reference comparison method mentioned in the Introduction section. The proposed method is based on the use of built PCBs and CAD files. To this end, images are provided by means of scanners and EAGLE layout editor software respectively. Figure 1 shows the developed AOI system, which consists of four main stages: reference image preprocessing (a), image calibration (b), inspection process (c), and classification process (d). A. Configuration and Preprocessing The AOI system consists of a graphical user interface (GUI), which first stage requires setting the application parameters by the user. These controls stand for rules that define the performance of the inspection and classification algorithms. Then, images are preprocessed to correct brightness, contrast and gamma values for each color plane separately by means of a look-up-table (LUT). Instead of using a mathematical expression to calculate new values for each pixel, LUT provides an easier and faster programmable method [13]. This step is used to transform image pixels so that brightness, contrast and gamma values typically nonlinear gain a more uniform distribution. This ensures that images taken with scanners have the required quality to be processed with the AOI system. Afterwards, a threshold function is applied to each RGB plane of an image to work with grayscale images (0, 255). As a result, setting the appropriate value for this parameter (T = 50), a binary image is obtained. Reducing information is crucial to process images swiftly and without compromising the essential properties, since the processing time becomes critical for the implementation of a feasible AOI system (see Experimentation section). Afterwards, pixels of PCB images are inverted to obtain a negative image that can be compared with its reference CAD image [14]. Second stage consists of an image calibration process, which comprises rotation, displacement and scaling processes. To do this programmatically, a common feature present in all PCBs is required. This singularity corresponds to a rectangular frame surrounding PCBs, since this element is easy to detect by means of blob analysis. Several parameters are taken into account when detecting the outer edge of an image (see Fig. 2). These comprise the number of pixels that belong and fall outside the edge (steepness and filter width), as well as parameters which define contrast threshold, searching Figure 1. An overview of the AOI system steps 152

INES 2012 IEEE 16th International Conference on Intelligent Engineering Systems June 13 15, 2012, Lisbon, Portugal TABLE II. CATEGORIZATION OF THE TYPES OF DEFECTS IN PCBS # Original Pad Defective Particle Definition Group 1 Missing Hole Missing 2 Pinhole Missing 3 Short Excess Figure 2. Edge searching method on an image order and direction (left, top, bottom, right), and distance between searching lines (subsampling ratio). The outer edge detection provides the board s angle and both horizontal and vertical offsets within an image. Then, it is possible to establish a region of interest (ROI) in which setting a coordinate system to correct rotation, displacement and scaling. These tasks involve the transformation of images pixel information; so a later reconstruction process based on bilinear interpolation is applied to reference images. This process is performed in two directions, thereby achieving more reliable images regarding original images. B. Detection Algorithm Third stage consists of two main processes: particle analysis for vias detection, and substraction operation for defective particles detection (see Table II). Firstly, a smoothing filter based on the Kernel family is applied to images. This consists of an averaging linear filter with a 3x3 convolution matrix for which images have a minimum border size of 1 pixel [15]. To this end, a relatively good compromise between the coefficients corresponding to the kernel size and the processing speed has been set to achieve optimal results. Secondly, a 3x3 erosion filter is applied to images in order to eliminate spurious particles due to imperfections in the etching process of PCBs. In this case, the algorithm for spurious particle detection is performed in connectivity mode 8 and treats the pixel frame as hexagonal during the transformation. On the one hand, vias on PCBs are detected by means of a particle analysis. This consists of a process that separates overlapping circular particles and classifies them based on their radius, surface area, and perimeter. To this end, a Danielsson algorithm based on the Euclidian distance map is used to determine the radius of each particle [16]. As a result, this analysis returns the number of detected circles in the image and an array of measurements (x and y positions, radius, and core area). On the other hand, the image comparison method by reference implemented in this work carries out a triple subtraction operation (see Fig. 3). This way, the range of a binary image is extended from (0, 255) to (-255, 510). These simple logical operations represent an improved approach of the basic reference comparison method. As an advantage, higher resolution is obtained in the subsequent particle detection process and thus the resulting objects can be classified into two categories: excess and missing particles (see Table II). 4 Overtech Pad Missing 5 Spur Excess 6 Mouse Bite Missing 7 Scratch Missing 8 Open Track Missing 9 Open Pad Missing 10 Undertech Pad Excess 11 Missing Pad Missing 12 Spurious Copper Excess C. Classification Algorithm The fourth stage involves the classification process of the detected particles. In this research we have studied and implemented various classification algorithms. Figure 4a shows a first implemented method based on point intensity measurements [17]. The operation consists in obtaining the pixel s average intensity in the upper left and lower right corners of the defective particle. Despite being a simple method, it provides limited results since it is capable to distinguish only three types of particles (holes, open, and mouse bites). In Figure 4b, the BST method mentioned in the Introduction section is shown [7]. This method traces an outer border around the detected particle, covers the pixels in counterclockwise direction (t, t +1, t +2, t +n ), and counts the number of transitions between background (0) and copper tracks (255). Despite BST method produces better results compared to the point intensity measurement method, it is capable to classify less number of defects than the following proposed method. Figure 3. Subtraction operations for particle analysis 153

T. J. M. Sanguino and M. Smolčić-Rodriguez Computer-aided System for Defect Inspection TABLE III. TYPE OF DEFECTS ON THE PCB LAYOUT The classification method finally implemented in this work consists in combining two strategies: measuring the intensity statistics of pixels within a rectangular region and along lines around a particle (see Fig. 4c-4d). On the one hand, a rectangle is traced internally to the particle; then average and maximum light intensities within the inner area are calculated. As an advantage, this allows distinguishing between different type of particles such as missing holes, pinholes, missing pads or spurious copper. Nevertheless, for other particles more difficult to detect due to their complex morphology or vicinity a method based on boundary lines is also used. The process consists in tracing four outer lines around the particle in clockwise direction (left, top, right and bottom) and obtaining the average light intensity of each line. To this end, the light intensity of defective particles must be previously parameterized and then the intensity thresholds should be established. By setting thresholds, the algorithm is capable to differentiate if border lines belong to copper tracks or background. As an advantage, the combination of both strategies area and border light meters allows classifying a total of twelve types of particles as shown in Table III. On the contrary, the algorithms implemented in [7] and [18] are able of automatically inspecting eight and ten type of defects in the etching process. In particular, these works include open, mouse bite, pinhole, missing conductor, short, spur, excess copper and missing holes. Finally, once the defective particles have been recognized, a labeling process is performed to identify particles individually, whose texts are overlaid on the PCB image. III. (a) (c) SYSTEM IMPLEMENTATION The Scanweiter system has been developed in LabVIEW TM 2010 SP1 and its GUI is shown in Figure 5. Different work areas of the application are available to users in the upper left area (a). These tabs correspond to the main stages of the AOI system described in Figure 1 and have been renamed as setup, rotation, offset, (b) (d) Figure 4. Object classification methods: point meter (a), boundary state transition (b), rectangle meter (c), and line meter (d) # Defect 1 Missing Hole Board Vicinity inspection and classification. The outcome of a PCB inspection is shown in the upper tab of the figure. An index corresponding to each defective particle is shown in the left part of the picture. Accordingly, the particles classified by the type of defect are displayed by means of a histogram (b). The processing time corresponding to each stage of the AOI system is also shown by means of a bar graph in the lower tab. A diagnosis table with the defective particles is available at the bottom of the figure (c). The table shows useful information to users in order to identify the particles on the PCB. This consists of the index, coordinates (x, y), magnitude of the defect (error or warning), and classification by type. Finally, different detailed reports are accessible to users through several tabs on the left side of the application (d). These reports provide analysis of the area, orientation, and both mass center and light intensity of detected particles. IV. Track Vicinity Typ. Light Intensity EXPERIMENTATION Light Intensity Threshold 0 4 211 Mean Area 350 2 Pinhole 0 4 510 Mean Area 350 3 Short 0 2-210 4 Overtech Pad 0 2-25 2 Borders Mean -150 2 Borders Mean -150 5 Spur 1 1-182 Border Mean -100 6 Mouse Border 1 1-42 Bite Mean -100 7 Scratch 1 2 8 Open Track 2 9 Open Pad 3 213 Mean Area 100 10 Undertech Pad 3-20 Mean Area 100 11 Missing Pad 4 510 Max. Area 255 12 Excess Copper 4 255 Max. Area 255 The automatic optical inspection system has been tested using a PCB image and a CAD image of 1188 x 1798 pixel resolution (see Fig. 6). A total of 77 drills have been detected after the particle analysis. Furthermore, 16 defective particles have been detected by the subtraction method, which are classified in 5 errors and 11 warnings (see Table IV). As a result, the bare PCB achieves a structural similarity of 99.80% due to the defective objects compared to the CAD image. The Scanweiter system obtained the following average times regarding the algorithms involved in the overall process (t setup = 899 ms, t rotation = 149 ms, t displacement = 69 ms, t inspection = 456 ms and t classification = 95 ms). In order to assess the performance of the Scanweiter system, the required time for detecting defective particles has been compared to some of the methods mentioned in the Introduction section (see Table V). The processing time 154

INES 2012 IEEE 16th International Conference on Intelligent Engineering Systems June 13 15, 2012, Lisbon, Portugal Figure 5. Graphical user interface of the AOI system (a) (c) varies mainly due to the CPU system and the image size, which are included in the table for comparison purposes. The performance of the several inspection methods described in [7] has also been compared with Scanweiter (see Fig. 7). The time required in the algorithms implemented in this work is the highest (Scanweiter 1). Nevertheless, authors propose to reduce the processing time without compromise the detection and classification (b) (d) Figure 6. Outcome of the AOI test: bare PCB before alignment correction (a), via inspection after particle analysis (b), image after subtraction operation (c), and image after classification process (d) of defective particles (Scanweiter 2). In this concern, the vias detection algorithm is the method that most influence on the overall system performance. However, it only provides to users information on via positions since it is not involved in the detection and classification process of defective particles. In the other hand, the algorithms shown in figure 7 region merged, boundary state transition and projection methods depend largely on the number of detected defects. On the contrary, the algorithm proposed in this work presents a more stable trend line, which remains invariant with the number of existing defects. Finally, methods with a model-based approach are quite costly in computational terms compared to this proposed work [19]. The reason is because the implementation of pattern matching techniques is quite complex. On the other hand, methods based on DRC also present high processing times [20]. Although they are simple to implement, the algorithm works directly with the image and requires the verification of track widths, pads and insulating areas of fiberglass resin. V. CONCLUSIONS This paper presents a CAE system devoted to the visual recognition of defects in the PCB etching process. This work proposes different detection and classification algorithms mainly supported by two complementary methods. On the one hand, the conventional reference comparison method has been improved, which achieves higher resolution to detect particles by means of three logical operations. On the other hand, a classification method of defective particles based on a twofold strategy has been implemented. As an advantage, the system 155

T. J. M. Sanguino and M. Smolčić-Rodriguez Computer-aided System for Defect Inspection TABLE IV. MEASUREMENT OF DEFECTIVE OBJECTS ON A PCB LAYOUT # X Y Area Orien- Defect Magnitude Position Position (pixels) tation 1 190 234 Pinhole Warning 25 0º 2 309 249 Excess Warning 58 71.45º 3 189 252 Pinhole Warning 25 0º 4 235 267 Hole Warning 53 51.37º 5 495 270 Bite Warning 29 90º 6 364 311 Short Error 106 51.45º 7 416 319 Undertech Warning 82 171.01º 8 227 324 Spur Warning 123 89.62º 9 417 371 Pad Error 206 73.49º 10 237 374 Open Error 101 9.92º 11 331 376 Bite Warning 22 73.53º 12 144 395 Overtech Warning 41 106.06º 13 487 475 Missing Error 206 169.08º 14 325 495 Scratch Warning 28 61.85º 15 325 504 Scratch Warning 28 55.92º 16 420 549 Open Error 62 177.79º allows classifying a total of twelve types of defects through statistical techniques of light intensity measurements inside and outside the particles. This includes missing holes, pinholes, shorts, overtechs, undertechs, spurs, mouse bites, scratchs, open tracks, open pads, missing pads, and spurious copper. Finally, the experimentation methodology and feasibility of the vision machine algorithms are presented. With this aim, the proposed methods have been analytically compared and their performances have been studied. ACKNOWLEDGMENT We are grateful to the Department of Electronic Engineering, Computer Systems and Automatics (UHU) for its collaboration and providing its laboratory for manufacturing the PCBs. REFERENCES [1] M. Moganti, F. Ercal, C.H. Dagli, and S. Tsunekawa, Automatic PCB inspection algorithms: a survey. Computer Vision and Image Understanding, Vol. 63, No. 2, pp. 287-313, 1996. [2] Y. Choi, and K. Chung, "An Efficient Model Generation for Model-based PCB Pattern Inspection", Journal of The Korea Information Science Society, Vol. 24, No. 7, July 1997. [3] A.R. Hidde, and A. Gierse, An AI-based manufacturing design rule checker and path optimizer for PCB production preparation and manufacturing. IEEE Transactions on Components, Hybrids, and Manufacturing Technology, Vol. 15, No. 3, pp. 299-305, 1992. [4] A. Fanni, M. Lera, E. Marongiu, and A. Montisci, Neural Network Diagnosis for Visual Inspection in Printed Circuit Boards. 12 th International Workshop on Principles of Diagnosis, pp. 47-54, 2001. [5] Z. Ibrahim, and S.A.R. Al-Attas, Wavelet-Based Printed Circuit Board Inspection System. International Journal of Information and Communication Engineering, Vol. 1, No. 2, pp. 73-79, 2005. [6] F.R. Leta, F.F. Feliciano, and F.P.R. Martins, Computer Vision System for Printed Circuit Board Inspection. ABCM Symposium Series in Mechatronics, Vol. 3, pp.623-632, 2008. [7] H. Rau, and C.H. Wu, Automatic optical inspection for detecting defects on printed circuit board inner layers. International Journal of Advanced Manufacturing Technology, Vol. 25, pp. 940 946, 2005. [8] N.K. Khalid, Z. Ibrahim, M. Shukri, and Z. Abidin, An Algorithm to Group Defects on Printed Circuit Board for Automated Visual Inspection. International Journal of TABLE V. COMPUTING TIME COMPARISON FOR SEVERAL AOI SYSTEMS System/Author Processing Image Size Time (pixels) CPU System Rau et al. 350 ms 1240 x 720 Pentium II Ibrahim et al. 1852 ms 1K x 1K Pentium III (800 Mhz) Leta et al. High 1.8 M - Scanweiter 1668 ms 1188 x 1798 Pentium 4 (3.06 Ghz, 1.5 Gb RAM) SVP500 15 s 17.5K x Intel Core i5-2400 17.5K (3.1 Ghz, 4 Gb RAM) YC-MC726 60 s 768 574 Intel Core 2 Quad (2.5 Ghz, 2G RAM) Figure 7. Computing time comparison for several AOI methods Simulation, Systems, Science and Technology, Vol. 9, No. 2, pp. 1-10, 2008. [9] A.P. Singh Chauhan, and S. Chandra Bhardwaj, Detection of Bare PCB Defects by Image Subtraction Method using Machine Vision. Proc. World Congress on Engineering, Vol. II, pp. 1597-1601, 2011. [10] J.M. Sebastián, D. García, and F.M. Sánchez, Remote Access Education Based on Image Acquisition and Processing through the Internet. IEEE Transactions on Education, Vol. 46, pp. 142-148, 2003. [11] H.W. Oh, J.H. Jung, and T.H. Park, Gerber-character recognition system of auto-teaching program for PCB assembly machines. SICE 2004 Annual Conference. Sapporo, Japan, Vol. 1, pp. 300-305, 2004. [12] G. Müller, The Inline solution for full automatic operation. Technical Report, 2010. [Online] Available: http://www.m3electronic.de/v3/datnes/ocw/en/opticheck_conveyor_eng.pdf [13] S. Ertürk, Digital Image Processing. National Instruments, Austin, USA, 2003. [14] T. Klinger, Image Processing with LabVIEW and IMAQ Vision. Prentice Hall PTR, New Jersey, USA, 2003. [15] R. Posada-Gómez et al., Practical Applications and Solutions Using LabVIEW Software. InTech Publishers, Rijeka, Croatia, 2011. [16] C.G. Relf, Image Acquisition and Processing with LabVIEW TM. CRC Press LLC, Florida, USA, 2007. [17] M. Smolčić-Rodríguez, Automatische Inspektion von Circuits. Master Thesis, Hochschule Darmstadt, Germany, 2010. [18] H. Rau, C.H. Wu, W.J. Shiang, and P.T. Huang, A decision support system of statistical process control for printed circuit boards manufacturing. Proc. Ninth International Conference on Machine Learning and Cybernetics, Qingdao, China, pp. 2454-2458, 2010. [19] Y.N. Sun, and C.T. Tsai, New Model-Based Approach for Indusrial Visual Inspection. Pattern Recognition, Vol. 25, pp.1327-1336, 1992. [20] J.J. Hong, K.J. Park, and K.G. Kim, Parallel processing machine vision system for bare PCB inspection. Proc. 24th Annual Conference of the IEEE Industrial Electronics Society. Vol 3, pp. 1346-1350, 1998. 156