Solder paste inspection and 3D shape estimation using directional LED lightings

Transcription

1 Ttle Solder paste nspecton and 3D shape estmaton usng drectonal LED lghtngs Advsor(s Pang, GKH Author(s Chu, Mng-he.; 朱 銘 熙. Ctaton Issued Date 27 UL ghts The author retans all propretary rghts, (such as patent rghts and the rght to use n future orks.

2 Abstract of thess enttled Solder Paste Inspecton and 3D Shape Estmaton Usng Drectonal LED Lghtngs Submtted by CHU Mng He for the degree of Master of Phlosophy at The Unversty of Hong Kong n Dec 27 In ths thess, a novel methodology for solder paste nspecton and 3D shape estmaton usng drectonal LED lghtngs s presented. The proposed methodology s based on a specally desgned drectonal LED lghtng to hghlght the geometrcal features of the solder paste block. Solder paste nspecton and shape estmaton are then carred out based on the hghlghted features. The am of ths research s to develop a methodology to nspect solder paste blocks and estmate ther shapes. To acheve these objectves, a specal drectonal LED lghtng s desgned to hghlght the geometrcal features of the blocks. Under the desgned lghtng, solder paste mages th hghlghted features are taken by a dgtal camera. The captured mages are then processed and used to nspect the prntng qualty and estmate the shape of the solder paste blocks. To carry out qualty nspecton, the processed solder paste mages are passed to a fuzzy system for Solder Paste Qualty Scorng. Frst, a number of features are extracted as the nputs for the fuzzy system. The qualty scores of the solder paste blocks are then calculated by the fuzzy system based on a set of fuzzy rules. Neural-fuzzy tranng can be appled to the proposed fuzzy system for fne-tunng the behavor accordng to the users preference. I

3 On the other hand, the processed solder pastes mages are gven to a Shape Estmaton System for estmatng the the shape of the solder paste. The Shape Estmaton System conssts of to parts: shape estmaton algorthm and parameter optmzaton. The shape estmaton algorthm uses a set of parameters to estmate the shape. The set of parameters ncludes the processed solder paste mages and the physcal propertes of the solder paste block, such as slope of edges, etc.. The estmaton process s then carred out by calculatng a set of surface heghts for the solder paste block. The estmated shape of the solder paste block s represented by the calculated set of surface heghts. To mprove the accuracy of the estmaton, parameter optmzaton can be carred out to search for an optmal set of algorthm parameters usng the actual solder paste shape. Expermental results sho that the proposed methodology gves hgh accuracy n the solder paste nspecton. The estmated shape from the 3D shape estmaton system s acceptable. In the evaluaton of the fuzzy system for Solder Paste Qualty Scorng, the proposed system s tested th dfferent types of solder paste blocks. esults sho that the fuzzy system gves hgh nspecton accuracy toards dfferent types of solder paste blocks. In addton, experments sho that the neural-fuzzy tranng can adjust the fuzzy system accordng to the users preference. For the Shape Estmaton System, the proposed algorthm can gve acceptable estmated shape toards dfferent types of solder paste block. The evaluaton results also sho that the parameter optmzaton can fnd sets of optmal algorthm parameters hch can gve estmated shape th hgher accuracy. II

4 Solder Paste Inspecton and 3D Shape Estmaton Usng Drectonal LED Lghtngs by CHU Mng He B. Eng. ( CE., The Unversty of Hong Kong, 25 A thess submtted n partal fulfllment of the requrements for the Degree of Master of Phlosophy at The Unversty of Hong Kong December 27 III

5 Declaraton I declare that ths thess represents my on ork, except here due acknoledgement s made, and that t has not been prevously ncluded n a thess, dssertaton or report submtted to ths Unversty or to any other nsttuton for a degree, dploma or other qualfcaton. Sgned CHU Mng He I

6 Acknoledgements I ould lke to express my greatest grattude to my supervsor, Dr. G. K. H. Pang, for hs supervson and encouragement on my research at the Industral Automaton esearch Laboratory of The Unversty of Hong Kong. Hs extensve knoledge n fuzzy logc, neural netork and computer vson has greatly nspred my research. He alays provdes me th valuable suggestons, deas and advces. Hs close supervson makes me have frequent reves of research progress and have a clear objectve throughout the research perod. I ould also lke to thank Dr. Davd Lam of HKUST for helpful dscusson and use of the Veeco 3D Profler. Next, I ould lke to thank all my colleagues n Industral Automaton esearch Laboratory, especally Mr. Thomas T. O. Kan and Mr. Henry Y. T. Ngan for ther advces and supports durng my research. Besdes, I ould lke to acknoledge the fnancal supports from my school, The Unversty of Hong Kong, ncludng the Postgraduate Studentshp and conference grant durng my MPhl. study. Fnally, I ould lke to thank my parents and my grl frend, May, for ther contnuous supports and encouragements n my postgraduate study. II

7 Contents Declaraton...I Acknoledgements...II Contents... III Lst of Fgures... VIII Lst of Tables... XIII Lst of Publcaton...XV Chapter Introducton.... Background....2 Motvaton Goals and Objectves Contrbuton of ths Thess Outlne of ths Thess...9 Chapter 2 Lterature eves Introducton Laser trangulaton Structural Lghtng Phase Proflometry...4 III

8 2.3.2 Bnary Pattern Projecton Neural Netork Three-dmensonal econstructon by Neural Netork Neural Netork and Fuzzy ule-based Classfcaton Shape-from-shadng (SFS Sgnfcance of the esearch...2 Chapter 3 Methodology Introducton Image Acquston and Processng Fuzzy System for Solder Paste Qualty Scorng Shape Estmaton System Summary...26 Chapter 4 Image Acquston and Processng Image Acquston Introducton Lghtng Desgn Expermental Setup Expermental esults Image Processng Introducton Sde Lghtng Images Top Lghtng Images Summary...39 IV

9 Chapter 5 Fuzzy System for Solder Paste Qualty Scorng Introducton Motvaton and Feature Extracton for Fuzzy System Normalzaton of the Extracted Features Fuzzfcaton of the Extracted Features ules for the Fuzzy System Defuzzfcaton Fuzzy-Neural Adaptaton for the Fuzzy System Fuzzy System Setup Procedures Evaluaton Square-shaped Solder Paste Block Expermental Setup and Procedures Experment Experment Experment Experment Experment ectangular Solder Paste Block Expermental Setup and Procedures Expermental esults Crcular Solder Paste Block Expermental Setup and Procedures Expermental esults Dscusson Summary...2 V

10 Chapter 6 Shape Estmaton System Introducton Descrpton of the algorthm Algorthm Parameters Experments eference Shape Square-shaped Solder Paste Block Generalzed Parameters Experment Experment Experment Experment ectangular Solder Paste Block Experment Experment Crcular Solder Paste Block Experment Experment Dscusson Summary...46 Chapter 7 Conclusons Introducton Image Acquston and Processng Fuzzy System for Solder Paste Qualty Scorng Shape Estmaton System...49 VI

11 7.5 Future esearch...5 eferences VII

12 Lst of Fgures Fgure. Fgure.2 Example of prnted crcut board (PCB before surface mount technology process (SMT Example of prnted crcut board (PCB after surface mount technology process (SMT 2 Fgure.3 Example of surface mount devce (SMD chps 2 Fgure.4 Components of through-hole technology 2 Fgure.5 Flo chart of surface mount technology process 3 Fgure.6 Fgure.7 Common solder paste defects - (a Brdgng, (b Excess Solder and (c Insuffcent Solder Flo chart of surface mount technology process th solder paste nspecton 4 5 Fgure 2. Geometry of laser trangulaton method 2 Fgure 3. Block dagram of the proposed methodology 2 Fgure 3.2 Block dagram for the proposed system setup 23 Fgure 4. Sde ve of the solder paste block under LED llumnaton 28 Fgure 4.2 The top ve of the solder paste block under to dfferent llumnatons 28 Fgure 4.3 Arrangement of LEDs n a straght LED lghtng plate 29 Fgure 4.4 Arrangement of LEDs n a rng LED lghtng plate 3 Fgure 4.5 Crcut board for controllng LED brghtness 3 Fgure 4.6 Crcut dagram for controllng LED brghtness 3 Fgure 4.7 Dagram of the expermental Setup 32 Fgure 4.8 Photo of the expermental Setup 33 Fgure 4.9 LEDs arrangement n sde lghtng source 33 Fgure 4. Images of a solder paste block thout defect under (a east-est lghtng, (b north-south lghtng and (c top lghtng 35 VIII

13 Fgure 4. Fgure 4.2 Images of a solder paste block th defect of nsuffcent solder under (a east-est lghtng, (b north-south lghtng and (c top lghtng Images of a solder paste block th defect of excess solder under (a east-est lghtng, (b north-south lghtng and (c top lghtng Fgure 4.3 Typcal extracted color clusters from solder paste blocks 38 Fgure 4.4 Example of extracted solder paste mask 39 Fgure 5. Block Dagram of Fuzzy System for Solder Paste Qualty Scorng 4 Fgure 5.2 Processed mages of a defectve solder paste block 42 Fgure 5.3 Processed mages of a non-defectve solder paste block 42 Fgure 5.4 Typcal shape of membershp functon f 48 Fgure 5.5 Typcal shape of membershp functon f 2 49 Fgure 5.6 Structure of the neural-fuzzy netork 58 Fgure 5.7 Intal Membershp Functons of the Fuzzy System 68 Fgure 5.8 Score dstrbuton of solder paste samples th ntal membershp functon 73 Fgure 5.9 Membershp functons of the fuzzy system after tranng 77 Fgure 5. Score Dstrbuton of Solder Paste Samples After Tranng 77 Fgure 5. Fgure 5.2 Fgure 5.3 Fgure 5.4 Membershp functons of the fuzzy system after tranng n experment 3 Score dstrbuton of solder paste samples after tranng n experment 3 Membershp functons of the fuzzy system after tranng n experment 4 Score dstrbuton of solder paste samples after tranng n experment Fgure 5.5 Score dstrbuton graph of testng solder paste samples 88 Fgure 5.6 (a Typcal rectangular solder paste blocks and (b quad flat package chp 9 IX

14 Fgure 5.7 Appearance of (a non-defectve and (b defectve rectangular solder paste block under proposed drectonal lghtng 9 Fgure 5.8 Membershp functons th ntal parameters from Table.4 94 Fgure 5.9 Membershp functons after tranng 95 Fgure 5.2 Score dstrbuton graph of testng solder paste samples 96 Fgure 5.2 Typcal crcular pads 97 Fgure 5.22 Typcal crcular solder paste block 97 Fgure 5.23 Membershp functons for crcular solder paste block Fgure 5.24 Score dstrbuton graph for testng crcular solder paste block Fgure 6. Examples of processed solder paste mages 6 Fgure 6.2 Inclned regons of solder paste blocks based on processed mage n Fg Fgure 6.3 Example postons of dfferent regons 8 Fgure 6.4 Example of non-defectve solder paste block Fgure 6.5 Example of defectve solder paste block th nsuffcent solder Fgure 6.6 Example of defectve solder paste block th excess solder Fgure 6.7 Block dagram of the shape estmaton algorthm 2 Fgure 6.8 Block dagram for the parameter optmzaton process 3 Fgure 6.9 Expermental setup for acqurng reference solder paste shape 6 Fgure 6. Fgure 6. Fgure 6.2 Fgure 6.3 Fgure 6.4 The shape of a 2m crcular solder paste block acqured by the laser sensor The actual shape of the testng 2m square-shaped solder paste block acqured by the laser sensor (a Orgnal and (b processed mages of the testng 2m squareshaped solder paste block The estmated shape of the testng 2m square-shaped solder paste block usng generalzed non-defectve algorthm parameters The estmated shape of the testng 2m square-shaped solder paste block usng optmzed algorthm parameters X

15 Fgure 6.5 Fgure 6.6 Fgure 6.7 Fgure 6.8 Fgure 6.9 Fgure 6.2 Fgure 6.2 Fgure 6.22 Fgure 6.23 The actual shape of the testng 2m square-shaped solder paste th defect of excess solder (a Orgnal and (b processed mages of the testng 2m squareshaped solder paste block th defect of excess solder The estmated shape of the testng 2m square-shaped solder paste block th defect of excess solder usng generalzed defectve algorthm parameters The estmated shape of the testng 2m square-shaped solder paste block th defect of excess solder usng optmzed algorthm parameters The actual shape of the testng 2m square-shaped solder paste th defect of nsuffcent solder (a Orgnal and (b processed mages of the testng 2m squareshaped solder paste th defect of nsuffcent solder The estmated shape of the testng 2m square-shaped solder paste th defect of nsuffcent solder usng generalzed defectve algorthm parameters The estmated shape of the testng 2m square-shaped solder paste th defect of nsuffcent solder usng optmzed algorthm parameters Appearance of rectangular solder paste block under the proposed lghtng Fgure 6.24 Actual shape of the testng 5m rectangular solder paste block 3 Fgure 6.25 Fgure 6.26 Fgure 6.27 (a Orgnal and (b processed mages of the testng 5m nondefectve rectangular solder paste block The estmated shape of the testng 5m non-defectve rectangular solder paste block usng generalzed non-defectve algorthm parameters The estmated shape of the testng 5m non-defectve rectangular solder paste block usng optmzed algorthm parameters Fgure 6.28 Actual shape of the 5m defectve rectangular solder paste block 33 Fgure 6.29 (a Orgnal and (b processed mages of the testng 5m defectve rectangular solder paste block 34 XI

16 Fgure 6.3 Fgure 6.3 Fgure 6.32 The estmated shape of the testng 5m defectve rectangular solder paste block usng parameters gven n Table 6.9 The estmated shape of the testng 5m defectve rectangular solder paste block usng optmzed parameters gven n Table 6.9 Appearance of crcular solder paste block under the proposed lghtng Fgure 6.33 Actual shape of the 5m non-defectve crcular solder paste block 38 Fgure 6.34 Fgure 6.35 Fgure 6.36 (a Orgnal and (b processed mages of the testng 5m nondefectve crcular solder paste block The estmated shape of the testng 5m non-defectve crcular solder paste block usng parameters gven n Table 6. The estmated shape of the testng 5m non-defectve crcular solder paste block usng optmzed algorthm parameters Fgure 6.37 Actual shape of the 5m defectve crcular solder paste block 4 Fgure 6.38 (a Orgnal and (b processed mages of the testng 5m defectve crcular solder paste block 42 Fgure 6.39 Fgure 6.4 The estmated shape of the testng 5m defectve crcular solder paste block usng algorthm parameters n Table 6.3 The estmated shape of the testng 5m defectve crcular solder paste block usng optmzed algorthm parameters XII

17 Lst of Tables Table 5. Feature scalng equatons for 5m square-shaped solder paste block 47 Table 5.2 Membershp Functons of Lngustc Terms 5-5 Table 5.3 Intal parameters of membershp functons for 5m solder paste block 67 Table 5.4 Solder Paste Block Samples 69-7 Table 5.5 Qualty score of solder paste block samples th ntal membershp functon parameters 72 Table 5.6 Tranng Solder Paste Block Samples of Experment 2 74 Table 5.7 Qualty Score of Solder Paste Block Samples After Tranng 76 Table 5.8 ule eghtngs for the fuzzy system n experment 3 78 Table 5.9 Tranng Solder Paste Block Samples of Experment 3 79 Table 5. Qualty Score of Solder Paste Block Samples After Tranng 8 Table 5. Tranng Solder Paste Block Samples of Experment 4 83 Table 5.2 Qualty Score of Solder Paste Block Samples After Tranng 84 Table 5.3 Table 5.4 Table 5.5 Qualty Score of Testng Solder Paste Block Samples Usng Fuzzy System n Experment 4 Scalng equatons of extracted features for rectangular solder paste block Intal parameters of membershp functons for rectangular solder paste block Table 5.6 ule eghtngs for nspectng rectangular solder paste block 94 Table 5.7 Scalng functons for crcular solder paste block Table 5.8 Intal membershp functons parameters for crcular solder paste block 99 Table 6. Specfcaton of the laser dsplacement sensor 5 Table 6.2 Algorthm parameters for square-shaped solder paste blocks 8 XIII

18 Table 6.3 Optmzed algorthm parameters for testng solder paste block 2 Table 6.4 Table 6.5 Optmzed algorthm parameters for testng solder paste block th defect of excess solder Optmzed algorthm parameters for testng solder paste block th defect of nsuffcent solder Table 6.6 Expermental results of experment 4 28 Table 6.7 Generalzed non-defectve algorthm parameters for 5m rectangular solder paste block 3 Table 6.8 Optmzed algorthm parameters for 5m rectangular solder paste block 32 Table 6.9 Defectve algorthm parameters for the defectve 5m rectangular solder paste block 34 Table 6. Table 6. Optmzed defectve algorthm parameters for the defectve 5m rectangular solder paste block Algorthm parameters for the non-defectve 5m crcular solder paste block Table 6.2 Optmzed algorthm parameters for the non-defectve 5m crcular solder paste block 4 Table 6.3 Table 6.4 Algorthm parameters for the defectve 5m crcular solder paste block Optmzed algorthm parameters for the defectve 5m crcular solder paste block Table 6.5 Summary of average estmaton error of the experments 46 XIV

19 Lst of Publcaton M. H. Chu and G. K. H. Pang, Solder Paste Inspecton By Specal LED Lghtng For SMT Manufacturng of Prnted Crcut Boards, 27 IFAC Workshop on Intellgent Manufacturng Systems, pp , May 27. XV

20 CHAPTE Introducton. Background Noadays, surface mount technology (SMT s used extensvely n the producton of prnted crcut boards (PCB (Fg.. & Fg..2. In ths producton process, electronc components, hch called surface-mount devces (SMDs (Fg..3, are drectly mounted on the surface of boards. The old through-hole technology (Fg..4, hch mounts components th res leads nto holes n the crcut boards, has been replaced by SMT. Usng SMT, components of smaller sze can be packed denser on the crcut boards. Fg.. Example of prnted crcut board (PCB before surface mount technology process (SMT

21 Fg..2 Example of prnted crcut board (PCB after surface mount technology process (SMT Fg..3 Example of surface mount devce (SMD chps Fg..4 Components of through-hole technology 2

22 PCB contans lots of solder pads on ts surface and the leads of SMDs are mounted onto the board through the pads. The jonts beteen the lead and the pad are connected by a stcky mxture of solder partcles and flux called solder paste. Before placng the SMDs onto the board, solder paste s frst prnted onto the solder pads of the PCB through a stanless steel stencl. Next, SMDs are placed onto the PCB th ts solder pads prnted th solder paste. Fnally, the PCB th SMDs s placed nsde a reflo solderng oven. In ths reflo process, all the solder partcles n the solder paste are melted and bond the component leads to the pads of the crcut board. The flo chart of the SMT process s shon n Fg..5. Solder Paste Prntng SMDs Placement eflo Fg..5 Flo chart of surface mount technology process It s ell knon that almost 52% - 7% of SMT defects can be traced to poor solder paste prntng [-4]. Examples of such defects (Fg..6 nclude brdge, bad jonts/connecton after reflo due to nsuffcent solder, off regstraton and excess solder. In addton, the volume of solder paste s also a key parameter that affects the qualty of solder jont. Lo solder paste volumes can produce solder jonts that pass electrcal test but have lo mechancal strength and hgh falure rates. 3

23 (a (b (c Source: Step By Step SMT Solderng: educng Solderng Defects Loctte Techncal Paper, Aug., 2 Fg..6 Common solder paste defects - (a Brdgng, (b Excess Solder and (c Insuffcent Solder The prntng qualty of solder paste can be reflected by ts appearance and shape. For example, the locaton, heght, and volume of the solder paste on pads ould contrbute to the appearance of defects n the subsequent process. Hence, the qualty of the solder paste block prnted on PCBs s hghly related to the qualty of the solder jont. In order to ncrease the qualty and yeld n the SMT producton, solder paste nspecton s essental. Ths ll ensure that the defects are detected at an earler stage of the process, here the reork cost can be mnmzed. The flo chart of the SMT process th solder paste nspecton s shon n Fg..7. 4

24 Solder Paste Prntng Solder Paste Inspecton SMDs Placement eflo Fg..7 Flo chart of surface mount technology process th solder paste nspecton The prnted solder paste has a nomnal thckness of 5um to 2um. Its small sze makes vsual nspecton dffcult by naked eyes. Also, nspecton on the solder paste should be done thn 4 seconds so that the producton rate of the SMT lne s not sloed don. Therefore, automated nspecton method s needed to do the job. Among all the avalable nspecton methods, solder paste nspecton can be categorzed nto to-dmensonal and three-dmensonal nspectons. In to-dmensonal solder paste nspecton, top-ve mages are requred and defects such as paste-to-pad offset, solder past smear, solder paste absence and excessve/underszed solder paste area can be effectvely determned [5-8]. Hoever, solder slump and nsuffcent/excessve solder paste volume cannot be determned usng ths type of nspecton as such defects need three-dmensonal measurement of surface profles and the volume of the solder paste. As the SMT components on the PCB board become denser, 3D nspecton become more and more mportant for qualty assurance [4, 9, ]. Three-dmensonal measurement of solder paste block not only can detect 3D defects hch cannot be acheved by to-dmensonal nspecton, but also can mprove the solder 5

25 paste prntng process by feedng back the statstcal nformaton of the solder paste volume to the stencl prnter [2, 23]. Although three-dmensonal solder paste nspecton can solve the problem of todmensonal nspecton, the nspecton speed s too slo for performng full-board nspecton on every PCB board manufactured []. In addton, the equpment cost for three-dmensonal nspecton s much hgher hen compared to to-dmensonal nspecton..2 Motvaton In the SMT producton process, nspecton at the stage of solder paste prntng can greatly ncrease the yeld and reduce producton cost. Varous studes from dfferent unverstes and companes have reported that 52% - 7% of SMT defects are related to the solder paste prntng process [-4]. Hoever, solder paste nspecton s usually not performed n the current SMT producton lne. There are several reasons for ths:. Current solder paste nspecton method s not cost-effectve due to hgh machnery cost. The solder paste nspecton machne usually nvolves the usage of laser measurement sensors [2] and some optcal nstruments []. These types of apparatus greatly ncrease the cost of solder paste nspecton hch, n turns, nhbts the nvestment of the nspecton machnery n the SMT producton lne. 2. To-dmensonal nspecton on solder paste cannot detect all the defects durng prntng. For example, solder slump and nsuffcent/excessve solder paste volume 6

26 cannot be determned usng ths type of nspecton as such defects need threedmensonal measurement of surface profles and the volume of the solder paste. Therefore, three-dmensonal nspecton s needed n order to ncrease the nspecton accuracy. 3. The slo nspecton speed of three-dmensonal nspecton makes t mpossble to perform full PCB nspecton thout any sgnfcant delay n the producton lne. Even though some commercal systems clam on hgh system throughput, the nspecton speed cannot meet the demandng requrement of an n-lne process hch s around 4 seconds per PCB producton. Therefore, only partal nspecton can be performed on the PCB durng the n-lne process and full board nspecton can only performed n several selected samples. As a result, many defects may be left undetected and the yeld s affected..3 Goals and Objectves The goal of ths research s to develop a fast and cost-effectve solder paste nspecton method based on only color mages. The proposed method makes use of a doubly-colored drectonal LED lghtng structure. The qualty of the solder paste can be determned from the processed mages of the solder paste under the LED llumnaton usng a set of fuzzy rules. Also, 3D physcal features of the solder paste can be extracted from 2D mages. The shape of the solder paste can then be estmated based on the LED lght drecton and the extracted physcal features. The proposed method should be developed and evaluated usng expermental results. The objectves of ths research can be summarzed as follos: 7

27 . To desgn the specal doubly-colored drectonal LED lghtng structure to hghlght the geometrcal features of the solder paste block. 2. To desgn and mplement a fuzzy system to judge the qualty of the solder paste block based on the geometrcal features hghlghted by the drectonal LED llumnaton. 3. To desgn and mplement a solder paste shape estmaton algorthm to estmate the shape of the solder paste block based on the geometrcal features hghlghted by the drectonal LED llumnaton..4 Contrbuton of ths Thess Ths thess presents a lterature reve of current solder paste nspecton method ncludng laser trangulaton, structural lghtng, neural netork and shape-from-shadng. Then a novel nspecton method usng fuzzy system and drectonal LED lghtng s presented and evaluated. Another contrbuton of ths thess s the development of a solder paste 3D shape estmaton method. The novel method can provde an estmated 3D shape of the solder paste block under nspecton, no matter t s defectve or nondefectve. The objectves lsted n Secton.3 can be fulflled n ths thess. 8

28 .5 Outlne of ths Thess Ths thess conssts of seven chapters. A bref summary of the follong chapters s as follos: Chapter 2 Ths chapter contans a reve of current methods used n solder paste nspecton. The dscusson s based on three man methods: laser trangulaton, structural lghtng, neural netork and shape-from-shadng. Clear dscusson on the prncple of each method s gven. Then ther advantages and dsadvantages are dscussed. Fnally, the sgnfcance of the proposed method n ths thess ll be mentoned. Chapter 3 In ths chapter, an overve of the proposed methodology for solder paste nspecton and shape estmaton s gven. The dscusson s based three man parts of the proposed methods: Image Acquston and Processng, Fuzzy System for Solder Paste Qualty Scorng and Shape Estmaton System. Ther orkng prncple and functonalty are brefly ntroduced durng the dscusson. Chapter 4 The detal of the drectonal LED lghtng desgn s dscussed n ths chapter. The dscusson ncludes the arrangement of the LED lghtng, the behavor of the solder paste block under the desgned lghtng and the experment setup of the proposed desgn. Vsual results of solder paste block under the lghtng are also presented. In addton, the mage processng technques used n the solder paste mages are dscussed. The dscusson s dvded nto to sectons ncludng the sde lght mage processng and top 9

29 lght mage processng. Procedures of the mage processng are gven n these sectons. Fnally, expermental results of processed mages are presented. Chapter 5 Ths long chapter presents the desgn and evaluaton of a fuzzy system for qualty scorng of solder paste block. It begns th the dscusson on the solder paste features for the fuzzy system nput. Next, the detals of fuzzy system desgn, ncludng fuzzfcaton, fuzzy rules and defuzzfcaton, are presented. Then an adaptaton technque usng neuralfuzzy netork s dscussed. Fnally, many evaluaton results of the proposed method are gven. Chapter 6 The solder paste shape estmaton algorthm s presented n ths chapter. The dscusson ncludes the detaled descrpton of the algorthm and optmzaton technque used for fndng optmal nput parameters for the algorthm. In addton, many results from the evaluaton of the method are gven n ths chapter. Chapter 7 The last chapter presents the concluson of ths thess. It concludes on the expermental results of the fuzzy system n Chapter 4 and the shape estmaton algorthm n Chapter 5. Fnally, the suggestons on future research are dscussed.

30 CHAPTE 2 Lterature eves 2. Introducton In the feld of SMT, solder paste prnt qualty s judged by the shape of the solder paste prnted on the PCB. Inspecton s done on the 2D or 3D shape of the solder paste block. Current three-dmensonal solder paste nspecton methods are manly based on the 3D surface profle of the solder paste block. In these methods, the surface profle s frst reconstructed usng varous 3D reconstructon technques. Then the geometrcal features of the solder paste block are analyzed. For example, the volume of the deposted solder paste block and the shape of the solder paste block are checked. If these features have a large devaton from those of reference solder paste block, the solder paste block ll be treated as a defectve one. The solder paste nspecton methods based on 3D shape of the solder paste nclude the follongs: laser trangulaton [2], structural lghtng [, 3, 4, 5], neural netork approach [6, 7] and shape-from-shadng [8-22]. 2.2 Laser trangulaton Laser trangulaton s a tradtonal approach for object measurement th hgh accuracy and resoluton. When t s appled to solder paste nspecton, the heght of the solder paste s measured pont-by-pont or lne-by-lne by the laser sensor. Then the 3D model of the solder paste can be created from all the heght data measured by the laser sensor. Fnally, nspecton can be done on the 3D model.

31 In ths method, a lght pont s projected by a laser dode onto the surface of the solder paste block beng measured. A dgtal lght-senstve sensor, such as charge-coupled devce (CCD, s then used to capture the lght scattered from the surface of the solder paste block. Because of the thckness of the solder paste block, the dstance beteen the sensor and the surface changes by Z. The reflected lght s maged at a ne poston X nstead of the orgnal poston X. The dstance beteen X and X s correlated to an accurate measurement of Z. The geometry of the laser trangulaton s shon n Fg. 2.. Laser Source X Detector Array X Object Surface eference Plane Z Fg. 2. Geometry of laser trangulaton method In order to construct the hole 3D profle of the solder paste block, the laser sensor needs to move across the hole surface of the paste. Dependng on the resoluton of the surface profle needed, a sgnfcant number of measurements may be needed. So the speed for reconstructng the hole surface of solder paste s very lo. 2

32 To ncrease the speed of ths method, a laser lne can be used nstead of a sngle laser pont. Therefore, a hole lne of heght data can be acqured n one measurement. In ths case, the shftng of the laser lne maged n the sensor s used to calculate the heght of the solder paste block. A number of lne scan s needed n order to construct the hole surface profle of the solder paste block. Ths approach has the advantage of hgh accuracy [, 2]. It s relatvely smple to set up for accurate laser sensor s commercally avalable n the market (e.g. lasers from Keyence. The setup of the nspecton machne requres only the ntegraton of several commercally avalable equpments such as the laser measurement sensor, the X-Y table, etc. Although nspecton usng laser trangulaton [, 2] can gve hghly accurate nspecton result, the measurement speed makes t mpractcal to perform onlne full prnted crcut board nspecton. Even lne scan type laser measurement sensor cannot meet the speed requrement of the SMT producton lne. As a result, usng ths approach, only part of the PCB s selected for nspecton n order to mantan a satsfactory producton rate. 2.3 Structural Lghtng In ths method of the structural lghtng, the 3D reconstructon of the solder paste shape s done by the analyss of a lght pattern projected onto the solder paste surface. The projected pattern s deformed accordng to the shape of the solder paste block. Phase analyss [, 3, 4, 5] or geometrcal analyss by correspondences [7] are used for the reconstructon of the 3D solder paste shape. Fnally, the nspecton can be carred out on the reconstructed shape by nspectng ts dmenson. 3

33 2.3. Phase Proflometry Phase proflometry s a non-contact 3D profle measurement technque usng perodc structural lght pattern [3, 4, 5]. In ths method, a perodc lght pattern s projected onto the object. The appearance of the object under the structural lght pattern s maged by a camera at an offset poston. The typcal lght pattern s a sequence of dark lnes hch s generated by the projecton of a square ave gratng. Snce the lght pattern captured by camera s phase-modulated accordng to the topography of the object surface, the reconstructon of the object shape can be done by extractng the phase nformaton of the lght pattern n the captured mage. The extracton of the phase nformaton can be done by Fourer Transform Proflometry or Sgnal Doman Proflometry. Yen et.al [] proposed a phase shft technque to reconstruct the solder paste surface profle usng LCD projector. The structural pattern s generated by a softare-controlled LCD panel. Then the pattern s projected onto the object usng a lght projector. The phase shft process s done by changng the pattern n the LCD panel. Four-step phase shftng method s used for the heght measurement. esults sho that t can reconstruct the solder paste shape th hgh speed and hgh resoluton. Usng phase proflometry, the heght measurement can be carred out n hgh speed hen compared th laser trangulaton. The hgh reconstructon speed makes t sutable for onlne nspecton of solder paste block. Hoever, the lght sources and the optcal system for the frnge pattern are dffcult to set up as t requres hgh precson optcal nstrument to generate the structural lght pattern. Also, the analyss of the phase shft s senstve to nose from the dgtal camera. In addton, f phase shft s nvolved, the phase-shftng devces ntroduce error to the system durng the phase shftng process. 4

34 2.3.2 Bnary Pattern Projecton Cheng et al. [7] proposed a ne 3D reconstructon method for nspectng solder bumps on afer. The proposed method s based on the projecton of a bnary lght pattern och Pattern hch conssts of brght and dark frnges. Therefore, the captured mage under the pattern contans only brght or dark regons and the mage nformaton s n form of bnary numbers. Durng measurement, the bnary pattern s shfted physcally for a number of tmes and a dstnct mage of the llumnated object surface s captured for each pattern. Then each poston on the surface can be attached th a dstnct bnary code n the sequence of captured mage. Postons on the same frnge have exactly the same bnary code. Then pont-to-pont correspondence beteen a reference plane and object surface plane by use of the bnary code and eppolar geometry. Usng the pont-to-pont correspondences, the heght nformaton beteen the reference plane and the object surface plane can be done by geometrcal analyss. As a result, the 3D shape of the object can be reconstructed by calculatng all the heght nformaton at all postons. Snce the projected pattern s bnary, the analyss only concerns about brght or dark regons. So the problem of mage brghtness, saturaton and senstvty of nose can be elmnated. Hoever, the proposed method only consders mage ponts that are close to the edge of the och pattern. The heght nformaton of non-edges ponts can only be nterpolated from those at edge ponts. Ths ll affect the accuracy of the 3D reconstructon. In addton, the geometrcal analyss requres hgh accuracy n the 5

35 nstrumental setup ncludng the angle of ncdence of the projected pattern, the dmenson of the projected pattern and the pxel sze n the CCD sensor for capturng the mage. These requrements actually ncrease the dffculty for actual mplementaton and, n turns, affect the accuracy of the reconstructed object shape. 2.4 Neural Netork 2.4. Three-dmensonal econstructon by Neural Netork In [6], neural netork estmaton s appled n the solder paste 3D surface model reconstructon. A vrtual laser 3D automatc optcal nspecton (AOI model s formed based on artfcal neural netork (ANN. It s clamed that the 3D surface model reconstructed by the proposed ANN model can acheve an accuracy of 9% on average. In ths model, the ANN uses mage features of the solder paste under dfferent lghtng condtons as nputs. The solder paste block s dvded nto a numbers of sub-areas. The gray-levels of a sub-area n the mage under dfferent lghtngs are used as nputs to the ANN. The output nodes of the ANN are the correspondng heghts of that solder paste sub-area. The proposed ANN model needs to be traned before performng 3D reconstructon. The tranng data s formed by a set of solder paste samples th knon 3D profle created by actual 3D laser measurement sensor. 6

36 The ANN vrtual laser model has the advantage of hgh nspecton speed. Once the ANN s traned, the estmated heght can be calculated by the ANN at a hgh speed. The hgh processng speed of ANN makes ths approach feasble for on-lne full nspecton. As ANN s a learnng archtecture, the tranng patterns are crucal to the accuracy of the vrtual laser model. If napproprate tranng patterns are used, the ANN may be unable to converge and cannot gve accurate results. In addton, a large number of tranng patterns s needed n the learnng stage of the ANN so that ANN vrtual laser model can be operated accurately n dfferent solder pastes. For example, f the ANN model s only traned for solder paste th certan thckness, t may have a lo accuracy hen t s used for thcker solder paste as the ANN model has not been traned th those cases Neural Netork and Fuzzy ule-based Classfcaton In [6], neural netork th fuzzy rule-based classfcaton s used n the nspecton of solder jonts. To nspect the solder jonts, a three-color crcular llumnaton system s used. The lghtng s coaxally tered upards n the sequence of green, red and blue from the bottom of the nspecton surface. Wth dfferent ncdence angle of the colored lghtng, Color patterns of dfferent slopes surfaces n the solder jonts can be captured at the same tme. As a result, the three-dmensonal shape of the solder jonts can be reflected by the captured color pattern. Usng the three-tered lghtng, color mages of solder jonts are captured. Next, the color mage of solder jont s passed to a neural netork and fuzzy rule-based classfcaton scheme. The proposed scheme conssts of three learnng vector quantzaton (LVQ clusterng classfers and fuzzy rule-based classfcaton module. It ll judge the qualty 7

37 of the solder jont from the captured mage. Expermental results sho that the scheme has the success detecton rate of 95.83% n the nspecton of solder jonts. The advantage of ths scheme s that the three dscrmnant functons can gve more complex decson boundares than the orgnal LVQ algorthms. In addton, expert knoledge of solder jont nspecton can be reflected n the predefned fuzzy classfcaton rule. So the classfcaton boundares can be adjusted by changng the rules and related membershp functons. The am of the approach n [6] s to nspect solder jonts nstead of solder paste block. Ther physcal propertes are totally dfferent. Solder jonts has curved and hghly specular reflectve surface. Hoever, solder paste block s not hghly specular reflectve. Therefore, the lghtng system proposed n ths method cannot effectvely deduce the three dmensonal shape of the solder paste block. As a result, the proposed scheme cannot be used n the solder paste nspecton. 2.5 Shape-from-shadng (SFS The frst shape-from-shadng (SFS technque as developed by Horn n the early 97s [8]. The technque s based on recoverng the shape from a gradual varaton of shadng n the mage. In SFS, the am s to recover the lght source and the surface shape at each pxel n the mage. The proposed SFS nvolves the usage of conventonal Lambertan model together th the mnmzaton of a cost functon. 8

38 After the ntaton of SFS by Horn, many other SFS algorthms th dfferent lghtng models are developed. Zhang et al. [9] have examned dfference SFS approaches that have emerged and compared ther performance. It has been found that none of the sx ell-knon SFS algorthms compared has consstent performance for all test mages. They all ork ell for certan mages, but perform poorly for others. Also, the processng tme typcally requred by the SFS algorthms s qute long. In [2-22], researchers have used neural netork to solve problem n SFS, hch s then used for 3D shape reconstructon. We et al. [2] have used a multlayer neural netork approach to tackle the SFS problem. Cho and Cho [2-22] have developed more sophstcated ANN models. In [22], ne neural-based reflectance models are presented. The feedforard neural netork (FNN model s able to generalze the dffuse term and the radal-bass functon (BF model s able to generalze the specular term. Also, a hybrd structure of FNN-based model and BF-based model s presented because most real surfaces are nether Lambertan models nor deally specular models. The accuracy of the SFS algorthm s greatly affected by the lghtng model and the object reflectance model appled n the algorthm. To make the SFS algorthm ork ell, the actual reflectance model must be formulated. As solder paste s made of large number of solder balls and flux, the reflectance model ll be very complcated and unpredctable. Therefore, ths method s unsutable for the reconstructon of 3D solder paste shape. 9

39 2.6 Sgnfcance of the esearch As dscussed above, each solder paste nspecton method has ts on advantages and dsadvantages n nspecton speed, accuracy and cost effectveness. In ths research, a novel and fast solder paste nspecton method th shape estmaton algorthm, based on only color mages, has been developed. The proposed method makes use of a doubly-colored drectonal LED lghtng structure to hghlght the 3D physcal features of the solder paste block n 2D mages. The qualty of the solder paste can be determned from the processed mages of the solder paste under the LED llumnaton usng a set of fuzzy rules. Usng the extracted 3D physcal features of the solder paste from 2D mages and the drecton of the lghtng, the shape of the solder paste can then be estmated. Experments have shon that the specally desgned LED lghtng system can effectvely determne the 3D features of the solder paste and the fuzzy system can effectvely detect the solder paste defects. The hgh nspecton speed of the proposed method makes t sutable for the on-lne full nspecton n SMT process. 2

40 CHAPTE 3 Methodology 3. Introducton In the case of solder paste stencl prntng, the 3D nformaton of the paste block s very mportant because ts volume has been lnked to the long-term solder jont relablty. The proposed methodology ams at nspectng solder paste block usng 3D nformaton deduced by drectonal lghtng. The deduced nformaton s captured by a hgh resoluton dgtal camera. The captured mages are 2.5D mages hch contan 3D nformaton n 2D mages. These mages are then used to nspect the qualty of solder paste block and to estmate the solder paste shape. Solder Paste Block Image Acquston and Processng Fuzzy System for Solder Paste Qualty Scorng Shape Estmaton System Qualty Score Estmated Shape Fg. 3. Block dagram of the proposed methodology 2

41 The block dagram for the proposed methodology s shon n Fg. 3.. It conssts of three man modules:. Image Acquston and Processng 2. Fuzzy System for Solder Paste Qualty Scorng 3. Shape Estmaton System Images of solder paste block are captured and processed by the Image Acquston and Processng Module. Next, the processed mages of solder paste block are passed to the Fuzzy System for Solder Paste Qualty Scorng Module. Then the qualty score of the solder paste block s calculated by the fuzzy system. On the other hand, the processed mages are also passed to the Shape Estmaton System Module, hch ll output an estmated shape of the nput solder paste block. The system setup of the proposed methodology nvolves varous hardare ncludng hgh resoluton dgtal camera, LED lghtng, lghtng controller, X-Y table and computer. The block dagram for the hardare connecton n the system setup s shon n Fg

42 Image Acquston and Processng X-Y Table Dgtal Camera Top LED Lghtng Lghtng Controller Sde LED Lghtng Computer Image Processor Solder Paste Block User-perceved score eference Shape Fuzzy-neural Adaptaton Fuzzy System for score calculaton Shape Estmaton Algorthm Parameters Optmzaton Fuzzy System for Solder Paste Qualty Scorng Shape Estmaton System Qualty Score 3D Shape 23

43 3.2 Image Acquston and Processng Ths module s responsble for the capturng and processng of solder paste mages. The solder paste mages are captured by a hgh resoluton dgtal camera under a set of drectonal LED lghtngs controlled by a lghtng controller. The drectonal LED lghtng s used to deduce 3D nformaton of the solder paste block by hghlghtng slopng regons of the block. The deduced 3D nformaton s captured n the 2D mages usng the dgtal camera. The captured mages are 2.5D mages as 3D nformaton s contaned n 2D mages. The captured 2.5D mages are passed to an mage processor to perform mage processng so the deduced 3D geometrcal nformaton s extracted. The processed mages are then passed to Fuzzy System for Solder Paste Qualty Scorng Module for score judgment and Shape Estmaton System Module for shape estmaton. To capture mages across the hole PCB board, the capturng devce, consstng of the dgtal camera and the drectonal LED lghtng, s mounted onto a hgh precson X-Y table. The X-Y table can be confgured to move the capturng devce to preset locatons so that the hole PCB board can be nspected. 3.3 Fuzzy System for Solder Paste Qualty Scorng Usng the processed solder paste mages from the Image Acquston and Processng Module, ths module judges the qualty of the solder paste block based on a set of fuzzy rules. A fuzzy system s frst setup based on a set of fuzzy rules for solder paste nspecton. The system extracts features from the processed solder paste mages and uses 24

44 the extracted features for judgment. A qualty score s then calculated from the extracted features. The ntal fuzzy system setup s based on the reference solder paste samples. Neuralfuzzy adaptaton can be carred out on the fuzzy system to fne tune ts behavor so that t can meet the users expected nspecton results. To carry out adaptaton, users of the system are requred to provde a set of solder paste tranng samples th expected qualty scores. Then standard neural-fuzzy tranng algorthm s used to tran the system usng the provded tranng samples. The traned fuzzy system ll ork closer to the users expectaton by gvng scores close to the users expected scores. Ths feature means the fuzzy system can adapt to nspect dfferent types of solder paste blocks. 3.4 Shape Estmaton System The processed solder paste mages are also nputted to ths module. Ths module conssts of to man parts: shape estmaton algorthm and parameter optmzaton. The shape estmaton algorthm s used for estmatng the solder paste shape. It uses a set of algorthm parameters and the processed solder paste mages from Image Acquston and Processng Module to estmate the shape. The output of the algorthm s the estmated heght map of the solder paste block. The algorthm parameters are based on reference solder paste samples. To fnd the optmal set of parameters, optmzaton algorthm s used to mnmze the dfference beteen the estmated shape and the actual shape. Wth the optmzed parameters, the shape estmaton algorthm can gve a more accurate shape. 25

45 3.5 Summary In ths chapter, a methodology for solder paste nspecton and shape estmaton from drecton lghtng s proposed. The proposed archtecture conssts of three modules: Image Acquston and Processng, Fuzzy System for Solder Paste Qualty Scorng and Shape Estmaton System. Frst, the Image Acquston and Processng module captures and processes solder paste mages th geometrcal nformaton deduced by the drectonal lghtng. Then, the results are passed to the Fuzzy System for Solder Paste Qualty Scorng module and the Shape Estmaton System module for qualty nspecton and shape estmaton respectvely. The detals of the Image Acquston and Processng module are dscussed n the next chapter. Then, the Fuzzy System for Solder Paste Qualty Scorng and Shape Estmaton System are dscussed n Chapter 5 and 6 respectvely. 26

46 CHAPTE 4 Image Acquston and Processng 4. Image Acquston 4.. Introducton In the proposed approach, mages of solder paste block are captured by a hgh resoluton dgtal camera under drectonal LED lghtng. The drectonal lghtng s used to hghlght the geometrcal features of solder paste block. By projectng the LED lghtng at an angle onto the solder paste surface, any nclned surface s hghlghted. The hghlghted nclned surfaces actually reflect the shape of the solder paste block. Hghlghted features n the captured mages can be used for three-dmensonal nspecton as they ndcate the three-dmensonal shape of the solder paste block. The detals of the lghtng desgn, expermental setup and results are gven the follong sectons Lghtng Desgn A solder paste block s llumnated th three consecutve llumnaton patterns: sde north-south, sde east-est and top lghtng. ed and blue LED lghtngs are used n the desgn. Fg. 4. shos a cross-sectonal ve of a typcal solder paste block under the desgned lghtng. 27

47 Fg. 4. Sde ve of the solder paste block under LED llumnaton A red sde lght and a blue sde lght are appled n opposte drectons at an angle of around 4 degrees th reference to the horzontal plane. Fg. 4.2(a and Fg. 4.2(b sho the top ve of the solder paste block under north-south and east-est lghtngs respectvely. North (Blue Lght Solder Paste West (Blue Lght Solder Paste East (ed Lght (a (b South (ed Lght Fg. 4.2 The top ve of the solder paste block under to dfferent llumnatons 28

48 Under the desgned llumnaton, the horzontal surface on the solder paste block s hghlghted by a mxture of red and blue lghts and ould appear purple n color. The nclned surface ould be hghlghted by the lght hch the nclned surface faces. Next, the solder paste block s llumnated by even top lghtng. The top lght makes use of hte LED to provde llumnaton. The am of the top lghtng s to locate the poston of the solder paste block and extract ts sze. Usng the top lghtng, a top ve of the actual edge boundares of the solder paste block can be extracted. The flux surroundng the solder paste block ould not be vsble from the top lght mage. For each llumnaton, an mage s taken by a dgtal camera for subsequent mage processng Expermental Setup The expermental setup conssts of four straght LED lghtng plates and one rng LED lghtng plate. In each straght LED lghtng plate, seven LEDs are arranged n a straght lne as the shon n the follong fgure: Fg. 4.3 Arrangement of LEDs n a straght LED lghtng plate 29

49 The straght LED lghtng plates are used to produce sde LED lghtng at four dfferent drectons at an nclned angle. For the rng LED lghtng plate, LEDs are arranged n a rng shape as shon n Fg The hole n the center of the rng allos the dgtal camera to ve the solder paste block. Ths lghtng plate s used to provde top lghtng durng the nspecton process. Fg. 4.4 Arrangement of LEDs n a rng LED lghtng plate The brghtness of the LEDs s controlled by a LED drvng crcut usng pulse-dth modulaton (PWM. In ths drvng crcut, same current s appled across all the LEDs. Ths can ensure that color of the LEDs s unform as the color of the LED vares th the current appled. The mplemented crcut board for controllng sx sets of LEDs and ts crcut dagram are shon n Fg. 4.5 and Fg. 4.6 respectvely. 3

50 Fg. 4.5 Crcut board for controllng LED brghtness 5VDC BCD Stch SX28 Mcro- Controller PWM Output PWM-I/P 47 BCD Stch Fg. 4.6 Crcut dagrams for controllng LED brghtness To capture mages of solder paste block, a CCD dgtal camera th resoluton of 6 x 2 pxels s set to ve from the top of the solder paste block as shon n Fg The 3

51 optcal axs of the camera s perpendcular to the nspecton plane hch s the plane of the PCB. The feld of ve of the camera s about mm x 7.5mm. A rectangular rng of sde LED lghtng s nstalled beteen the camera and the nspecton plane. ed or blue LED lghtng s used on each sde of the lghtng rng. In addton, a crcular rng of hte LED lghtng s placed from the top of the nspecton plane th hole allong the camera to ve the solder paste block n the nspecton plane. Dagram and photo of the expermental setup are shon n Fg. 4.7 and Fgure 4.8 respectvely. Computer Dgtal Camera Top Lghtng Source Lens LED Lghtng Controller Sde Lghtng Source Inspecton Plane Solder Paste Block Fg. 4.7 Dagram of the expermental Setup 32

52 Fg. 4.8 Photo of the expermental Setup To provde the llumnaton pattern n Secton 4..2, the LED arrangement n the sde lghtng source s as follos: Fg. 4.9 LEDs arrangement n sde lghtng source 33

53 In the sde lghtng source, there are four LED sources north, south, east and est lghtng. Blue LEDs are used n north and est lghtng hereas red LEDs are used n south and est lghtng. The sde lghtng source provdes to dfferent lghtng patterns north-south llumnaton and east-est llumnaton. Together th the top lghtng source hch gves the top lght llumnaton, there are totally three llumnaton patterns used n the expermental setup. For each llumnaton pattern, an mage s captured. Therefore, a total of three mages are taken for nspecton and they are summarzed as follos:. Image th north-south llumnaton 2. Image th east-est llumnaton 3. Image th top lght llumnaton Image and 2 contan the three-dmensonal geometrcal features of the solder paste block. Therefore, they are 2.5D mages hch contan three dmensonal shape nformaton n a to dmensonal mage. Expermental results of the proposed lghtng desgn are shon n the follong secton Expermental esults Fg. 4.(a shos a typcal solder paste block under sde east-est lghtng. The same block under sde north-south lghtng s shon n FIG. 4.(b. 34

54 (a (b (c Fg. 4. Images of a solder paste block thout defect under (a east-est lghtng, (b north-south lghtng and (c top lghtng; Next, the solder paste block s llumnated by even top hte lghtng as shon n fgure Fg. 4.(c. Fg. 4. shos the three acqured mages of a solder paste block th nsuffcent solder under the three consecutve llumnatons. (a (b (c Fg. 4. Images of a solder paste block th defect of nsuffcent solder under (a east-est lghtng, (b north-south lghtng and (c top lghtng The frst to mages (Fg. 4.(a and Fg. 4.(b are 2.5D mages hch contan the defect nformaton. The defect features of the solder paste block have been represented by the red and blue pxels near the center area. Fg. 4.2 gves another example of a solder paste block, hch has excess solder n the center area of the block. 35

55 (a (b (c Fg. 4.2 Images of a solder paste block th defect of excess solder under (a east-est lghtng, (b north-south lghtng and (c top lghtng Agan, the 2.5D mages have hghlghted the defect and enable ts detecton n subsequent mage processng. 4.2 Image Processng 4.2. Introducton The drectonal sde LED lghtng n Chapter 3 can effectvely hghlght the geometrcal features of the solder paste block. Under the desgned lghtng, nclned surfaces n the solder paste block appear red or blue color n the captured mages. These hghlghted nclned surfaces can be used for the nspecton of solder paste block. To facltate the nspecton process, they should be extracted from the captured mage. Image processng technques are used to extract these hghlghted surfaces n the mage. For the top lghtng mage, the locaton and the sze of the solder paste block can also be extracted by mage processng technques. The processed sde lghtng mages and top lghtng mage are used n the fuzzy system for qualty scorng and the shape estmaton algorthm n the later chapters. 36

56 4.2.2 Sde Lghtng Images For mages of sde north-south and sde east-est lghtng, the hghlghted regons by red or blue color are extracted usng the HSV color space. The to mages are frst converted from the GB color space to HSV color space. egons are hghlghted by the blue lght and can be extracted as blue pxels by consderng the hue value of those pxels that are around 24 th a range of around 2. Smlarly, regons hghlghted by the red lght are extracted as red pxels by consderng pxels th hue value around. Next, openng and closng of the extracted color regons are performed to remove any nose and to connect any small area of dsconnected pxels n a regon. Fg. 4.3(a and Fg. 4.3(b sho the extracted color clusters of a normal solder paste block due to sde east-est lghtng and sde north-south lghtng respectvely. For a typcal defect block th a hole due to nsuffcent solder, the extracted color clusters are shon n Fg. 4.3(c and Fg. 4.3(d. Another common defect s the appearance of a slump hch may due to the absorpton of mosture by the paste or excess solder. The typcal extracted color clusters are shon n Fg. 4.3(e and Fg. 4.3(f. The procedure of mage processng for sde-lght mages s summarzed as follos:. Convert the mages from the GB color space to HSV color space. 2. Extract blue hghlghted regon as blue pxels by consderng pxels th hue value around Extract red hghlghted regon as red pxels by consderng pxels th hue value around. 4. Perform openng and closng on the color regons. 37

57 Wthout defect (a (b Wth defect (Insuffcent solder (c (d Wth defect (Excess solder (e (f Fg. 4.3 Typcal extracted color clusters from solder paste blocks Top Lghtng Images Under the hte LED top lght, the solder paste block appears to be grey n color. A mask hch ndcates the poston and the sze of the solder paste block can be extracted usng color segmentaton of the solder paste color. The solder paste color can be found by the mages of reference solder paste sample under the top lght envronment. Then, openng and closng of the extracted color regons are performed to remove any nose and to connect any small area of dsconnected pxels n a regon. Example of the extracted solder paste mask s shon n the follong fgure: 38

58 Orgnal Image Extracted Solder Paste Mask Fg. 4.4 Example of extracted solder paste mask 4.3 Summary In ths chapter, the acquston and processng solder paste mages are dscussed. In the acquston of solder paste mages, solder paste blocks are llumnated by specally desgned drectonal lghtngs. Images are then captured usng a dgtal camera. The captured mages are 2.5D mages hch contans 3D geometrcal nformaton of the solder paste block. Fnally, the captured mages are processed to extract the geometrcal features contaned n them. The proposed Image Acquston and Processng module s evaluated th defectve and non-defectve solder paste samples. Expermental results of the processed mages sho that the desgned lghtng can effectvely hghlght the geometrcal nformaton of the solder paste blocks. The hghlghted slopng regons hch are colored by red and blue lght are extracted from the orgnal mages to form the processed mages. These processed mages are then passed to the fuzzy system n Chapter 5 and the shape estmaton system n Chapter 6 for further analyss. 39

59 CHAPTE 5 Fuzzy System for Solder Paste Qualty Scorng 5. Introducton The objectve of the fuzzy system s to judge the qualty of the prnted solder paste block from a number of features extracted from the processed 2.5D and the top lght mages. The extracted features are served as the nput of the fuzzy system and the fuzzy system gves a sngle score value as an output. A score n the range beteen - and ould ndcate the qualty of the solder paste block. A lo score ould be an ndcaton of bad solder paste prntng qualty hle a hgh score s an ndcaton of good prntng qualty. It s ell-establshed that the human reasonng and judgment process can be modeled by a fuzzy system, hch enables the use of fuzzy rule-base for qualty judgment. The rulebase can reflect the actual qualty judgment process of doman experts. In addton, the adaptaton can be carred out usng neural-fuzzy tranng algorthms so that the fuzzy system can be adjusted accordng to the user s preferences. The overall system conssts of to modules as shon n fgure 5.. The frst module, Module A, s the fuzzy system for qualty score calculaton. It calculates the qualty score of the solder paste based on a set of fuzzy rules from the nput solder paste features. The second module, Module B, s the fuzzy-neural netork for fne tunng the performance of the Module A. The tunng process s carred out by adjustng the membershp functons of the fuzzy system. 4

60 Module B User-perceved Score Value Adaptaton to Membershp Functons and ule eghtngs Processed Top Lghtng Image Processed Sde East-West Lghtng Image Feature Extracton Fuzzfcaton Fuzzy ules Defuzzfcaton Qualty Score Processed Sde North-South Lghtng Image Module A Fg. 5. Block Dagram of Fuzzy System for Solder Paste Qualty Scorng 5.2 Motvaton and Feature Extracton for Fuzzy System From the 2.5D mages of sde east-est lghtng and sde north-south lghtng, features of solder paste block are extracted from them and the respectve top lght mage. In these mages, edge regons are colored and they are drectly related to the geometrcal shape of the solder paste block. For example, the appearances of the colored regon n the mages ould appear dfferently for dfferent edge slope nclnatons. Fgure 5.2 shos the processed mages of a defectve solder paste block. Comparng th the non-defectve solder paste block shon n fgure 5.3, the color regons n defectve solder paste, such as the central regon, south and east edge regon, s more complcated than the non-defectve one. 4

61 West Edge egon North Edge egon East Edge egon South Edge egon (a (b (c Fg. 5.2 Processed mages of a defectve solder paste block Central egon North Edge egon West Edge egon East Edge egon South Edge egon Central egon (a (b (c Fg. 5.3 Processed mages of a non-defectve solder paste West Edge egon and South Edge egon are the red color clusters at the est-most and south-most sde of the solder paste block respectvely. Smlarly, North Edge egon and East Edge egon are the north-most and east-most blue regon respectvely. Examples of these regons n a defectve solder paste block are shon n Fg. 5.2(a and Fg. 5.2(b. Typcal examples of these regons n a non-defectve sample are shon n Fg. 5.3(a and Fg. 5.3(b. Central egon of the solder paste block s the regon bounded by the north, south, east and est edge regons. Example of central regon n a defectve and non-defectve solder paste block s shon n Fg. 5.2(c and Fg. 5.3(c respectvely. 42

62 Currently, the extracton of these regons s not automated as ths research concentrates on the desgn and evaluaton of the nspecton process. The automaton of the regon extracton s expected to be handled n the future research. In general, a typcal non-defectve solder paste block conssts of a central regon and four edge regons: North, South, East and West Edges. The central regon s the regon bounded by the four edges. The poston, sze and shape of these regons reflect the physcal shape of the solder paste block and, hence, the qualty of the solder paste block. From the study of a large number of defectve and non-defectve solder paste samples, t s found that non-defectve solder paste samples th smlar thcknesses gves smlar postons and szes of the edge regons and central regons. For defectve samples, any defects n the central regon ll appear as blue or red color clusters. To determne the qualty of the solder paste blocks, the above regons need to be analyzed to gve the fnal qualty score. For edge regons, the shape of non-defectve solder paste blocks should not have large devaton n the shape and sze. As the edge thckness and the edge area reflects the shape of the edge regons, the average edge thckness and edge surface area for each edge are selected as nput features of the fuzzy system. Also, the thckness devatons of the edges are useful for judgng the qualty of the edge regons. In addton, dscontnutes n the edge regons mean that defects exst n the edges. In addton, the connectvty of the edge regon needs to be studed. As a result, for edge regons, edge thckness, edge thckness devaton, edge surface area and edge connectvty are chosen as the nput features of the fuzzy system. 43

63 For the central regon, f t contans any defect, t ll contan red or blue color clusters due to the drectonal lghtng. As the sze of the defect reflects the qualty of the solder paste block, the sze of these red or blue regons s drectly related to the solder paste qualty. Also, for non-defectve solder paste blocks, the sze of the central regon should not have large devaton from that of the reference non-defectve samples. Therefore, the sze of the central regon needs to be checked durng the nspecton process. For the hole solder paste block, the poston and the surface area are closely related to the prntng qualty. Shftng of the prntng poston s one of the major defects n solder paste prntng. Also, the surface area of the solder paste block can help to nspect hether there s suffcent solder paste s prnted. As a result, solder paste surface area and block shftng are ncluded as nput features of the fuzzy system. After above analyss of the solder paste shape, tenty features of the solder paste block are extracted and ther descrptons are gven as follos: Feature West Edge Thckness - By calculatng the average dth of all the ros of blue pxels n the West Edge egon Feature 2 East Edge Thckness - By calculatng the average dth of all the ros of red pxels n the East Edge egon Feature 3 North Edge Thckness - By calculatng the average dth of all the columns of blue pxels n the North Edge egon Feature 4 South Edge Thckness 44

64 - By calculatng the average dth of all the columns of red pxels n the South Edge egon Feature 5 West Edge Thckness Devaton - By calculatng the standard devaton of the West Edge Thckness Feature 6 East Edge Thckness Devaton - By calculatng the standard devaton of the East Edge Thckness Feature 7 North Edge Thckness Devaton - By calculatng the standard devaton of the North Edge Thckness Feature 8 South Edge Thckness Devaton - By calculatng the standard devaton of the South Edge Thckness Feature 9 West Edge pxels connectvty - By countng the number of gaps vertcally n the West Edge egon Feature East Edge pxels connectvty. - By countng the number of gaps vertcally n the East Edge egon Feature North Edge pxels connectvty - By countng the number of gaps horzontally n the North Edge egon Feature 2 South Edge pxels connectvty - By countng the number of gaps horzontally n the South Edge egon Feature 3 West Edge egon Surface Area - By calculatng the number of red pxels n the West Edge egon Feature 4 East Edge egon Surface Area - By calculatng the number of blue pxels n the East Edge egon Feature 5 North Edge egon Surface Area - By calculatng the number of blue pxels n the North Edge egon Feature 6 South Edge egon Surface Area 45

65 - By calculatng the number of red pxels n the South Edge egon Feature 7 Central egon Surface Area - By calculatng the number of pxels thn the boundares of the Central egon Feature 8 Central egon Defect Area - By calculatng the number of red and blue pxels thn the Central egon Feature 9 Solder Paste Block Shftng - By calculatng the horzontal and vertcal shft of the block th respect to a reference locaton Feature 2 Solder Paste Surface Area - By calculatng the number of pxels thn the block as seen from the top lght mage - Features -8 are extracted from the processed 2.5D mages of north-south and east-est lghtng. On the other hand, features 9 and 2 are extracted from the top lght mage of the solder paste block. 5.3 Normalzaton of the Extracted Features The extracted features of the solder paste block are of dfferent scales. For example, the order of the solder paste surface area s ten tmes the order of the solder paste edge area. Therefore, these features are normalzed to the same scale before fuzzfcaton. The nomnal range of each feature s scaled to a range beteen and. They are determned by analyzng the reference solder paste samples. For the 5m thck square-shaped solder paste block, an example of scalng equatons s shon n the follong table: 46

66 Feature Nomnal ange Scalng Functon West Edge Thckness ( t East Edge Thckness ( t e North Edge Thckness ( t n South Edge Thckness ( t s West Edge Thckness Devaton ( d 5 pxels 5 pxels 5 pxels 5 pxels pxels t 5 t e 5 t n 5 t s 5 d East Edge Thckness Devaton ( d North Edge Thckness Devaton ( d South Edge Thckness Devaton ( d e n s pxels pxels pxels d e d n d s West Edge Pxels Connectvty ( East Edge Pxels Connectvty ( e North Edge Pxels Connectvty ( n South Edge Pxels Connectvty ( s West Edge egon Surface Area ( e East Edge egon Surface Area ( e e North Edge egon Surface Area ( e n South Edge egon Surface Area ( e s Central egon Surface Area ( s c Central egon Defect Area ( d Solder Paste Block Shftng ( b Solder Paste Surface Area ( s s c 5 gaps c 2 c 5 gaps c 2 c 5 gaps c 2 c 5 gaps c 2 5 pxels 5 pxels 5 pxels 5 pxels pxels pxels pxels 4 6 pxels e n s e 4 e e 4 e 4 e s 4 s c 25 2 d b s s 4 2 Table 5. Feature scalng equatons for 5m square-shaped solder paste block 47

67 All these equatons make sure that the values of feature have been normalzed to a range beteen and. 5.4 Fuzzfcaton of the Extracted Features After the normalzaton of the extracted features, they are fuzzfed by a set of membershp functons. The shape of the membershp of the membershp functon s determned based on the nature of the extracted features and the fuzzy rules. In ths fuzzy system, to types of sgmodal functon are used. Sgmodal functons can be used to specfy asymmetrc membershp functon hch allos more flexblty on the adjustment of membershp durng tranng. The frst type of membershp functon (f s a smple sgmodal functon (f th to parameters (r, s defnng ts shape. The equaton of ths functon s: = (5. e ( x, r, s r ( x s f here r s postve value that defnes the curvature of the sgmodal shape and s defnes the poston hen the functon attans. Typcal functon shape s shon n Fg f x Fg. 5.4 Typcal shape of membershp functon f 48

68 The second type of membershp functon (f 2 s composed of the dfference beteen to sgmodal functons. It depends on four parameters, a, b, c and d. The equaton s as follos: f = (5.2 e e 2( x, a, b, c, d a( x b c( x d here a, c defne the curvature of the functon shape and b, d defne poston here the functon value s. Fgure 5.5 shos the typcal shape of ths type of membershp functon. f 2 x Fg. 5.5 Typcal shape of membershp functon f 2 All the scaled feature values are fuzzfed by one of these to types of membershp functons. The type of membershp functon for each feature s summarzed n the follong table: 49

69 5 Lngustc Term of Scaled Feature Membershp Functon West Edge Thckness ( ' t s GOOD ' ( ' ( 2,,, ', ( d t c b t a e e d c b a t f = = East Edge Thckness ( ' e t s GOOD ' ( ' ( ,,, ', ( d t c b t a e e e e e d c b a t f = = North Edge Thckness ( ' n t s GOOD ' ( ' ( ,,, ', ( d t c b t a n n n e e d c b a t f = = South Edge Thckness ( ' s t s GOOD ' ( ' ( ,,, ', ( d t c b t a s s s e e d c b a t f = = West Edge Thckness Devaton ( ' d s BAD ' (, ', ( s d r e s r d f = = East Edge Thckness Devaton ( ' e d s BAD ' ( , ', ( s d r e e e s r d f = = North Edge Thckness Devaton ( ' n d s BAD ' ( , ', ( s d r n n e s r d f = = South Edge Thckness Devaton ( ' s d s BAD ' ( , ', ( s d r s s e s r d f = = West Edge Pxels Connectvty ( ' c s BAD ' ( , ', ( s c r e s r c f = = East Edge Pxels Connectvty ( ' e c s BAD ' ( , ', ( s c r e e e s r c f = = North Edge Pxels Connectvty ( ' n c s BAD ' ( , ', ( s c r n n e s r c f = = South Edge Pxels Connectvty ( ' s c s BAD ' ( , ', ( s c r s s e s r c f = = West Edge egon Surface Area ( ' e s GOOD ' ( ' ( ,,, ', ( d e c b e a e e d c b a e f = = East Edge egon Surface Area ( ' e e s GOOD ' ( ' ( ,,, ', ( d e c b e a e e e e e d c b a e f = = North Edge egon Surface Area ( ' n e s GOOD ' ( ' ( ,,, ', ( d e c b e a n n n e e d c b a e f = = South Edge egon Surface Area ( ' s e s GOOD ' ( ' ( ,,, ', ( d e c b e a s s s e e d c b a e f = =

70 ' c Central egon Surface Area ( s s GOOD Central egon Defect Area ( d ' s BAD Solder Paste Block Shftng ( b ' s BAD Solder Paste Surface Area ( s s GOOD ' s = = f2( s 9 c', a9, b9, c9, d9 a9 ( sc ' b9 c9 ( sc ' d 9 e e = = 9 f( d', r9, s9 r9 ( d ' s9 = f b r s = e = e 2 ( ',, r ( b' s f2( s s ', a, b, c, d a ( ss ' b c ( ss ' d e e Table 5.2 Membershp Functons of Lngustc Terms = All these membershp functons gve a fuzzfed feature n a range beteen and. These fuzzfed features are then appled to a set of fuzzy rules for analyss. 5

71 5.5 ules for the Fuzzy System After fuzzfcaton of the nputs, the follong set of fuzzy rules s then appled to the fuzzfed nputs. A Mamdan fuzzy nference model s used. The respectve rule eghtng s shon nsde the brackets.. IF West Edge Thckness s GOOD and East Edge Thckness s GOOD and North Edge Thckness s GOOD and South Edge Thckness s GOOD, THEN qualty score s HIGH. ( 2. IF West Edge Thckness Devaton s BAD or East Edge Thckness Devaton s BAD or North Edge Thckness Devaton s BAD or South Edge Thckness Devaton s BAD, THEN qualty score s LOW. ( 2 3. IF West Edge Pxels Connectvty s BAD or East Edge Pxels Connectvty s BAD or North Edge Pxels Connectvty s BAD or South Edge Pxels Connectvty s BAD, THEN qualty score s LOW. ( 3 4. IF West Edge egon Surface Area s GOOD and East Edge egon Surface Area s GOOD and North Edge egon Surface Area s GOOD and South Edge egon Surface Area s GOOD, THEN qualty score s HIGH. ( 4 5. IF Central egon Surface Area s GOOD, THEN qualty score s HIGH. ( 5 6. IF Central egon Defect Area s LAGE, THEN qualty score s LOW. ( 6 7. IF Solder Paste Block Shftng s HIGH, THEN qualty score s LOW. ( 7 8. IF Solder Paste Surface Area s GOOD, THEN qualty score s HIGH. ( 8 Among these eght fuzzy rules, rule, 4, 5 and 8 ould contrbute to a qualty score close to the upper sde of the range after defuzzfcaton. On the other hand, rule 2, 3, 6 and 7 52

72 ould contrbute a qualty score to the loer sde of the range. User can adjust the sgnfcance of each rule by changng the eghtng of the rule. For each rule, an output value beteen and s gven to ndcate the postve/negatve contrbuton to the qualty score. In rule, 4, 5 and 8, an output value of by the rule ndcates a hgh postve contrbuton to the qualty score hle an output value of n rule 2, 3, 6 and 7 ndcates a hgh negatve contrbuton to the qualty score. The desgn of the fuzzy rules s based on the reference solder paste blocks. If the edge thckness, edge regon surface area, central regon surface area and solder paste surface area are close to the nomnal values, then a good qualty score ould be obtaned. On the other hand, a bad qualty score ould be obtaned for a bad edge thckness devaton, edge connectvty, central regon defect area and solder past block shftng. The fuzzy O and fuzzy AND operators are used to assocate relevant features n a rule. For fuzzy O operator, the maxmum functon and probablstc O functon are commonly used. In ths fuzzy system, the probablstc O functon s chosen for fuzzy O operator because the maxmum functon s a dscontnuous functon hch ncreases the dffculty n the dervng the equatons for tranng. Therefore, the equaton for a or b s: a b = a b = a b ab (5.3 For fuzzy AND operaton, the mnmum functon, product operator and averagng functon can be used. The mnmum functon s a dscontnuous functon and t ll ncrease the complexty of the equatons for tranng. On the other hand, product operator 53

73 makes the value of a and b and c and d too small as a, b, c and d are beteen and. Therefore, averagng functon s chosen n ths fuzzy system and the equaton for a and b and c and d s defned as: a b c d a b c d = (5.4 4 Let be the output value of rule here =,2, K, 8. The equatons are summarzed as follos: = ( = (5.6 3 = ( = ( = 9 (5.9 6 = 9 (5. 7 = 2 (5.2 8 = (5.3 All these output values from the rules are defuzzfed to form a crsp qualty score durng defuzzfcaton process. 54

74 Defuzzfcaton In the defuzzfcaton process, the postve contrbuton ( p y and negatve contrbuton ( n y by the rules are calculated separately. Wth rule eghtng, p y and n y are calculated usng the follong equatons: p y = ( n y = (5.5 p y and n y are then combned to form the crsp qualty score (y beteen - and as follos: n p y y y = = (5.6 The crsp qualty score tends to f the nput features cause lo negatve contrbuton and hgh postve contrbuton durng fuzzy rules analyss. Ths s the case hch a nondefectve solder paste block s under nspecton. On the other hand, hen a defectve solder paste block s nspected the system, a score tends to - s gven as ts nput cause hgh negatve contrbuton and lo postve contrbuton durng fuzzy rules analyss.

75 5.7 Fuzzy-Neural Adaptaton for the Fuzzy System Fuzzy-neural system s a combnaton of fuzzy system and neural netork. Therefore, adaptaton can be carred out to adjust the parameters of the fuzzy systems usng a set of tranng samples and tranng algorthm derved from neural netork theory. The adjustment of the parameters ould lead to a change n the shape of the membershp functons. The changes ould optmze the output of the fuzzy system hch means that the system ould behave closer to the users expectaton. Durng tranng, fuzzy-neural netork ould mnmze the dfference beteen actual fuzzy system s output and the desred output n provdng a score for the solder paste block nspecton. Backpropagaton algorthm s used for the tranng of the fuzzy-neural netork. From the structural pont of ve, the fuzzy-neural system can be veed as a fve-layer feed-forard neural netork as shon n Fgure 5.6. Other than the nput and output layers, the fuzzy-neural system has three hdden layers that represent the membershp functons and fuzzy rules. The tenty extracted features are crsp nput values to the fuzzy-neural netork. There s only one output from the netork, hch represents the value of the qualty score. The second layer generates the membershp grades of the nput lngustc terms usng equatons n Table 5.4. Layer three s the rule layer descrbed n secton 5.5. A rule neuron receves nputs from the respectve fuzzfcaton neurons and calculates the frng strength of the rule. Layer four and fve are responsble for the defuzzcaton process. In layer four, as descrbed n secton 5.6, the frng strengths of the postve rules are combned to form the postve contrbuton ( y p and the frng strengths of the negatve rules are combned to form the 56

76 negatve contrbuton ( y n. Then to represent the qualty score of the solder paste block. y p and y n are combned to form an output crsp value From Table 5.2, there are totally 6 parameters n the membershp functons. For ule, 4, 5 and 8, there are 4 parameters ncludng a, b, c, d for =,2, K,. The remanng 2 parameters, r, s for =,2, K,, are n rule 2, 3, 6 and 7. Durng tranng, the qualty scores of the tranng samples are frst calculated usng the fuzzy system th ntal parameters. Next, the results are compared th the desred output of the tranng samples. The dfference beteen the actual output and desred output s then propagated backards through the netork from the output layer to the nput layer. Let E be the dfference (error beteen the actual qualty score ( y and the desred output qualty score ( y d. E 2 = 2 ( y d y (5.7 To mnmze E, gradent descent-based method s used. The gradent of E s calculated and parameters are adjusted so that the mnmum s reached. The dervaton of the parameter changes s as follos: Let α be the learnng rate of the tranng algorthm and a, b, c, d, r, s for =,2, K, are changes of the parameters n each tranng teraton. 57

77 Layer Layer 2 Layer 3 Layer 4 Layer 5 West Edge Thckness GOOD East Edge Thckness North Edge Thckness South Edge Thckness GOOD GOOD GOOD West Edge Surface Area East Edge Surface Area North Edge Surface Area South Edge Surface Area GOOD GOOD GOOD GOOD Central egon Surface Area GOOD Solder Paste Surface Area GOOD HIGH y p Σ Score West Edge Thckness Devaton BAD 2 East Edge Thckness Devaton North Edge Thckness Devaton BAD BAD LOW y n South Edge Thckness Devaton BAD 6 West Edge Connectvty BAD 5 3 East Edge Connectvty North Edge Connectvty South Edge Connectvty BAD BAD BAD Central egon Defect Area LAGE Solder Paste Block Shftng HIGH 2 7 Fg. 5.6 Structure of the neural-fuzzy netork 58

78 59 For edge thcknesses ( 4 =,2,3,, a u y y E a E a = = α α (5.8 b u y y E b E b = = α α (5.9 c u y y E c E c = = α α (5.2 d u y y E d E d = = α α (5.2 For edge regon surface areas ( 8 = 5,6,7,, a u y y E a E a = = α α 4 4 (5.22 b u y y E b E b = = α α 4 4 (5.23 c u y y E c E c = = α α 4 4 (5.24 d u y y E d E d = = α α 4 4 (5.25 For central regon surface area ( 9 =, a u y y E a E a = = α α 5 5 (5.26 b u y y E b E b = = α α 5 5 (5.27 c u y y E c E c = = α α 5 5 (5.28

79 6 d u y y E d E d = = α α 5 5 (5.29 For solder paste surface area ( =, a u y y E a E a = = α α 8 8 (5.3 b u y y E b E b = = α α 8 8 (5.3 c u y y E c E c = = α α 8 8 (5.32 d u y y E d E d = = α α 8 8 (5.33 For edge thckness devatons ( 4 =,2,3,, r u y y E r E r = = 2 2 α α (5.34 s u y y E s E s = = 2 2 α α (5.35 For edge connectvty ( 8 = 5,6,7,, r u y y E r E r = = 3 3 α α (5.36 s u y y E s E s = = 3 3 α α (5.37

80 6 For central regon defect area ( 9 =, r u y y E r E r = = α α (5.38 s u y y E s E s = = α α (5.39 For solder paste block shftng ( =, r u y y E r E r = = α α (5.4 s u y y E s E s = = α α (5.4 To calculate the changes of the parameters, the partal dervatves of other equatons are needed. They are shon as follos: ( y y y E d = ( y = ( y = ( y = ( y = ( y = (5.47

81 y = ( y = ( y = ( = = = = u u u u ( = = = = u u u u ( = u ( = u ( = u ( = u ( = u ( = u ( = u (5.59

82 u 3 = u 3 = (5.6 (5.6 u 3 = (5.62 u 6 = 9 u 7 = 2 (5.63 (5.64 For =,2, K, u a u b u c u d u r u s ( x b e = = a ( x b a ( ( 2 x b e a e a ( x b a ( ( 2 x b e ( x d e = = c ( ( 2 x d e c e c ( x d c ( ( 2 x d e ( x s e = = c ( x d r ( x s r ( ( 2 x s e r e r ( x s r ( ( 2 x s e (5.65 (5.66 (5.67 ( (5.7 Combnng all the partal dervatves, the changes for each parameter ((5.8 to (5.4 can be formulated. 63

83 The above adaptaton process allos the fne tunng of the fuzzy system to match th users desred output. For solder paste sample th unsatsfactory nspecton result, users can present t to the fuzzy system for fne tunng. The tranng process nvolves a number of teratons for parameters adjustment. After the tranng process, the system ould then perform as users expectaton. 5.8 Fuzzy System Setup Procedures The fuzzy system s desgned to perform nspecton on solder paste blocks. The setup procedures nvolve the follong steps:. Desgn of scalng functons for nput features 2. Assgnment of ntal membershp functons parameters 3. Assgnment of rule eghtngs 4. System Testng 5. Adaptaton to meet users expectaton Frstly, the scalng functons for each features need to be formulated. They are used to normalze the features value to the range beteen and before nputtng to the fuzzy system. To formulate the scalng functon for a feature, nomnal range of the feature value s frst found from reference solder paste samples. Then the functon s desgned so that the nomnal range s scaled to the range beteen and. As dfferent types of solder paste block have ther on nomnal range of feature value, dfferent set of scalng functons are used. 64

84 Secondly, the ntal membershp functon parameters are assgned. The parameters are assgned n a ay that the system gves hgh score for non-defectve solder paste samples and lo scores for defectve samples. To acheve ths, membershp functons for postve lngustc terms, such as West Edge Thckness s GOOD, are set so that the functons gve values close to for non-defectve samples and gve values close to for defectve sample. Smlarly for negatve lngustc terms, the membershp functons gve values close to for non-defectve samples and gve values close to for negatve samples. Thrdly, the rule eghtngs are set. The assgnment of the rule eghtngs s based on the user s percepton on the mportance of a fuzzy rule. If a partcular fuzzy rule s more mportant than the others, a hgher rule eghtng can be assgned to t n order to reflect ts mportance. After adjustng the rule eghtngs, they need to be normalzed to value beteen and. The most mportant rule has the rule eghtng equals to. The normalzaton process s done separately for the group of postve contrbuton rules and the group of negatve contrbuton rules. Then the fuzzy system s tested th the set of reference solder paste samples. The fuzzy system gves scores close to for non-defectve samples and scores close to - for defectve samples. If there are msclassfed samples, fne-tunng of the fuzzy system usng neural-fuzzy adaptaton s needed. The fnal step s to fne-tune the fuzzy system through neural-fuzzy adaptaton. Ths step s only carred out hen the users do not satsfy th the testng results n the prevous step. Before startng the adaptaton process, a tranng set of solder paste samples s formed. For each tranng sample, a users desred score s assgned. The desre score s 65

85 based on the orgnal score gven by the fuzzy system th small adjustment on the score value. Over-scored samples ll have desred scores loer than ther orgnal scores, hereas under-scored samples ll have desred scores hgher than ther orgnal scores. Then tranng samples are used to tran the fuzzy system. After tranng, the behavor of the fuzzy system ll be closer to the user s expectaton. 5.9 Evaluaton 5.9. Square-shaped Solder Paste Block Expermental Setup and Procedures The fuzzy system s frst set up to nspect the qualty of rectangular solder paste block th thckness of 5m. The ntal parameters of all the membershp functons are set based on the reference 5m solder paste samples. The scaled features of the reference solder paste blocks are extracted and calculated based on the functons n Table 5.. Then parameters of the membershp functons are set manually so that the output of the membershp functon tends to for postve contrbuton rules and tends to for negatve contrbuton rules th the gven reference solder paste features. Parameters used n the membershp functons for 5m solder paste block are shon n the follong table: 66

86 a b c Table 5.3 Intal parameters of membershp functons for 5m solder paste block d r s The shapes of membershp functons are shon n fgure 5.7. All the rule eghtngs ( for =,2, K, 8 are set to so that all the rules have equal contrbuton to the fnal qualty score. To test the adaptaton process of the fuzzy-neural netork, solder paste samples are chosen to form a tranng dataset. Then the tranng algorthm s run usng the tranng set th teratons. The results are compared th the results before tranng and users expectaton. 67

87 Fg. 5.7 Intal Membershp Functons of the Fuzzy System Experment In experment, the behavor of the fuzzy system th ntal membershp parameters s tested. Solder paste samples th three dfferent thcknesses (5m, 2m and 25m are presented to the fuzzy system. There are totally 3 solder paste samples used n ths experment. Among these solder paste samples, 8 of them (A-A8 are of 5m thck, 3 samples (B-B3 are from 2m group and the remanng 9 samples (C-C9 are 25m thck. Solder paste samples used n ths experment are shon n the follong table. 68

88 * Scaled Features are arranged n the follong orders: t t e t n t s e e e e n e s s c s s d d e d n d s c c e c n c s d b here t : est edge thckness t e : east edge thckness t n : north edge thckness t s : south edge thckness e : est edge regon surface area e e : east edge regon surface area e n : north edge regon surface area e s : south edge regon surface area s c : central regon surface area s s : solder paste surface area d : est edge thckness devaton d e : east edge thckness devaton d n : north edge thckness devaton d s : south edge thckness devaton c : est edge pxels connectvty c e : east edge pxels connectvty c n : north edge pxels connectvty c s : south edge pxels connectvty d : central regon defect area b : solder paste block shftng No. Appearance Scaled Features* Comment 5m A A A A A A Defect: Edge Connectvty No defect Defect: Edge Connectvty No defect Defect: Excess Solder No defect 69

89 A7 A Defect: Block Shftng No defect 2m B B2 B3 B4 B5 B6 B7 B8 B9 B B B Thcker Solder Paste Thcker Solder Paste Thcker Solder Paste Thcker Solder Paste Thcker Solder Paste Defect: Block Shftng Thcker Solder Paste Thcker Solder Paste Defect: Edge Connectvty Thcker Solder Paste Defect: Insuffcent Solder Thcker Solder Paste 7

90 B Thcker Solder Paste 25m C C2 C3 C4 C5 C6 C7 C8 C Thcker Solder Paste Thcker Solder Paste Thcker Solder Paste Defect: Excess Solder Thcker Solder Paste Thcker Solder Paste Thcker Solder Paste Thcker Solder Paste Thcker Solder Paste Table 5.4 Solder Paste Block Samples 7

91 Scores of each solder paste sample are gven by the ntal fuzzy system: No. Score Comment 5m A.74 The score appears too hgh as t has defect of broken edges A2 6 Correct A3.279 Correct judgment for defect: edge connectvty A4 795 Correct A Correct judgment for defect: Excess Solder A6.923 Correct A The score appears too hgh as t has defect of block shftng A The score appears too lo as t s a normal sample 2m B Correct B Correct B Correct B Correct B Correct B Correct B Correct B8 -.2 Correct B Correct B Correct B Correct B Correct B3 -.9 Correct 25m C Correct C Correct C Correct C4-3 Correct C Correct C Correct C Correct C Correct C Correct Table 5.5 Qualty score of solder paste block samples th ntal membershp functon parameters 72

92 Non-Defectve Defectve Fg. 5.8 Score dstrbuton of solder paste samples th ntal membershp functon Wth the ntal parameters, the fuzzy system can dentfy non-defectve 5m solder paste samples from all the samples th a postve qualty score as output. Defectve solder paste sample th excess solder gves a negatve qualty score so ths type of defect s clearly dentfed by the fuzzy system. Hoever, defectve solder paste samples th problems n edge connectvty and block shftng cannot be judged by the fuzzy system. Adaptaton s needed to mprove the performance of the system. For 2m solder paste samples, the score range for non-defectve ones s beteen -.9 and Defectve samples result n loer score values n the range beteen and The score range s clearly dfferent from that of 5m solder paste samples. Ths sho that the system can fgure out the dfference beteen 5m and 2m solder paste samples. For 25m solder paste samples, the score range for non-defectve ones s beteen and Defectve sample leads to loer score value about -. The score range s qute close to that of 2m solder paste samples. To mprove the performance of the fuzzy system, the score ranges of 2m and 25m solder paste samples should be separated further aay th each other. 73

93 Experment 2 The am of ths experment s to mprove the performance of the fuzzy system n experment through fuzzy-neural adaptaton. A set of tranng samples s prepared for the fuzzy system to carry out the adaptaton so that problems found n experment could be elmnated. The tranng set s shon n the follong table: No. Score n Desred Score Explanaton Experment 5m A To loer the score for the defect: edge connectvty A2 6.9 To rase the score for non-defectve solder paste sample A To loer the score for the defect: excess solder A To loer the score for the defect: block shftng A To rase the score for non-defectve solder paste sample 2m B To den the score separaton beteen 2m and 25m samples B To den the score separaton beteen 2m and 25m samples B To loer the score of defectve 2m solder paste sample B To den the score separaton beteen 2m and 25m samples 25m C To den the score separaton beteen 2m and 25m samples C To loer the score of defectve 25m solder paste sample C To den the score separaton beteen 2m and 25m samples C To den the score separaton beteen 2m and 25m samples C To den the score separaton beteen 2m and 25m samples Table 5.6 Tranng Solder Paste Block Samples of Experment 2 74

94 Solder paste sample A and A7 contan the defect of edge connectvty and block shftng respectvely. A loer desred score s gven to them so that they can be dstngushed from the non-defectve solder paste block. Other non-defect 5m solder paste samples are gven a hgh desred score value of.9 hle defect one has a lo desred score value of -. For 2m solder paste samples, normal samples are gven a hgher score value around. so that the score dfference beteen 2m and 25m samples can be ncreased. To further ncrease ther dfference, 25m samples are gven a loer desred score of -. After the tranng process, all samples are presented to the adapted fuzzy system for testng. The result s shon n the follong table: 75

95 No. Old Score Ne Score Comment 5m A Correct judgment for defect: edge connectvty A Correct A Correct judgment for defect: edge connectvty A Score s too close to defectve solder paste, A and A3 A Correct judgment for defect: excess solder A The score appears too lo as t s a normal sample A Correct judgment for defect: block shftng A Correct 2m B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct 25m C Correct C Correct C Correct C Correct C Correct C Correct C Correct C Correct C Correct Table 5.7 Qualty Score of Solder Paste Block Samples After Tranng 76

96 The shapes of membershp functons after tranng are shon n the follong fgure: Fg. 5.9 Membershp functons of the fuzzy system after tranng Non-Defectve Defectve Fg. 5. Score Dstrbuton of Solder Paste Samples After Tranng After tranng, the scores of 5m solder paste sample A and A7 are loer than that before tranng. For A7, the score drops to.443 hch s sgnfcantly dfferent from the score of non-defectve solder paste hch s around to.7. For solder paste samples 77

97 (A, A3 th problems n edge connectvty, the score s slghtly loer than that of nondefectve one. Hoever, sample A6 s msclassfed as a defectve sample as the score s loer than. For 2m and 25m solder paste samples, the score dfference beteen them s dened. 25m solder paste samples have a score loer than -.45 hle the score of 2m solder paste samples s beteen and Experment 3 The fuzzy system n experment 2 has the problem of msclassfyng the solder paste sample A6 as defectve snce a lo score s gven to t. To further mprove the performance of the system, sample A6 s added nto the tranng set and the rule eghtng s changed. In the group of negatve contrbuton rules, the eghtng for ule 7 s tce of other rules n the same group. The rule eghtngs and the tranng data set n ths experment are as follos: Postve Contrbuton Negatve Contrbuton ule No Weghtng..... Table 5.8 ule eghtngs for the fuzzy system n experment 3 78

98 No. Score n Desred Score Explanaton Experment 5m A Same as n experment 2 A2 6.9 Same as n experment 2 A Same as n experment 2 A For score adjustment of non-defectve solder paste A Same as n experment 2 A Same as n experment 2 2m B Same as n experment 2 B Same as n experment 2 B Same as n experment 2 B Same as n experment 2 25m C Same as n experment 2 C Same as n experment 2 C Same as n experment 2 C Same as n experment 2 C Same as n experment 2 Table 5.9 Tranng Solder Paste Block Samples of Experment 3 Usng the ntal membershp parameters n experment, the system s traned th the above tranng data. Then all samples are presented to the adapted fuzzy system for testng. The result s shon as follos: 79

99 No. Old Score Ne Score Comment 5m A Correct judgment for defect: edge connectvty A Correct A Correct judgment for defect: edge connectvty A Correct A Correct judgment for defect: excess solder A Correct A Correct judgment for defect: block shftng A Correct 2m B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct 25m C Correct C Correct C Correct C Correct C Correct C Correct C Correct C Correct C Correct Table 5. Qualty Score of Solder Paste Block Samples After Tranng 8

100 u u 2 u 3 u 4 5 t 5 t e 5 t n 5 t s u 5 u 6 u 7 u 8 5 e 5 e e 5 e n 5 e s u 9 u u u 2 5 s c 5 s s 5 t 5 t e u 3 u 4 u 5 u 6 5 t n 5 t s 5 e 5 e e u 7 u 8 u 9 u 2 5 e n 5 e s 5 s c 5 s s Fg. 5. Membershp functons of the fuzzy system after tranng n experment 3 Non-Defectve 5m 2m 25m Defectve Fg. 5.2 Score dstrbuton of solder paste samples after tranng n experment 3 Wth the ne set of tranng data, the fuzzy system can classfy sample A6 as nondefectve solder paste sample. The scores of 5m non-defectve solder paste sample are beteen.664 and.6945 hle defectve ones are belo ths range. esults for 2m and 25m samples are smlar to that of experment 2. So the ne set of tranng data 8

101 together th the changes n rule eghtng mproves the performance of the fuzzy system successfully Experment 4 In ths tranng experment, all the solder paste samples n Table 5.4 are used for the tranng of the fuzzy system n Experment. The assgnments of desred scores are based on the results of Experment 2 and Experment 3. The tranng data set s lsted n Table 5.. The result of usng all the solder paste samples for tranng s smlar to that n Experment 3. The fuzzy system can successfully gve a correct score to all the samples. The score of each solder paste sample after tranng s summarzed n Table

102 No. Score n Experment Desred Score 5m A A2 6.9 A A A A A A m B B B B B B B B B B B B B m C C C C C C C C C Table 5. Tranng Solder Paste Block Samples of Experment 4 83

103 No. Old Score Ne Score Comment 5m A Correct A Correct A Correct A Correct A Correct A Correct A Correct A Correct 2m B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct B Correct 25m C Correct C Correct C Correct C Correct C Correct C Correct C Correct C Correct C Correct Table 5.2 Qualty Score of Solder Paste Block Samples After Tranng 84

104 u u 2 u 3 u 4 5 t 5 t e 5 t n 5 t s u 5 u 6 u 7 u 8 5 e 5 e e 5 e n 5 e s u 9 u u u 2 5 s c 5 s s 5 t 5 t e u 3 u 4 u 5 u 6 5 t n 5 t s 5 e 5 e e u 7 u 8 u 9 u 2 5 e n 5 e s 5 s c 5 s s Fg. 5.3 Membershp functons of the fuzzy system after tranng n experment 4 Non-Defectve 5m 2m 25m Defectve Fg. 5.4 Score dstrbuton of solder paste samples after tranng n experment 4 85

105 Experment 5 In ths experment, a set of 69 solder paste samples are used to evaluate the performance of the fuzzy system n Experment 4. It contans three dfferent thcknesses, 5m, 2m and 25m. For each thckness, there are 23 solder paste samples hch nclude 2 non-defectve samples and defectve samples. Defectve samples contan defects of Insuffcent Solder, Excess Solder, Bad Edge Connectvty and Brdgng. All the testng samples are nputted to the traned fuzzy system n Experment 4. The detecton results are gven n Table 5.3 and Fg. 5.5 Sample No. Score Comment 5m TA25 TA26 TA27 TA28 TA29 TA3 TA3 TA32 TA33 TA34 TA35 TA36 TA5 TA55 TA56 TA57 TA58 TA59 TA6 TA6 TA62 TA63 TA No Defect.462 No Defect.35 No Defect.7685 No Defect 429 No Defect.42 No Defect 5 No Defect.2845 No Defect.3866 No Defect 56 No Defect 846 No Defect.425 No Defect -.58 Insuffcent Solder -923 Excess Solder -446 Excess Solder Insuffcent Solder Bad Edge Connectvty -93 Excess Solder.266 Bad Edge Connectvty.684 Brdgng Brdgng -.42 Brdgng -.68 Brdgng 86

106 2m TB25 TB26 TB27 TB28 TB29 TB3 TB3 TB32 TB33 TB34 TB35 TB36 TB5 TB55 TB56 TB57 TB58 TB59 TB6 TB6 TB62 TB63 TB64 25m TC25 TC26 TC27 TC28 TC29 TC3 TC3 TC32 TC33 TC34 TC35 TC36 TC5 TC No Defect No Defect.2274 No Defect.2379 No Defect.923 No Defect.96 No Defect.2298 No Defect.722 No Defect.73 No Defect.244 No Defect.835 No Defect.889 No Defect Insuffcent Solder -339 Excess Solder Excess Solder Insuffcent Solder.789 Bad Edge Connectvty Excess Solder.64 Bad Edge Connectvty -.36 Brdgng Brdgng -.26 Brdgng Brdgng No Defect No Defect -355 No Defect No Defect No Defect No Defect -423 No Defect -.2 No Defect No Defect -393 No Defect No Defect -33 No Defect.694 Insuffcent Solder Excess Solder 87

107 TC56 TC57 TC58 TC59 TC6 TC6 TC62 TC63 TC64-26 Excess Solder -.7 Insuffcent Solder Bad Edge Connectvty -.68 Excess Solder -.66 Bad Edge Connectvty -525 Brdgng Brdgng Brdgng -.49 Brdgng Table 5.3 Qualty Score of Testng Solder Paste Block Samples Usng Fuzzy System n Experment 4 Non-Defectve 5m 2m 25m Defectve Fg. 5.5 Score dstrbuton graph of testng solder paste samples The score dstrbuton graph (Fg. 5.5 shos that all the defectve solder paste samples can be successfully dentfed as defectve solder paste block by the traned fuzzy system. These defectve samples have a score range sgnfcantly loer than non-defectve 5m solder paste samples (.2845 to Therefore, the traned fuzzy system acheves the am of defect detecton. For non-defectve samples th thckness of 2m, ther score range (-.672 to.244 s sgnfcantly dfferent from that of 5m block (.2845 to The same results are gven for 25m solder paste blocks. The score range for 25m blocks s -393 to 88

108 -.2. Ths range s much loer than that of 5m non-defectve samples. These results sho that the traned fuzzy system can classfy solder paste samples th dfferent thcknesses nto dfferent categores. As a result, the fuzzy system can be able to detect solder paste samples th ncorrect prntng thcknesses. esults from ths experment sho that the proposed fuzzy system can be used for detectng defects n the solder paste blocks. In addton, t can detect any ncorrect prntng thcknesses durng the SMT process. Wth proper tranng and setup, the fuzzy system can provde hgh detecton accuracy. 89

109 5.9.2 ectangular Solder Paste Block Apart from square-shaped solder paste block used n Secton 5.9., rectangular solder paste block shon n Fg. 5.6(a s commonly used n the PCB manufacturng. Ths type of pad s usually used by chps th Quad Flat Package as shon n Fg. 5.6(b. It has a thn dth and a long length. The sze s smaller than the square-shaped solder paste block. The small sze makes the nspecton process more dffcult to accomplsh. (a (b Fg. 5.6 (a Typcal rectangular solder paste blocks and (b quad flat package chp Under the proposed drectonal LED lghtng, a typcal non-defectve solder paste block has the appearance shon n Fg. 5.7(a and an example of defectve solder paste block s shon n Fg. 5.7(b. 9

110 (a (b Fg. 5.7 Appearance of (a non-defectve and (b defectve rectangular solder paste block under proposed drectonal lghtng Expermental Setup and Procedures A fuzzy system for the nspecton of rectangular solder paste block s frst set up. The setup of the fuzzy system ncludes the assgnment of scalng functons for the normalzaton of extract features as shon n Secton 5.3. The assgnment of the scalng functons s based on the reference samples of ths type of rectangular block. The scalng functons n ths experment are shon n the follong table: 9

111 Feature Nomnal ange Scalng Functon West Edge Thckness ( t East Edge Thckness ( t e North Edge Thckness ( t n South Edge Thckness ( t s 3 pxels 5 45 pxels 3 pxels 3 pxels West Edge Thckness Devaton ( d pxels East Edge Thckness Devaton ( d pxels e t 3 t e 5 3 t n 3 t s 3 North Edge Thckness Devaton ( d South Edge Thckness Devaton ( d n s pxels pxels d n d s West Edge Pxels Connectvty ( East Edge Pxels Connectvty ( e North Edge Pxels Connectvty ( n South Edge Pxels Connectvty ( s West Edge egon Surface Area ( e East Edge egon Surface Area ( e e North Edge egon Surface Area ( e n South Edge egon Surface Area ( e s Central egon Surface Area ( s c Central egon Defect Area ( d Solder Paste Block Shftng ( b Solder Paste Surface Area ( s s c 2 gaps c 5 c 2 gaps c 5 c 2 gaps c 5 c 2 gaps c 5 pxels 2 pxels 3 pxels 3 pxels pxels 4 pxels pxels 5 5 pxels e n s e e e 2 e 2 e s 2 s c 25 5 d 4 b s s 5 Table 5.4 Scalng equatons of extracted features for rectangular solder paste block 92

112 As the sze and the dmenson of ths type of solder paste block are dfferent from the one n Secton 5.9., most of scalng functons are adjusted to reflect the characterstcs of the ne solder paste type. Among these scalng functons, functons for the West Edge and East Edge Thckness Devaton are set to zero. The consequence s that these to features are excluded from the nspecton process. Ths s because the est and east edges of the solder paste block have very short edge length as shon n Fg The thckness devaton along ther length does not gve a good measurement on the solder paste prntng qualty. On the other hand, the north and south edges have a suffcent length. Therefore, only north and south edge thckness devatons are gven to the fuzzy system for qualty judgment. Next, the ntal parameters of the nput membershp functons are set based on some reference solder pastes samples so that the qualty score for reference samples tends to. The ntal parameters of the nput membershp functons are shon n the follong table: a b c Table 5.5 Intal parameters of membershp functons for rectangular solder paste block d r s 93

113 u u 2 u 3 u 4 5 t 5 t e 5 t n 5 t s u 5 u 6 u 7 u 8 5 e 5 e e 5 e n 5 e s u 9 u u u 2 5 s c 5 s s 5 t 5 t e u 3 u 4 u 5 u 6 5 t n 5 t s 5 e 5 e e u 7 u 8 u 9 u 2 5 e n 5 e s 5 s c 5 s s Fg. 5.8 Membershp functons th ntal parameters from Table.4 The rule eghtng for postve and negatve contrbuton rules are summarzed n Table 5.6. The eghtngs are normalzed to values beteen and. Postve Contrbuton Negatve Contrbuton ule Weghtng..... Table 5.6 ule eghtngs for nspectng rectangular solder paste block Then solder paste samples, ncludng defectve and non-defectve samples, are chosen to form a tranng set. The tranng set s used for fne tunng the performance of the fuzzy system. Fnally, testng samples are presented to the traned fuzzy system to test the detecton accuracy. 94

114 Expermental esults There are 94 solder paste samples used n ths experment. Among them, 59 samples do not contan any defect and 35 samples have defect on them. All the samples are of the same prntng thckness. 45 out of 94 samples are chosen to form a tranng set. In the tranng set, there are 25 non-defectve samples and 2 defectve samples. Defects nclude excess solder, nsuffcent solder and edge connectvty problem. For the remanng 49 solder paste samples, they are used as a testng set to test the nspecton accuracy of the traned fuzzy system. After tranng, the membershp functons of the fuzzy system are adapted. They are summarzed n the follong fgure: u u 2 u 3 u 4 5 t 5 t e 5 t n 5 t s u 5 u 6 u 7 u 8 5 e 5 e e 5 e n 5 e s u 9 u u u 2 5 s c 5 s s 5 t 5 t e u 3 u 4 u 5 u 6 5 t n 5 t s 5 e 5 e e u 7 u 8 u 9 u 2 5 e n 5 e s 5 s c 5 s s Fg. 5.9 Membershp functons after tranng 49 samples are used to test the traned fuzzy system. The score dstrbuton graph of the testng results s as follos: 95

115 Non-Defectve Defectve Fg. 5.2 Score dstrbuton graph of testng solder paste samples The testng results sho that most of defectve solder paste samples are separated from the non-defectve solder paste group. Defectve solder paste samples have score belo.3 and non-defectve ones have score above.3. Among the 49 samples, to defectve samples are msclassfed as non-defectve samples. Also, one non-defectve sample has a score around.3 hch s the decson boundary. Ths may gve ambguty n the nspecton result. Apart from them, the traned fuzzy system gves correct nspecton result. Therefore, the nspecton accuracy n ths experment s 93.88%. 96

116 5.9.3 Crcular Solder Paste Block Apart from square-shaped and rectangular solder pads, crcular pads are often used n the PCB. Ths type of pads s crcular n shape as shon n Fg After solder paste prntng, the solder paste depost s crcular n shape as shon n Fg Fg. 5.2 Typcal crcular pads Fg Typcal crcular solder paste block 97

117 Expermental Setup and Procedures Smlar to prevous sectons, a fuzzy system s set up for the nspecton of crcular solder paste block. Snce the sze and the dmenson are dfferent from square-shaped and rectangular type, adjustments on the scalng functons are needed. The adjustments are based on the reference crcular solder paste block. The scalng functons am at normalzng the nomnal range of features to the range beteen and. The scalng functons for crcular solder paste block are shon n the follong table: Feature Nomnal ange Scalng West Edge Thckness ( t East Edge Thckness ( t e North Edge Thckness ( t n South Edge Thckness ( t s West Edge Thckness Devaton ( d 25 pxels 25 pxels 25 pxels 25 pxels pxels Functon t 2.5 t e 2.5 t n 2.5 t s 2.5 d East Edge Thckness Devaton ( d North Edge Thckness Devaton ( d South Edge Thckness Devaton ( d West Edge Pxels Connectvty ( c East Edge Pxels Connectvty ( c e North Edge Pxels Connectvty ( c n South Edge Pxels Connectvty ( c s West Edge egon Surface Area ( e East Edge egon Surface Area ( e e e n s pxels pxels pxels gaps gaps gaps gaps 4 pxels 4 pxels d e d n d s c c e c n c s e 4 e e 4 98

118 North Edge egon Surface Area ( e n South Edge egon Surface Area ( e s Central egon Surface Area ( s c Central egon Defect Area ( d Solder Paste Block Shftng ( b Solder Paste Surface Area ( s s 4 pxels 4 pxels 2 5 pxels 4 pxels pxels 5 2 pxels e 4 e s 4 s c 2 3 d 4 b s s 5 Table 5.7 Scalng functons for crcular solder paste block Based on non-defectve samples of crcular solder paste block, ntal parameters of the membershp functons are assgned. The parameters and the shape of the membershp functons are shon n Table 5.8 and Fg a b c Table 5.8 Intal membershp functons parameters for crcular solder paste block d r s 99

119 u u 2 u 3 u 4 5 t 5 t e 5 t n 5 t s u 5 u 6 u 7 u 8 5 e 5 e e 5 e n 5 e s u 9 u u u 2 5 s c 5 s s 5 t 5 t e u 3 u 4 u 5 u 6 5 t n 5 t s 5 e 5 e e u 7 u 8 u 9 u 2 5 e n 5 e s 5 s c 5 s s Fg Membershp functons for crcular solder paste block All fuzzy rules are assgned th the same rule eghtngs. The fuzzy system s then tested aganst a set of crcular solder paste samples Expermental esults A set of 9 crcular solder paste samples s used to test the accuracy of the fuzzy system. The testng set ncludes 6 defectve samples and 3 non-defectve samples. It s nputted the fuzzy system and the results are shon n the follong score dstrbuton graph.

120 Non-Defectve Defectve Fg Score dstrbuton graph for testng crcular solder paste block esults sho that the fuzzy system can classfy non-defectve samples and defectve samples nto to dfferent categores. Non-defectve samples have scores hgher than and defectve samples have scores loer than. Ths experment shos that the proposed fuzzy system can be used to nspect crcular solder paste block. Wth proper setup of ntal membershp functons parameters and the rule eghtngs, the fuzzy system can ork effectvely on crcular solder paste block th hgh nspecton accuracy. If fne-tunng of the system s needed, neural-fuzzy adaptaton can be done usng a set of tranng samples Dscusson The proposed fuzzy system has been evaluated th solder paste blocks of dfferent shapes and thcknesses. In the evaluaton of square-shaped solder paste blocks, the proposed fuzzy system s tested aganst solder paste blocks th dfferent thcknesses. The processng tme for samples s about.2 second. esults sho that t can classfy solder paste blocks th dfferent thcknesses nto dfferent categores. Also, the fuzzy system successfully dstngushes defectve blocks from the testng samples. In

121 addton, the neural-fuzzy tranng can help to fne-tune the behavor of the fuzzy system accordng the users preferences. Experment results sho that the fne-tuned fuzzy system gves a hgh accuracy n the defect detecton of solder paste blocks. Apart from square-shaped solder paste blocks, the fuzzy system s evaluated th rectangular and crcular solder paste blocks. Wth adjustments n the fuzzy system setup, the proposed approach also gves consstent results n these to types of solder paste blocks. 5. Summary The desgn and evaluaton of the proposed fuzzy system for solder paste qualty scorng are dscussed n ths chapter. Intally, the fuzzy system s set up usng a set of reference solder paste samples. Next, a number of features are extracted from the processed mages and are used as nput of the fuzzy system. The system calculates the score of the testng solder paste block usng a set of fuzzy rules. If the output score does not meet the users preference, the fuzzy system can be adapted usng a neural-fuzzy tranng algorthm. The fne-tuned fuzzy system ll ork closer to the users expectaton. In ths chapter, the proposed fuzzy system s evaluated th dfferent types of solder paste samples. Expermental results sho that t can classfy solder paste blocks of dfferent thcknesses nto dfferent categores and dstngushes defectve blocks from the testng samples th hgh accuracy. In addton, the fne-tunng of the system usng neural-fuzzy tranng can successfully adjust the system accordng to the users preferences. 2

122 CHAPTE 6 Shape Estmaton System 6. Introducton In ths chapter, a novel shape estmaton algorthm s presented. It conssts of to parts: shape estmaton and parameters optmzaton. The shape estmaton algorthm s used to estmate the 3D shape of the solder paste block. A set of algorthm parameters are passed to the algorthm to perform the 3D shape estmaton. The set of algorthm parameters ncludes the processed 2.5D mages acqured by the dgtal camera n Chapter 4 and the physcal propertes of solder paste blocks, such as slope of edges, etc., hch are obtaned from reference samples. Ths s carred out by calculatng a set of surface heghts n the vertcal and horzontal drectons. Evaluaton on the accuracy of the algorthm s carred out by fndng the average absolute heght dfference beteen the estmated shape and the actual shape. 6.2 Descrpton of the algorthm The proposed shape estmaton algorthm makes use of the follong parameters to estmate the shape of the solder paste block. Parameter To processed top-veed mages of the solder paste block under sde north-south and sde east-est LED llumnaton patterns Parameter 2 Parameter 3 Parameter 4 Parameter 5 Parameter 6 Slope of East Edge (mcron/pxel Slope of West Edge (mcron/pxel Slope of North Edge (mcron/pxel Slope of South Edge (mcron/pxel Wdth lmt of East Edge (pxel 3

123 Parameter 7 Parameter 8 Parameter 9 Parameter Wdth lmt of West Edge (pxel Wdth lmt of North Edge (pxel Wdth lmt of South Edge (pxel Upards slope of mddle secton n north-south llumnaton pattern (mcron/pxel Parameter Upards slope of mddle secton n east-est llumnaton pattern (mcron/pxel Assumpton:. PCB s perfectly horzontal hen the solder paste s prnted on t. 2. The north, south, east and est margns of the solder paste block are of heght equals to zero. Parameter s the processed mages th the deduced geometrcal nformaton of the solder paste block from Chapter 4. Parameters 2- are physcal propertes of the solder paste block. For dfferent type of solder paste block, a dfferent set of algorthm s used for estmaton. The choce of parameters 2- are based on the results of processed mages from Chapter 4 and the general shape of solder paste block. In general, as dscussed n Chapter 4, a nondefectve solder paste block has nclned regons at the edges and flat regon at the mddle secton of the block. To estmate the solder paste shape, slopes of the nclned regon at the edges need to be assgned. As the desgned drectonal lghtng are from the north, south, east and est drectons, the set of algorthm contans four dfferent slopes (Parameters 2-5 for the estmaton of edge nclned regons. 4

124 For each edge regon, a dth lmt (Parameters 6-9 s needed to avod the overestmaton of the edge heght. Ths s because defects n the center part of the solder paste block may connect to the edge regons. If there s no dth lmt for edges, the dth of edge regon ll be over-estmated and thus the heght. For the mddle secton, any nclned regons due to defects ll be hghlghted by the desgned lghtng and ll appear as blue or red regons. To estmate the shape of the defects, slope values need to be assgned to these regons n the mddle secton. Therefore, Parameter and are responsble for the estmaton of the defect shapes contaned n the mddle secton. Only the upards slopes are nput as parameters and the respectve donards slope values are calculated based on the rato of red to blue pxels n the mddle secton and the heght of slopes at to ends. The detals are gven n the algorthm procedures. The detaled procedures of the shape estmaton algorthm are as follos:. To processed mages of sde east-est and north-south lghtng are used as the nput for ths algorthm. Three examples of processed mage par of solder paste are shon n Fg. 6.. The frst example s a solder paste block thout defect. The remanng to examples are solder paste blocks th defects of nsuffcent solder and excess solder respectvely. 5

125 Wthout defect (a (b Wth defect (Insuffcent solder (c (d Wth defect (Excess solder (e Fg. 6. Examples of processed solder paste mages (f 2. o-by-ro processng s carred out for east-est LED llumnaton pattern. Each ro of pxels s frst segmented nto red regon, black regon and blue regon. Pxels are processed from est to east. As blue lght comes from the est sde of the mage, blue regon s treated as regon of upards slopng (denoted by U n Fg On the other hand, as red lght comes from the east sde of the mage, red regon s treated as regon of donards slopng denoted by D. The remanng black regon s treated as regon of almost horzontal (denoted by H. Fg. 6.2 shos the regons th dfferent slopng based on processed mages of Fg

126 U H D U H D H U H D U H U H D Wthout defect (Not to scale Wth defect (Insuffcent solder Wth defect (Excess solder Fg. 6.2 Inclned regons of solder paste blocks based on processed mage n Fg After a segmentaton of regon for each ro of pxels, a sequence of U, D and H regons are formed. The frst non-horzontal regon from the est should be an U regon. Hence, from the est of the sequence, the regons are treated as H before the U regon s encountered. Smlarly, the frst non-horzontal regon from the east should be a D regon. Therefore, the regons are treated as H before the frst D regon s encountered. 4. Next, the dth lmt of the east and est edges are appled. If the dth of the edge exceeds ts lmt, t ould be dvded nto to parts. The frst part s the edge th dth equals to the lmt. The remanng pxels ould be treated as part of the mddle secton. 7

127 5. The frst U regon on the est s then treated as the est edge of the solder paste block. Based on the nput slope and the physcal dmenson of the est edge, the heght (h W s calculated as follos: h W = dth of est edge slope of est edge (6. Smlarly, the frst D regon on the east s treated as the east edge. The heght of the east edge (h E s calculated as follos: h E = dth of east edge slope of east edge (6.2 The heght of each pxel on the est and east edge s assumed to vary lnearly beteen zero and h W and h E respectvely. Next s on the shape estmaton of the mddle secton. h W and h E are the startng and endng heghts of the mddle secton respectvely. The postons of all the regons are summarzed n Fg h E h W h E h W h E h W West Edge Mddle Secton East Edge West Edge Mddle Secton East Edge West Edge Mddle Secton East Edge (Not to scale Wthout defect Wth defect (Insuffcent solder Fg. 6.3 Example postons of dfferent regons Wth defect (Excess solder 8

128 6. Dependng on hether there are U and D regons n the mddle secton, there are to cases. Case (A. For a defect-free solder paste block, the mddle secton should contan only one H regon. The slope of the mddle secton s calculated usng the follong equaton based on h W and h E : slope of mddle secton (um/px he hw = (6.3 dth of mddle secton The heght of each pxel n the mddle secton ould vary lnearly beteen h W and h E. Case (B. For a solder paste block th defect, t should contan U and D regons n the mddle secton. a. The donards slope of the defect regon s calculated usng the follong equaton: Donards slope of mddle secton h = W α number of pxels n mddle secton U regons h number of pxels n mddle D regons here α s the upards slope of mddle secton n east-est llumnaton E (6.4 (Note that alpha s one of the algorthm parameters. b. Processng from est to east usng h W as the startng heght, any U regon pxel ncreases the heght accordng to the slope α. Smlarly, any D regon pxel decreases the heght accordng to the donards slope of the mddle secton calculated n the prevous step. The remanng H regon pxels do not affect the heght. 9

129 7. After processng all the ros of pxels, all the estmated heghts are grouped to form a heght map due to east-est llumnaton pattern. 8. Column-by-column processng s carred out for north-south LED llumnaton pattern. Step 2-7 are repeated for north-south llumnaton pattern. The processng drecton s from north to south nstead of from est to east. Then a heght map, hch s caused by north-south llumnaton, s then formed. 9. The heght maps of the to llumnaton patterns are combned by carryng out an O operaton n the to heght maps. In the O operaton, the larger heght value n the same poston of the to heght maps s chosen as the fnal heght value at that poston.. As the to heght maps are calculated from LED llumnatons hch are perpendcular to each other, the combned heght map n step 9 s then smoothed usng a dsk shape flter for the nterpolaton of heght data n other drectons.. Fg. 6.4(a shos the reconstructed shape of a solder paste thout defect. Ths result s obtaned from an evaluaton of the developed algorthm. The actual heght profle of the same solder paste block as shon Fg. 6.4(b as obtaned from the 3D scanner. The estmated shapes of solder paste th excess and nsuffcent solder are shon n Fg. 6.5 and Fg. 6.6 respectvely.

130 (m (m (mm (mm (mm (mm (a Estmated Shape (b eference Shape Fg. 6.4 Example of non-defectve solder paste block (m (m (mm (mm (mm (mm (a Estmated Shape (b eference Shape Fg. 6.5 Example of defectve solder paste block th nsuffcent solder (m (m (mm (mm (mm (mm (a Estmated Shape (b eference Shape Fg. 6.6 Example of defectve solder paste block th excess solder

131 The algorthm can be summarzed n Fg Upard slope of Mddle Secton for east-est llumnaton Slope of West Edge Slope of East Edge Wdth Lmt of West Edge Wdth Lmt of East Edge Processed East-West Lghtng Image Processed North-South Lghtng Image Slope of North Edge Slope of South Edge Wdth Lmt of North Edge Wdth Lmt of South Edge Upard slope of Mddle Secton for north-south llumnaton o-by-ro processng for segmentaton nto U, D and H regons Column-by-column processng for segmentaton nto U, D and H regons Calculate West Edge and East Edge Calculate North Edge and South Edge Wth Defect Wthout Defect Wthout Defect Wth Defect Calculate Donard Slope of Mddle Secton Calculate Slope of Mddle Secton Calculate Slope of Mddle Secton Calculate Donard Slope of Mddle Secton Calculate all the pxel heghts on the ro Calculate all the pxel heghts on the column Combnaton of to heght maps Dsk shape flterng of heght map Estmated Heght map 2

132 6.3 Algorthm Parameters Intal algorthm parameters 2- are found from reference solder paste blocks th the same shape and prntng thckness. The actual shapes of reference solder paste blocks are frst acqured by a laser sensor. Then algorthm parameters, such as slope of edges and dth lmt of edges, are found by measurng the physcal dmenson of the actual shapes. Wth the ntal set of parameters, the shape estmaton algorthm ould calculate an estmated shape. Then, the estmated shape s compared th the reference shape by calculatng the average absolute heght dfference beteen them. Parameter optmzaton can be further carred out to fnd the optmal algorthm parameters for a solder paste block. Ths s done by mnmzng the average absolute heght dfference usng an optmzaton algorthm such as the smplex search algorthm. The block dagram for the optmzaton process s shon n Fg Optmzaton usng smplex search algorthm Changes Slope of North Edge Slope of South Edge Slope of West Edge Slope of East Edge Wdth Lmt of North Edge Wdth Lmt of South Edge Wdth Lmt of East Edge Wdth Lmt of West Edge Upard slope of Mddle Secton for north-south llumnaton Upard slope of Mddle Secton for east-est llumnaton Processed East-West Lghtng Image Processed North-South Lghtng Image Shape econstructon Algorthm Estmated Heght Map Average Heght Dfference eference Heght Map Fg. 6.8 Block dagram for the parameter optmzaton process 3

133 Fnally, a generalzed set of algorthm s found by calculatng the averages of algorthm parameters among reference solder paste blocks. For a partcular solder paste type, there are to sets of generalzed algorthm parameters. One s for defectve samples and the other one s for non-defectve samples. 6.4 Experments Solder paste samples are selected to test the proposed shape estmaton system. Images of the selected samples are captured by the dgtal camera under the proposed lghtng n Chapter 3. The captured mages are processed as descrbed n Chapter 4. Then the processed mages are nputted to the shape estmaton system. Wth a set of algorthm parameters found from reference solder paste shapes, the shape estmaton system gves the estmated shapes of the solder paste blocks as output. The estmated shapes are compared to ts correspondng reference shapes. The average absolute heght dfference beteen the reference shape and the estmated shape s used for comparson. In addton, the reference shape s used for optmzng the nput parameters of the shape estmaton algorthm to mnmze the average heght dfference beteen estmated and reference shape eference Shape The reference shape of solder paste block s acqured by the laser dsplacement sensor from Keyence Corporaton. The laser dsplacement sensor s based on the prncple of laser trangulaton to measure the heght of an object. The specfcaton of the laser sensor s as follos: 4

134 Manufacturer Model Measurement ange Keyence LK-G3 3 ± 5mm Laser Beam Dameter 3m Wavelength esoluton Lnearty 65nm.m.5% of F.S. (F.S. = 5mm Table 6. Specfcaton of the laser dsplacement sensor Keyence LK-G3 s a pont source laser sensor. It can only measure the object thckness at one pont for each measurement cycle. To get the hole shape of the solder paste block, the laser sensor s mounted onto an X-Y table so that t can move across the solder paste surface. Therefore, the hole surface of the object s measured pont-by-pont. All measurements are transferred to a computer for veng. The reference shape can then be found by puttng all the pont measurements together to form a heght map of the solder paste block. Fg. 6.9 shos the experment setup used for acqurng reference solder paste shape and Fg. 6. shos the reference shape of a 2m crcular solder paste block acqured by the setup. 5

135 Fg. 6.9 Expermental setup for acqurng reference solder paste shape (mm (mm (mm Fg. 6. The shape of a 2m crcular solder paste block acqured by the laser sensor 6

136 6.4.2 Square-shaped Solder Paste Block Square-shaped solder paste blocks th 2m thckness are used to test the proposed shape estmaton algorthm. To sets of generalzed algorthm parameters are found from reference non-defectve solder paste block and reference defectve solder paste block. They are then used to test varous solder paste samples and the accuracy of the proposed algorthm s gven n average absolute heght dfference n the solder paste surface beteen the estmated and the actual shape Generalzed Parameters The generalzed non-defectve algorthm parameters for square-shaped solder paste are based on the algorthm parameters of fve reference solder paste blocks. After measurng parameters from ther actual shape, they are used to fnd the average parameters to form the generalzed shape. Among all the parameters, Parameters and cannot be measured from the reference solder paste blocks as they are the defect slopes n the mddle secton of the solder paste block. Usng the heurstc that non-defectve solder paste block does not have steeply nclned regon n the mddle secton, the to defect slopes are set to a lo value hch s about one-tenth of edge slopes. The measured algorthm parameters from the fve reference non-defectve solder paste samples and the generalzed algorthm parameters are shon n the follong table: 7

137 Sample Parameter m/pxel m/pxel m/pxel m/pxel pxel pxel pxel pxel m/pxel m/pxel Generalzed (Non-defectve Generalzed (Defectve Table 6.2 Algorthm parameters for square-shaped solder paste blocks For defectve solder paste samples, another set of algorthm parameters called generalzed defectve algorthm parameters s used. Ths set of the algorthm parameter s based on the non-defectve algorthm parameters of the same type of solder paste blocks. The only dfference s that the defect slopes of the mddle secton (Parameter and are assgned th value comparable to the edge slopes nstead of a small value. As defect often occurs n the mddle secton of the solder paste block, the generalzed defectve algorthm parameters can estmate the shape of defect usng the defect slopes of the mddle secton Experment In ths experment, a non-defectve solder paste sample s used to test the accuracy of the shape estmaton algorthm. The actual shape of the testng solder paste block acqured by the laser sensor s shon n the follong fgure: 8

138 (m (mm (mm Fg. 6. The actual shape of the testng 2m square-shaped solder paste block acqured by the laser sensor (m (a (b Fg. 6.2 (a Orgnal and (b processed mages of the testng 2m square-shaped solder paste block 9

139 Wth the proposed drectonal lghtng, the nclned regons of the testng solder paste block are hghlghted as shon n Fg. 6.2(a. Usng the generalzed non-defectve algorthm parameters found n Secton and the processed solder paste mages shon n Fg. 6.2(b, the shape estmaton algorthm gves the follong estmated shape: (m (mm (mm (m Fg. 6.3 The estmated shape of the testng 2m square-shaped solder paste block usng generalzed non-defectve algorthm parameters The surface of the estmated solder paste block s compared th ts actual shape. The average absolute heght dfference for the solder paste surface s 3.43m. Therefore, the error of the shape estmaton algorthm for ths sample s 3.43m/2m = 6.72%. The accuracy of the shape estmaton algorthm can be further mproved by optmzng the algorthm parameters th the actual solder paste shape. The optmzed parameters are the optmal parameters for a partcular solder paste block. The optmzaton s carred out by a smplex search algorthm provded by the Matlab functon fmnsearch(. The optmzed parameters are shon n Table 6.3 and the estmated shape s shon n Fg

140 2 (m/pxel 3 (m/pxel 4 (m/pxel 5 (m/pxel Algorthm Parameter 6 (pxel 7 (pxel 8 (pxel 9 (pxel (m/pxel (m/pxel Table 6.3 Optmzed algorthm parameters for testng solder paste block (m (mm (mm Fg. 6.4 The estmated shape of the testng 2m square-shaped solder paste block usng optmzed algorthm parameters (m Comparng th the actual shape done by the laser sensor, the average absolute heght dfference s 9.23m. The error of the algorthm becomes s 9.23m/2m = 4.62%. Wth the optmzed parameters, the proposed algorthm can estmate solder paste shape th hgher accuracy Experment 2 A solder paste sample th defect of excess solder s presented to the shape algorthm system for shape estmaton n ths experment. The actual shape of the testng solder paste block s shon n the follong fgure: 2

141 (m (mm (mm Fg. 6.5 The actual shape of the testng 2m square-shaped solder paste th defect of excess solder (m (a (b Fg. 6.6 (a Orgnal and (b processed mages of the testng 2m square-shaped solder paste block th defect of excess solder 22

142 Usng the generalzed defectve algorthm parameters found n Secton and the processed solder paste mages shon n Fg. 6.6, the shape estmaton algorthm gves the follong estmated shape: (m (mm (mm Fg. 6.7 The estmated shape of the testng 2m square-shaped solder paste block th defect of excess solder usng generalzed defectve algorthm parameters (m The estmated shape s compared th the actual shape and the average absolute heght dfference beteen them s gven by 3.78m. The error for the estmated shape usng the generalzed defectve algorthm parameters s 3.78m/2m = 5.89%. The optmal set of algorthm parameters for ths solder paste block can be found by mnmzng the average absolute heght dfference usng an optmzaton algorthm. Usng the smplex search algorthm, the follong set of optmzed algorthm parameters s found. 23

143 2 (m/pxel 3 (m/pxel 4 (m/pxel 5 (m/pxel Algorthm Parameter 6 (pxel 7 (pxel 8 (pxel 9 (pxel (m/pxel (m/pxel Table 6.4 Optmzed algorthm parameters for testng solder paste block th defect of excess solder Usng the above set of optmzed algorthm parameters, the shape estmaton algorthm gves the follong optmzed shape. (m (mm (mm Fg. 6.8 The estmated shape of the testng 2m square-shaped solder paste block th defect of excess solder usng optmzed algorthm parameters (m The average absolute heght dfference beteen the optmzed shape and the actual shape s 2.49m. The error of ths estmaton s 2.49m/2m = 6.25%. Ths shos that the shape estmaton algorthm can gve better results th the optmzed algorthm parameters. 24

144 Experment 3 The shape estmaton system s tested th a solder paste th nsuffcent solder n ths experment. The testng solder paste block has a small hole n the mddle of the block. The actual shape of the testng solder paste block s shon n Fg The orgnal and processed mages of the testng solder paste are shon n Fg. 6.2(a and Fg. 6.2(b respectvely. (m (mm (mm Fg. 6.9 The actual shape of the testng 2m square-shaped solder paste th defect of nsuffcent solder (m 25

145 (a (b Fg. 6.2 (a Orgnal and (b processed mages of the testng 2m squareshaped solder paste th defect of nsuffcent solder Usng the generalzed defectve algorthm parameters and the processed solder paste mages, the shape estmaton algorthm gves the follong shape: (m (m (mm (mm Fg. 6.2 The estmated shape of the testng 2m square-shaped solder paste th defect of nsuffcent solder usng generalzed defectve algorthm parameters 26

146 The average absolute heght dfference beteen the actual and the estmated shape s 47.76m. Therefore, the error of ths estmaton s 47.76m/2m = 23.88%. After parameter optmzaton, the average absolute heght dfference can be reduced to 22.68m. As a result, the error of ths estmaton s 22.68m/2m =.34%. The optmzed parameters are gven n Table 6.5 and the estmated shape th optmzed parameters s shon n Fg (m/pxel 3 (m/pxel 4 (m/pxel 5 (m/pxel Algorthm Parameter 6 (pxel 7 (pxel 8 (pxel 9 (pxel (m/pxel (m/pxel Table 6.5 Optmzed algorthm parameters for testng solder paste block th defect of nsuffcent solder (m (mm (mm Fg The estmated shape of the testng 2m square-shaped solder paste th defect of nsuffcent solder usng optmzed algorthm parameters (m 27

147 Experment 4 In ths experment, more square-shaped solder paste samples are used to test the shape estmaton algorthm. The prntng thcknesses of all the testng solder paste blocks are 2m. The testng non-defectve samples are estmated usng the generalzed nondefectve algorthm parameters. All the estmated shapes are compared th ther actual shape by calculatng the average absolute heght dfference. The expermental results are summarzed n the follong table: Sample Paste Type Average Absolute Heght Dfference Estmaton Error (m (% Non-defectve Non-defectve Non-defectve 2. 4 Non-defectve Non-defectve Non-defectve Non-defectve Non-defectve Table 6.6 Expermental results of experment 4 Among the eght testng samples n ths experment, the shape estmaton algorthm gves an average estmaton error of 4.5%. 28

148 6.4.3 ectangular Solder Paste Block The shape of rectangular solder paste block s smlar to that of square-shaped solder paste block. Same as square-shaped block, ths type of solder paste block has four edges. But to out of four edges of rectangular blocks are shorter than that of square-shaped block. The appearance of rectangular solder paste block under the proposed lghtng s shon n the follong fgure: Fg Appearance of rectangular solder paste block under the proposed lghtng As the dmenson of the rectangular solder paste blocks s dfferent from the squareshaped ones, a dfferent set of algorthm parameters s needed for the estmaton of the ne shape. Smlar to the testng n the prevous secton, the shape estmaton system s used to estmate the shape of rectangular solder paste samples usng algorthm parameters from the reference sample. The estmated shape s then compared th the actual shape scanned by the laser sensor by computng the average absolute heght dfference beteen them. 29

149 Experment In ths experment, a 5m rectangular solder paste block s used. A set of generalzed non-defectve algorthm parameters s found by measurng the physcal dmenson of the reference non-defectve solder paste block. The set of generalzed non-defectve algorthm parameters used n ths experment s shon n the follong table: 2 (m/pxel 3 (m/pxel 4 (m/pxel 5 (m/pxel Algorthm Parameter 6 (pxel 7 (pxel 8 (pxel 9 (pxel (m/pxel (m/pxel Table 6.7 Generalzed non-defectve algorthm parameters for 5m rectangular solder paste block The actual shape of the testng solder paste block s acqured usng the laser sensor and ts 3D plot s shon n the follong fgure: (m (m (mm (mm Fg Actual shape of the testng 5m rectangular solder paste block The testng solder paste block s put under the proposed lghtng. Images are captured and processed accordng to the method descrbed n Chapter 4. The orgnal and the processed solder paste mages are shon n the Fg. 6.25(a and Fg. 6.25(b respectvely. 3

150 (a (b Fg (a Orgnal and (b processed mages of the testng 5m nondefectve rectangular solder paste block Usng the processed solder paste mages and the generalzed non-defectve algorthm parameters, the shape estmaton algorthm gves the follong estmated shape: (m (m (mm (mm Fg The estmated shape of the testng 5m non-defectve rectangular solder paste block usng generalzed non-defectve algorthm parameters The average absolute heght dfference beteen the actual and estmated shape s 4.63m. As the prntng thckness of the testng solder paste s 5m, the error for the estmated shape s 4.63m/5m=9.75%. 3

151 Then parameter optmzaton usng the actual shape s carred out to fnd a set of optmal parameters for ths testng solder paste. The optmzed parameters and the estmated shape are gven n Table 6.8 and Fg respectvely. 2 (m/pxel 3 (m/pxel 4 (m/pxel 5 (m/pxel Algorthm Parameter 6 (pxel 7 (pxel 8 (pxel 9 (pxel (m/pxel (m/pxel Table 6.8 Optmzed algorthm parameters for 5m rectangular solder paste block (m (m (mm (mm Fg The estmated shape of the testng 5m non-defectve rectangular solder paste block usng optmzed algorthm parameters After optmzaton, the average absolute heght dfference beteen the actual and estmated shape s.26m. So the error of the estmaton usng optmzed parameters s.26m/5m=6.84%. Comparng to estmated shape usng generalzed parameters, the shape estmaton algorthm gves a more accurate shape. 32

152 Experment 2 In ths experment, the shape estmaton system s used to estmate the shape of a defectve rectangular solder paste block. The prntng thckness of the testng solder paste block s 5m and t has the defect of nsuffcent solder. The actual shape of the testng solder paste block s shon n the follong fgure: (m (m (mm (mm Fg Actual shape of the 5m defectve rectangular solder paste block Images of the solder paste block are captured under the proposed lghtng setup. They are then processed to extract features. The orgnal and processed mages of the testng solder paste block are shon n the follong fgure: 33

153 (a (b Fg (a Orgnal and (b processed mages of the testng 5m defectve rectangular solder paste block 2 (m/pxel 3 (m/pxel 4 (m/pxel 5 (m/pxel Algorthm Parameter 6 (pxel 7 (pxel 8 (pxel 9 (pxel (m/pxel (m/pxel Table 6.9 Defectve algorthm parameters for the defectve 5m rectangular solder paste block Usng the parameters gven n Table 6.9 and the processed mages n Fg. 6.29(b, the shape estmaton algorthm gves the follong estmated shape: (m (m (mm (mm Fg. 6.3 The estmated shape of the testng 5m defectve rectangular solder paste block usng parameters gven n Table

154 Comparng th the actual shape, the average absolute heght dfference s 35.43m. Then the error for ths estmaton s 35.43m/5m=23.62%. After parameters optmzaton, the average absolute heght dfference drops to 23.37m th estmaton error of 23.37m/5m=5.58%. The set of optmzed parameters and the estmated shape are shon n Table 6. and Fg. 6.3 respectvely. 2 (m/pxel 3 (m/pxel 4 (m/pxel 5 (m/pxel Algorthm Parameter 6 (pxel 7 (pxel 8 (pxel 9 (pxel (m/pxel (m/pxel Table 6. Optmzed defectve algorthm parameters for the defectve 5m rectangular solder paste block (m (m (mm (mm Fg. 6.3 The estmated shape of the testng 5m defectve rectangular solder paste block usng optmzed parameters gven n Table

155 6.4.4 Crcular Solder Paste Block The structure of the crcular solder paste block s dfferent from that of square-shaped and rectangular solder paste block. From Fg. 6.32, the hghlghted edge regons for crcular solder paste samples appear to be curved color cluster. Fg Appearance of crcular solder paste block under the proposed lghtng As the processng drectons of the shape estmaton algorthm are from north to south and est to east, parameters for edge slopes need to be adjusted. Ths s because the real edge dths of crcular solder paste block should be along the lne passng through the center of the block. Hoever, the processng drecton of the algorthm s not along the radus of the crcular block. Along the processng drecton, the edge slopes near the to ends of the edge are smaller than the mddle of the edges. Therefore, the follong equaton s used to adjust the slope of the crcular solder paste block: 36

156 k s = s x k m k s = s x 2 k m ( x m (6.5 ( x > m (6.6 here s and s are edge slope before and after adjustment respectvely k s the slope adjustment factor x s the poston along the edge length m s the mddle poston of the edge length After slope adjustment, s = s at the mddle poston of the edge. s decreases lnearly toards to ends of the edge. s becomes k hen t reaches the to ends of the edge along the length. Then the adjusted slopes are used for the shape estmaton. After the slopes are adjusted, the correspondng edge dth lmts are also adjusted. The follong equaton s used for the edge dth lmt adjustment: = k ( x m (6.7 x k m = k ( x > m (6.8 x 2 k m here and are edge dth lmts before and after adjustment respectvely k s the slope adjustment factor x s the poston along the edge length m s the mddle poston of the edge length 37

157 In the evaluaton of the crcular solder paste block,.6 s chosen as the slope adjustment factor. To crcular solder paste samples are used n the evaluaton process ncludng non-defectve and defectve sample. Smlar to prevous tests, the estmaton accuracy s gven n average absolute heght dfference Experment In ths experment, the shape estmaton system s tested usng a 5m non-defectve crcular solder paste. The actual shape s shon as follos: (m (mm (mm (m Fg Actual shape of the 5m non-defectve crcular solder paste block The algorthm parameters for estmaton, hch are found from the reference solder paste samples, are gven n Table (m/pxel 3 (m/pxel 4 (m/pxel 5 (m/pxel Algorthm Parameter 6 (pxel 7 (pxel 8 (pxel 9 (pxel (m/pxel (m/pxel Table 6. Algorthm parameters for the non-defectve 5m crcular solder paste block 38

158 Usng the proposed lghtng, mages of the testng crcular solder paste sample are captured and then processed before nputtng to the shape estmaton system. The orgnal and the processed solder paste mages are shon n the follong fgure: (a (b Fg (a Orgnal and (b processed mages of the testng 5m nondefectve crcular solder paste block The shape estmaton system gves the follong estmated shape for the above testng crcular solder paste sample. 39

159 (m (mm (mm Fg The estmated shape of the testng 5m non-defectve crcular solder paste block usng parameters gven n Table 6. (m The average absolute heght dfference beteen the above estmated shape and the actual shape s 2.48m. As the prntng thckness s 5m, the error for ths estmaton s 2.48m/5m=4.32%. Wth parameter optmzaton usng the actual shape, the accuracy of the estmated shape can be further mproved. Usng the optmzed parameters, the average absolute heght dfference drops to 2.76m. So the shape estmaton error s reduced to 2.76m/5m=8.5%. The optmzed parameter for ths testng crcular solder paste block and the estmated shape are shon n Table 6.2 and Fg (m/pxel 3 (m/pxel 4 (m/pxel 5 (m/pxel Algorthm Parameter 6 (pxel 7 (pxel 8 (pxel 9 (pxel (m/pxel (m/pxel Table 6.2 Optmzed algorthm parameters for the non-defectve 5m crcular solder paste block 4

160 (m (mm (mm Fg The estmated shape of the testng 5m non-defectve crcular solder paste block usng optmzed algorthm parameters (m Experment 2 After testng non-defectve crcular solder paste block, shape estmaton of defectve crcular solder paste block s tested. The testng solder paste block has defect of nsuffcent solder. The actual shape s as follos: (m (m (mm (mm Fg Actual shape of the 5m defectve crcular solder paste block 4

161 The follong algorthm parameters are used for shape estmaton: 2 (m/pxel 3 (m/pxel 4 (m/pxel 5 (m/pxel Algorthm Parameter 6 (pxel 7 (pxel 8 (pxel 9 (pxel (m/pxel (m/pxel Table 6.3 Algorthm parameters for the defectve 5m crcular solder paste block Under the proposed drectonal lghtng, the follong solder paste mages are captured and processed: (a (b Fg (a Orgnal and (b processed mages of the testng 5m defectve crcular solder paste block Wth the algorthm parameters n Table 6.3 and processed solder paste mages n Fg. 6.38(b, the follong shape s estmated by the shape estmaton algorthm: 42

162 (m (m (mm (mm Fg The estmated shape of the testng 5m defectve crcular solder paste block usng algorthm parameters n Table 6.3 The estmated shape has an average absolute heght dfference of 33.83m hen t s compared th the actual shape. Therefore, the error of the estmaton s 33.83m/5m=22.55%. Then the parameter optmzaton process s further tested th ths sample. The optmzed parameters and the estmated shape usng optmzed parameters are shon as follos: 2 (m/pxel 3 (m/pxel 4 (m/pxel 5 (m/pxel Algorthm Parameter 6 (pxel 7 (pxel 8 (pxel 9 (pxel (m/pxel (m/pxel Table 6.4 Optmzed algorthm parameters for the defectve 5m crcular solder paste block 43

163 (m (m (mm (mm Fg. 6.4 The estmated shape of the testng 5m defectve crcular solder paste block usng optmzed algorthm parameters Comparng th the actual shape, the estmated shape usng optmzed parameters gves an average absolute heght dfference of 2.2m. So, the error of the estmaton s reduced too 2.2m/5m=4.3%. Ths shos that the optmzaton process can help to fnd a set of parameters hch gves hgher accuracy n shape estmaton. 44