A Smart Machine Vision System for PCB Inspection

Transcription

1 A Smart Mache Vso System for PCB Ispecto Te Q Che, JaX Zhag, YouNg Zhou ad Y Lu Murphey Please address all correspodece to Departmet of Electrcal ad Computer Egeerg Uversty of Mchga - Dearbor, Dearbor, MI 488 Phoe: , Emal: ylu@umch.edu Jabl Crcut, Ic. 7 Atlatc Boulevard Aubur Hlls, Mchga 4836 Y Lu Murphey Departmet of Electrcal ad Computer Egeerg Uversty of Mchga-Dearbor Dearbor, Mchga , U.S.A. Voce: , Fax: ylu@umch.edu Keywords: computer vso, automated problem solvg, tellget maufacturg Abstract -- I ths paper, we preset a smart mache vso (SMV system for prted crcut board (PCB specto. It has advatages over the tradtoal maual specto by ts hgher effcecy ad accuracy. Ths SMV system cossts of two modules, LIF (Learg Ispecto Features ad OLI (O-Le Ispecto. The LIF module automatcally lears specto features from the CAD fles of a PCB board. The OLI module rus o-le to spect PCB boards usg a hgh-resoluto -D sesor ad the kowledge provded by the LIF compoets. Key algorthms developed for SMV are preseted the paper. The SMV system ca be deployed o a maufacturg le wth a much more affordable prce comparg to other commercal specto systems. Keywords: mache vso, automatc specto, PCB specto. Itroducto The PCB(Prted Crcut Board dustry cotues to adopt creasgly hgher levels of tegrato ad achevg hgher ad hgher levels of compoet desty. As a cosequece, the toleraces o PCB assembly become tghter ad tghter. Ths causes a creased eed for relable ad accurate vsual specto of PCB boards[,,3]. The maufacturg of PCB crcuts uses the SMT (Surface Mout Techology. The SMT crcut assembly cossts of three major processes, scree prtg solder paste o the PCB, compoet placemet ad the solder re-flow a covecto ove. Correspodgly, there are three ma tasks of vso specto PCB assembly: Solder paste specto, Compoet placemet, ad Post-reflow specto. Placg a proper amout of solder paste o a pad s the key to prevet uwated opes or shorts. Sometmes, t s possble to catch these uwated opes or shorts usg a -crcut-test after all compoets are placed

2 o the board, but most solder paste defects are mpossble to catch after compoets are mouted. The focus of ths research s to develop a techology for specto of solder paste o PCB s. Due to the developmet of the semcoductor techology, the electroc compoets are gettg smaller ad smaller, ad more ad more compoets ca ft o oe PCB board. A pad o a PCB ca be as small as. ch (see Fgure. Mache vso specto of solder paste o PCB s s a o-trval task. I ths paper we descrbe a smart mache vso (SMV system for spectg defects of solder paste o PCB s. SMV was developed usg mache learg techology combed wth advaced mache vso techques. SMV has bee deployed o a maufacturg le ad bee tested o more tha PCB boards, the accuracy of detecto has exceeded 97%. pxels for a small pad ad the paste o a small pad ca be as small as oe or two pxels. The SMV system was developed ad tested usg mages of a varety of PCB boards acqured by a hgh resoluto KxK Kodak camera. Fgure 3 gves a overall vew of SMV system. pad Fgure a The pads wthout solder paste fducal paste fducal Fgure b The pads wth solder paste Pad CAD Fle Paste CAD Fle Learg Subsystem LIF Ispecto Template: q fducal locato, sze, shape q pad locato, sze, shape q paste locato, sze, shape q occupacy rato of paste o pad Fgure A PCB board array wth thousads of pads. Overvew of SMV system The objectve of the SMV system s to detecto whether there s a suffcet amout of solder paste o a pad, or f there s smear o a solder pasted pad. I theory, the bare pads ad the solder pastes o a PCB should have dfferet reflecto rate uder drect llumatg (see Fgure. The major challegg s the hgh desty of PCB boards ad low cotrast bare pads ad paste. Eve wth the hghest resoluto CCD cameras o the market, mages of hgh desty PCB boards typcally have gve less tha fve OLI PCB Hgh-resoluto Image Ispecto -D Sesor Pass/Fal? Subsystem Fgure 3 The SMV system The SMV s composed of two modules: LIF (Learg Ispecto Features ad OLI (O-Le Ispecto. The LIF module was developed to lear vsual specto features of a PCB from CAD desg data, ad the OLI module apples the kowledge leart by LIF to o-le PCB

3 specto. The LIF module automatcally lears the specto features ad outputs the kowledge a template fle to be used the o-le specto program, OLI, durg the maufacturg process. The learg program cossts of algorthms for extractg specto features from the put, mage processg, ad computg statstcs. The OLI program cossts of algorthms for mage regstrato, mage processg, comparg the features of the put mage wth the leart features from the LIF. The output from the OLI wll dcate whether the product has faled or pass the test. The exact format of the output from the OLI ca vary depedg o the task specfcato. 3 Learg specto feature (LIF module The prmary fucto of the LIF s to lear the solder features of a PCB from ts CAD desg fles. PCBs are desged usg CAD tools ad the desg formato s cotaed a CAD fle. A PCB CAD fle cotas a set of structos that, whe terpreted, eable a photoplotter or laser mager to produce a mage of the PCB o paper or other meda. Oe example of such CAD fle format s GERBER, a o-propretary superset of Electroc Idustres Assocato Stadard RS-74D. A CAD desg laguage typcally has more tha 4 dfferet operators (or commads. We mplemeted 5 operators that are ecessary for learg spectos features. The challege of mplemetg these operators les the dyamcs of these operators. For example, a umber of operators may dyamcally chage the appearace of a le, the shape of the jot of two les; ad the target coordates. I order to effectvely spectg PCB mages, the LIF module eeds to lear the followg features from the CAD fle: umber of pads o each array, locato of each pad, shape of each pad, fll or o-fll status of each pad, ad locato of every fducal pots o each PCB mage. A learg algorthm has bee developed for the LIF module ad t has three major steps, detectg compoets, fdg boudg boxes, ad computg occupacy rato. The occupacy rato s a measuremet to be used the o-le specto procedure ad s crtcal to the result of specto. A compoet o a PCB s a rego that ca be a pad, a paste, or a fducal. Fducals(see Fgure used to map CAD data to the mages captured o le ca be ay shape. A compoet a CAD fle typcally has oe or more closed paths, each of them cossts of a umber of strokes. A stroke ca be as smple as a straght le, or as complcated as a part of Bézer curve. The formato we are most terested s the boudg box, whch tells us the exact locato of a compoet, ad the occupacy rato, whch wll be used paste specto the OLI module. The algorthm repeatedly searches for the boudg boxes of the strokes that are coected. The stroke boudg boxes are the merged to form the boudg boxes of the closed paths. Fally the boudg boxes of overlapped closed paths are merged to form the boudg box of a compoet. The occupacy rato s defed as the rato of the solder paste area verses the boudg box of a pad. I order to spect the solder paste durg the maufacturg, accurate occupacy rato s the key. The computato of occupacy rato s a otrval task. I ths algorthm we represet complcated curves by straght-le segmets, whch the area of a stroke s qute close to the area of ts boudg box. The occupacy rato s calculated by computg the sum of the areas of the boudg boxes of the strokes wth the overlappg regos subtracted. The followg outles the computatoal steps used to compute occupacy rato. Let R be a array of N rectagles, A be the total area of R[], R[],, R[], ad B be the area of R[+] subtractg the part where t overlaps wth the frst rectagles. Thus we have, N N + BN =... = A + = A = A B ( N where A s the area of R[], whch s easy to calculate. I order to calculate B, we frst compare R[+] wth R[]. If they overlap, we splt R[+] to up to 4 rectagles deoted as r, r, r 3 ad r 4 (see Fgure 4. The we compare r, r, r 3 ad r 4 wth R[] oe by oe. They splt f ecessary. Ths procedure cotues utl R[] s splt ad ts four rectagles are compared. B equals to the sum of the areas of the rectagles from fal splttg. It s guarateed that the rectagles from splttg after comparso wth R[] do ot overlap wth ay rectagle of R[],, R[], ad they do ot overlap wth each other ether of course.

4 r r R[] r 4 Fgure 4 R[+] splts to up to 4 rectagles f R[+] ad R[] overlap I order to facltate egeers to vew the formato of the PCB ad edt the formato for robust specto, we developed a graphcs user terface accompaed by the followg fuctos: Brd s Eye Vew of the PCB. Zoom- fucto to allow user to vew dfferet part of the board detal. The Compoet-Clppg wdow. Some tme, CAD fles cota rrelevat formato, e.g. captos ad frames. We developed a Compoet-Clppg wdow fucto that allows user to defe a rego of terest to exclude rrelevat formato. Oly the compoet sde the rego of terest wll be leared by the LIF algorthm. The Compoet-Edtg wdow. The Compoet-Edtg wdow allows user to edt the formato such as the compoet ame, the occupacy rato, ad the threshold of test. User ca also specfy the fducals by checkg the rado butto. A dvdual ame wll help a egeer to fd whch compoet fals the test. As the result of learg, LIF module stores the followg attrbutes for every compoet o a PCB, compoet ame, locato, occupacy rato, ad test threshold. The test threshold s determed by the sze of the compoet. I addto, the locato ad sze of each fducal s detected for use the specto module. The fle that cotas the kowledge of specto features s referred to as a template fle. 4 O-le specto (OLI The OLI module spects PCBs o a maufacturg le uder the gudace of the specto kowledge provded by the LIF module. The major processes wth the OLI clude: Fdg the fducals the mage of a PCB barzg the mage; r 3 R[+] 3 Detectg the tlt agle ad the correct the mage; 4 Mappg the compoets based o the formato provded by the LIF to the mage; 5 Detectg the paste occupacy o each compoet the mage. The accuracy of fducal fdg algorthm s crtcal to the specto result, because the tlt agle ad the mappg scale ad the mappg offset all come from the fducals. The fducals are usually symmetrc shapes such as roud shape. Accordg to the formato from the template we ca get the approxmate locatos of the fducals o the mage. Aroud these approxmate locatos we set searchg areas for the fducals. I each searchg area, the least-square fttg method s appled to calculate the ceter of the roud-shape fducal. The objectve fucto of the least square fttg s defed as: E( x,, (, c yc r = ρ x y = m ( x ( xc + y r ( where the weght ρ ( x, y s the testy value of (x, y o the orgal mage. I order to avod usg o-lear optmzato method to solve Eq.(, aother objectve fucto s defed: E ( xc, yc, r = ρ( x, y = m Eq.(3 ca be rewrtte as: [( x xc + ( y r ] (3 E ( x,, ( ( c yc r = x xc + y + r ρ( x, ( ( y x xc + y r = m (4 From Eq.(4 we ca see, Eq.(3 equals to Eq.( multpled by a scale k. ( x x + ( y y + c c k = r (5

5 If k s a costat, the soluto of Eq.( ad that of Eq.(3 wll be same. If k s ot a costat, but we ca get better approxmate values for x c, y c ad r, the Eq.(3 ca be chaged to: [( x x + ( y y r ] ρ( x, y c c E ( xc, yc, r = ( ( x x + y y + r = m (6 A terato algorthm has bee developed for calculatg x c, y c ad r. Frst, we used Eq.(3 to obta the orgal values for x, y ad r. The we used Eq.(6 to calculate x c, y c ad r, ad replaced x, y ad r by these ew x c, y c ad r the ext calculato of Eq.(6 utl the dfferece betwee x, y, r ad x c, y c, r was very small. Ths fducal fdg algorthm s very robust. The fducals ca be foud eve f may pads are wth the searchg areas. After the fducals are accurately located o the mage, the global boudg box of the specto area o the mage s calculated, ad the followg barzato operato s appled o the mage cotet wth the global boudg box. Based o the aalyss of PCB mages, a effectve barzato algorthm s developed, whch cossts of the followg steps: [Step-] Geerate the hstogram V(X of the PCB mage, where V(X s the umber of pxels at pxel value X. [Step-] Smooth the hstogram usg a averagg flter. I most cases, the peaks aroud P L ad the peaks aroud P H wll ot be removed. [Step-3] Fd the hghest peak P L. Suppose P L locates at X L. [Step-4] Defe V MIN (X, for X (X L, 55, V MIN (X equals to the mmum V(X, X (X L, X]. [Step-5] Search from X L to 55, fd the locato correspodg to the maxmum of [V(X/V MIN (X]. We deote t as X H. We cosder X H the locato of P H because ths peak has the largest peak-to-valley rato tha ay other X (X L, 55. By fdg the peak accordg to [V(X/V MIN (X], we ca skp the peaks aroud P L whch are usually hgher tha P H. [Step-6] Fd the locato X where V(X = V MIN (X H. We deote t as X t. X t s the barzato threshold we wll use. The reaso s that t s the locato of the deepest valley betwee P L ad P H ad should be the most sutable border betwee {P L } ad {P H }. O a maufacturg le, a PCB s ot always mouted perfectly o the test table. It may tlt a small agle. But ths small tlt agle may cause bg errors whle the template mappg o the mage. So the PCB mage has to be utlted frst. The least mea square method s used estmatg the tlt agle. Suppose there are fducals, ther locatos the learg template are (X, Y, ad ther locatos the mage are (x, y, where,,.... Suppose the tlt agle of the mage relatve to the template s α, the scalg factor s k, ad (x, y are offset, the we should have, x k y k ( X cosα Y sα ( Y cosα + X sα + x + y (,,..., (7 Clearly, α, k ad (x, y ca be determed by mmzg the followg objectve fucto E(α, k, x, y : E( α, k, x, y = { [ k( X cosα Y sα + x x ] [ k( Y cosα + X sα + y y ] } + = m (8 By lettg = = = =, we wll get the α k x y followg soluto: ( xy yx ( xx + yy ( xx + yy cos xy yx taα = xx + yy [ xx + yy ] α [ xy yx ( xy yx ] sα k = X + Y ( X + Y x = x k( X cosα Y sα y = y k( Y cosα + X sα (9 where f = f s the mea. After the tlt agle s estmated, the mage s utlted the the ew locatos of the fducals the modfed mage are re-calculated. These ew locatos of the fducals wll be used mappg the template

6 oto the mage. Because utltg the mage s the most tme cosumg part the whole process, efforts have bee made to speed t up:. Oly the part of the mage sde the global boudg box s utlted;. Fuctos s ad cos are called oly oce; 3. Floatg pot multplcato/dvso has bee reduced to mmum; 4. The lower lmt of the tlt agle s estmated before utltg, the mage s utlted oly whe t s ecessary (.e. the tlt agle s above the lower lmt. The least mea square method s also used mappg the learg template oto the mage. Suppose we have fducals, ther locatos the learg template are (X, Y, ad ther locatos the mage are (x, y, where,,.... Suppose the scalg factor s a. The we should have, x ax + x (,,..., ( y ay + y where (x, y are traslato. Both a ad (x, y ca be determed by mmzg the followg objectve fucto E(a, x, y : E( a, x, y = [( ax + x x + ( ay + y y ] ( = m By lettg = = = =, we wll get the α k x y followg soluto: xx + yy ( xx + yy where a = X + Y x = x ax y f = = y ay f ( X + Y s the mea. ( Fally, the occupacy rato of paste o each pad s calculated. The occupacy ratos calculated at the LIF stage are the occupacy ratos of the pastes themselves. Ad t s the bare pads (ot the pastes that reflect the lght whle the CCD camera captures the mage. Therefore, f the occupacy rato of a compoet s above the test threshold, the t mples a mssg or damaged paste. For a K K mage wth more tha 4 compoets, the processg tme of automatc specto s about 6 secods o a Petum Pro computer. 5 Coclusos We have preseted a automatc solder paste specto system, SMV (Smart Mache Vso, whch releves huma test operators from a stressful ad urealstc specto task. The SMV system has two modules, amely the LIF (Learg Ispecto Features ad the OLI (O-Le Ispecto. Durg the off-le learg process, the LIF lears from the CAD fles of the PCB ad geerates a specto template for every ew type of PCB layout. The OLI module rus o the producto le accurately ad effcetly spects PCBs. The key algorthms for supportg the SMV system are troduced. Ths automatc solder paste specto system fds defects at the early stage o the producto les, whch ca sgfcatly reduce the maufacturg cost. The whole system has bee tested over 8 boards o a maufacturg le ad the detecto accuracy was above 97%. 6. Ackowledgemets Ths work s supported part by a cotract from Jabl Crcut, Ic. I partcular, we would lke to thak Mr. Athoy Tsler for troducg the problem ad hs cotrbuto the kowledge of prted crcut board maufacturg. 7. Refereces [] F. J. Lagley, Imagg systems for PCB specto, Crcut Mauf. 5(, pp5-54, 985. [] M. Beck ad D. Clark, SMT specto strateges: Maxmzg cost effectveess, Proceedgs of Techcal Program: NEPCON West'9, pp75-8, 99. [3] B. R. Taylor, Automatc specto electrocs maufacturg, SPIE Autom. Opt. Ispecto 654l, pp57-59, 986. [4] S. Muka, PCB cotuous le system proceeds from maufacturg to specto, J. Electro. Eg. 9(35, pp [5] J. W. Foster III, P. M. Grff, S. L. Messmer ad J. R. Vllalobos, Automated vsual specto: A tutoral, Comput. Id. Eg. 8(4, pp493-54, 99. [6] S. Yu, W. Cheg ad S. C. Chag, Prted crcut board specto system PI/, SPIE Autom. Isp. Hgh Speed Vso Archt. II 4, pp6-34, 988.

7 [7] E. B-Nu, Automatc optcal specto focuses o defects, Electro. Packag. Prod. pp8-87, 984. [8] J. Raglad, Automatg erlayer specto, Crcuts Mauf. Pp9-94, 995. [9] H. W. Markste, Automatc optcal specto mproves multlayer yelds, Electro. Packag. Prod. Pp6-64, 983. [] W. Wu, M. Wag ad C. Lu, Automated specto of prted crcut boards through mache vso, Computers Idustry 8, pp3-, 996. [] Y L et al "Mache Vso Algorthms Usg Iteractve Learg For VFD Ispecto," submtted to Joural of Appled Itellgece,