MACHINE VISION SYSTEM FOR SPECULAR SURFACE INSPECTION: USE OF SIMULATION PROCESS AS A TOOL FOR DESIGN AND OPTIMIZATION

MACHINE VISION SYSTEM FOR SPECULAR SURFACE INSPECTION: USE OF SIMULATION PROCESS AS A TOOL FOR DESIGN AND OPTIMIZATION R. SEULIN, F. MERIENNE and P. GORRIA Laboratore Le2, CNRS FRE2309, EA 242, Unversté de Bourgogne, Le Creusot, FRANCE, 7200 Phone: +33 (0)3 85 73 0 90, Fax: +33 (0)3 85 73 0 97, e-mal : r.seuln@utlecreusot.u-bourgogne.fr Abstract - Ths work ams at detectng surface defects on reflectng ndustral parts. The lghtng system s the crtcal pont of the machne vson system. It has been carefully desgned to ensure the magng of defects. In machne vson desgn, the magng condtons are often the fact of experments. To brng help n the concepton of these condtons, a complete smulaton s proposed. The magng and lghtng system has been completely modeled. Ths modelng s based on ray tracng. The smulaton s used to optmze the lghtng and optcal features of our machne vson prototype.. INTRODUCTION Hghly reflectve surfaces nspecton s a problem met frequently wthn the automatc control of ndustral parts [-3]. Ths nspecton s generally done manually. It mples subjectvty and tredness nfluence on classfcaton results. A machne vson system offers objectvty, better relablty and repeatablty and s able to carry out defects measurement to classfy the ndustral parts qualty. Ths work ams at detectng surface defects on reflectng ndustral parts. The objects to be controlled are hghly reflectve and so, act as perfect mrrors. Surface defects are dents, bumps and scratches. The defects areas have the same reflectve propertes as the flawless area of the surface: they reflect ncdent lght only n the specular drecton. Industral parts dmensons are 200 x 50 mm and defects surface s less than mm². 2. LIGHTING PRINCIPLE The lghtng prncple used n our system enables to ensure to separate the defects from the flawless area. A tred technque to reveal the aspect defects s the magng of the reflecton of a structured lghtng through the surface [3][4]. The surface mperfectons provoke mportant lght rays' devatons. Ths property s used to detect defects wth a partcular lghtng system. Ths lghtng s bnary type. It s composed of a successon of zones of null lumnous ntensty and zones of maxmal lumnous ntensty. In these condtons, a defect appears n the captured mage as a set of lumnous pxels among a dark zone or a set of dark pxels among a lumnous zone. Fgure llustrates the lghtng prncple and shows a typcal mage acqured wth the lghtng devce. In the frst case, wthout defect, the surface reflects a dark zone of the lghtng. In the second case, the defect deflects lumnous rays comng from the lumnous zone and so, the defect appears as a clear spot n a dark zone. Imagng of reflectve surfaces s not easy. We observe the entre object envronment through ts surface. In order to capture mages wthout unwanted nformaton, we need to completely master the envronment of the surface. By choosng an adapted lghtng system, the magng of defects s possble. bnary lghtng lumnous zone dark zone camera We choose to saturate the camera n order to obtan mages where defects appear very contrasted on a dark background and so to enable a smple mage processng for detecton (see 3 - Defects magng and segmentaton). In these llumnaton and magng condtons, defects only appear as hgh gray level pxels n dark zones. defect 2 : wth defect : wthout defect Image Fgure - Lghtng prncple

3. DEFECTS IMAGING AND SEGMENTATION A. Defects magng In order to nspect the whole part surface, an element of the lghtng structure has to scan every part of the surface. The lghtng devces have to be dffuse and homogenous. So, the lghtng system s realzed by lumnous panels made of dffusers placed n front of fluorescent tubes. A hgh resoluton CCD camera, postoned vertcally to the object plan, nspects the objects (see Fgure 2). To carry out surface nspecton, one can magne that the object s movng n front of the camera and the lghtng system [6]. In the case of mportant surface curvature gradents, the projecton of the lumnous and dark strpes on the complex geometry surface vares a lot between two consecutve mages. So, entre scannng s not ensured f the object s movng n front of the statc lghtng. To overcome ths lmtaton, an nverse process s proposed: the lghtng structure s dynamc whle the object s statc. Lghtng tunnel Object Camera Havng statc object durng the nspecton presents numerous advantages: the strpes projectons and the poston of the strpes between two mages are completely mastered an a pror knowledge of the object to be controlled can enable defnton of regon of nterest n the surface nspecton shape defect detecton can be computed by nspectng the slhouette. In order to reduce the number of necessary mages to perform the scannng of large ndustral parts, the lghtng system s composed of juxtaposed lumnous and dark strpes. It enables a large number of lght transtons to scan the surface. The strpes are dsposed n a tunnel. It enables so a complete mastery of the envronment of ndustral parts to be controlled. Fgure 2 - Surface magng system The surface aspect magng s performed by dfferent lghtng system postons. The lghtng tunnel s translated along the man object axs. For each regular spatal poston, an mage s captured. We fnally obtan an mage sequence as seen n Fgure 3. B. Defects segmentaton In the sequence, defects always appear as hgh gray level pxels because of the saturaton of the CCD Matrx. By computng the mean mage of the sequence, we obtan a synthetc mage called the aspect mage. In ths mage defects appear as hgh gray level pxels and the entre flawless area of the mage appear wth medum gray level (see Fgure 4). Lghtng translaton Fgure 3 - Part of mage sequence (expermental results) Fgure 4 Aspect mage and segmented mage (expermental results)

The segmentaton of defects zones s then very easy to compute. A threshold enables to segment defects (see Fgure 4). A post processng s then appled on the aspect mage and the thresholded mage to dstngush defects types and to compute defects measurements. Defects measurements conssts frst n blob colorng to label the defects. The sze and the weght (the sum of gray level pxels representng the defect) are then computed. The defects are fnally classfed upon ther sze and weght. The revealng of defects s so effectve. The elementary processng for defect segmentaton, so very fast computng, s possble because of the well-desgned lghtng system. Ths lghtng system smplfes a lot the mage processng and so a real tme nspecton of reflectve products s thus possble. The machne vson prototype enables an effcent detecton at the ndustral producton rate. 3. SYSTEM MODELING Lghtng and magng condtons need to be modeled n order to analyze the mages features provded by a structured lghtng system. In our case, the geometrcal optcs s applcable because the wavelength of the ncdent lght s weak compared to the dmensons of the surface mperfectons [5]. So, ray tracng s used to analyze the reflecton of the lghtng through the surface. A pnhole model s used to descrbe the camera and a vector models each lght ray. The surface orentaton s descrbed by ts normalzed normal vectors N. The surface reflects ncdent lght only n the specular drecton. For each pxel C of the CCD matrx, the lghtng pont L reflected by the surface s computed from the reflected ray. The Fgure 5 descrbes ths modelng. H Lghtng L N R S I j x x Surface Sensor z C y O Os Fgure 5 - Imagng system modelng y f h Defects are zones where the surface s affected by geometrcal mperfectons. The surface s stll smooth n the defect zones, so the specular property s mantaned. For example, a dent s modeled by a gaussan curve. The gaussan functon characterzes the smoothness of the surface. Normal vectors are computed from the surface model. The nfluence of surface heght and surface orentaton on the lghtng pont reflected by the surface has been studed [7]. Ths study s based on physcal values taken from the machne vson prototype and s applcable n the case of a vewng pont stuated at a large dstance from the object (compared to the object dmensons). We demonstrate that the nfluence of the surface heght nduces neglgble lght rays devatons compared to the nfluence of the surface orentaton. The surface s so modeled as flat and s only descrbed by ts orentaton. The normal vectors completely descrbe the surface. Ths type of modelng can be compared to the bump mappng technque used n computer graphcs to artfcally represent the surface orentaton modfcaton. Durng experments, we notced that the sze of the defect sgnature on the mage depends on the dstance between the lght transton and the defect. It can be schematcally explaned as shown n Fgure 6. camera d w camera d 2 Fgure 6 - Defect sze varaton If the lght transton, projected on the surface, s close to the defect, the defect "sgnature" sze on the mage s close to ts real sze. But f the dstance between the defect and the lght transton ncreases, the defect "sgnature" sze decreases and can even be null for an mportant dstance. Ths partcular property can be analyzed by computng the defect "sgnature" sze from our lghtng and magng model. Fgure 7 represents the computaton of a defect sze (percentage of real sze) versus the dstance between the lght transton and the center of the defect (normalzed by the defect dmensons). w 2

A. Image renderng The magng and lghtng model enables to smulate the mage acquston process. Defects are modeled by representatve surfaces. The lghtng system was modeled n order to represent the developed prototype: lumnous panels are lambertan lghtng sources shaded by opaque strpes. Ray tracng s used to compute realstc mages. Ray tracng s well adapted for ths knd of renderng because the surfaces are completely specular. The phenomenon of sensor saturaton s extremely mportant wthn the framework of ths applcaton. It condtons the defect revealng, computed by the mean mage of the sequence. To represent ths saturaton effect, a flter s appled on each mage of the sequence. Fgure 7 Defect sze varaton versus dstance to the lght transton The defect s a dent modeled by a gaussan surface. The mage sgnature equals the physcal sze f the defect s close to the lght transton and t decreases f the dstance ncreases. In our ndustral applcaton, we have to obtan mage sgnatures proportonal to the defects physcal sze. So, the lght transton has to scan all over the surface to ensure each defect to be close to a lght transton n the mage sequence. 4. SIMULATION The magng condtons have been partcularly studed because they nfluence strongly the qualty of acqured mages and consequently the qualty of mage processng results. These magng condtons are often the fact of experments: numerous attempts on lghtng features and on the relatve postons between the camera, the lghtng and the object are stll necessary. To brng help n the choce of these magng features, a complete smulaton s proposed. Defect segmentaton processng s ntegrated n the renderng process n order to acheve a complete smulaton from the surface magng to the defects measurements. As n the real acquston process, an mage sequence s computed (see Fgure 8) and the aspect mage s computed from the sequence and segmented (see Fgure 9) B. Effcency evaluaton method A comparatve method s then used to quanttatvely evaluate the effcency of ntroduced parameters [8]. The segmented mage s compared to a reference mage defned by the a pror knowledge of the defects features ntroduced n the smulaton. The method s nspred by the Fram & Deutsch [9] crtera modfed by Cocquerez [0]. The defect segmentaton conssts n detectng a maxmum of pxels belongng to defects and a mnmum of pxels n the flawless area. We have so to make a compromse between overdetecton and msdetecton. The comparson s based on those two crtera called P and P 2. Lghtng translaton Fgure 8 Part of mage sequence (smulaton results) Fgure 9 Aspect mage and segmented mage (smulaton results)

We defne: Z the defect zone (all the pxels belongng to the defects) n d Z the number of pxels detected nto Z n d 0 the number of pxels detected outsde Z NZ the number of pxels that consttute Z P are computed as follows: n d Z d d Z 0 d = f n0 0 P n + n () d = f n = 0 0 d nz P2 = (2) N Z P take values from 0 (worst case) to (deal case of detecton). The features optmzaton s done by computng P from a range of ntroduced parameters. The best set of parameters s obtaned when P are both close to. Because we are workng n a qualty control context, we choose, n the segmentaton stage, to optmze the overdetecton than the msdetecton. The fnal decson about the part qualty s taken durng the classfcaton stage. C. Prototype optmzaton The proposed optmzaton method s appled to the machne vson prototype. The nput parameters to be optmzed concerns the lghtng devce, the sensor and the mage processng. Concernng the lghtng devce : the wdth of the lumnous strpes the wdth of the dark strpes Concernng the sensor : The saturaton features Concernng the sequence mage processng : The number of necessary mages for the sequence The threshold value appled on the "aspect" mage to segment the defects. values. For each parameter value, an mage sequence s computed and processed. P effcency parameters are then computed on the resultng mage. A dagram representng P 2 versus P enables an easy readable effcency measurements for each nput parameters range. In our applcaton, the smulaton s manly used to optmze the dmensons of the lghtng strpes. Ths smulaton tool brngs a frst help n the choce of the magng and lghtng parameters. The smulaton s used here as a tool for machne vson system optmzaton. Concepton rules for the lghtng structure are so provded by the smulaton process. 5. CONCLUSIONS AND FUTURE WORK A machne vson system for specular surface nspecton has been presented. Ths system enables the detecton of geometrc aspect surface defects. The revealng of defects s realzed by a partcular lghtng devce. It has been carefully desgned to ensure the magng of defects. Defects appear well contrasted n resultng mages. A smple, so fast processng, s then appled for defects segmentaton. Each system devce has been modeled. A pnhole model s used for the camera. The surface to be nspected s completely descrbed by ts normal vectors and defects are modeled by realstc surfaces. The lghtng sources are lambertan emttng surfaces shaded by dark strpes. A machne vson system desgn method, based on computer graphcs, s then proposed. The system modelng enables defects magng smulaton. Defect segmentaton processng s ntegrated n the renderng stage. We obtan a complete smulaton from the surface magng to defects measurements. An effcency estmaton s then computed on the synthetc resultng mages. The effcency s measured by two parameters whch quantfy overdetecton and msdetecton performed by the system. The developed prototype has been optmzed by ths method. Smulaton provdes here a very effcent way of desgn compared to the necessary numerous attempts from the manual experments. Future work concerns the renderng phase n the smulaton process. An enhanced renderng kernel wll be soon be mplemented n order to extend the smulaton for non-specular surfaces nspecton. 6. ACKNOWLEDGMENT These research works have been realzed wth the fnancal support of the Regonal Councl of Burgundy. The parameters value are varyng n a range of realstc

7. REFERENCES [] Kerrkegaard P. "Reflecton Propertes of Machned Metal Surfaces", Optcal Engneerng, Vol. 35, N 3, Mars 996. [2] Sanderson C., Wess L. E. and Nayar S. K. "Structured Hghlght Inspecton of Specular Surfaces", IEEE Trans. On Pattern Analyss and Machne Intellgence, Vol. 0, N, pp. 44-55, January 988. [3] Bakolas C. and Forrest A. K. "Dark Feld, Schempflug Imagng for Surface Inspecton", SPIE Conference on Machne Vson Applcatons n Industral Inspecton V, Vol. 3029, pp. 57-68, 997. [4] Batchelor B. G., Hll D. A. and Hodgson D. C. "Automated Vsual Inspecton - Chapter 7: Lghtng and vewng technques", IFS Publcatons Ltd. and North Holland, 985. [5] Nayar S. K., Ikeuch K. and Kanade T. Surface Reflecton: Physcal and Geometrcal Perspectves IEEE Trans. On Pattern Analyss and Machne Intellgence, Vol. 3, N 7, pp. 6-634, July 99. [6] Delcrox G., Seuln R., Lamalle B., Gorra P. and Merenne F. Study of the Imagng Condtons and Processng for the Aspect Control of Specular Surfaces, SPIE - Journal of Electronc Imagng, Vol. 0, N, January 200, pp. 96-202. [7] Seuln R., Merenne F. and Gorra P. Dynamc Lghtng System for Specular Surface Inspecton, SPIE Conference on Machne Vson Applcatons n Industral Inspecton VII, January 22-23 200, San José (USA), Vol. 430. [8] Seuln R., Delcrox G. and Merenne F. Comparatve Performance for Isolated Ponts Detecton Operators: Applcaton on Surface Defects Extracton, Vson Interface 2000, May 4-7 2000, Montreal (Canada), pp. 269-273. [9] Fram J.R. and Deutsch E.S. A Quanttatve Study of Orentaton Bas of some Edge Detecton Schemes, IEEE Trans. on Comp., vol. 27, pp. 205-23, 978. [0] Cocquerez J.P. and Devars J. Détecton de contours dans les mages aérennes : nouveaux opérateurs, Tratement du sgnal, vol. 2, n, 985.