A Comparison Between Square and Hexagonal Sampling Methods for Pipeline Image Processing.

Transcription

1 A Comparion Between Square and Hexagonal Sampling Method for Pipeline Image Proceing. Richard C. Staunton and Neil Storey The Univerity of Warwick, Coventry, UK, CV4 7AL. ABSTRACT The majority of machine viion ytem derive their input data by digitiing an image to produce a quare grid of ampled point. However, other ampling technique can repreent equal picture information in a maller number of ample, with a conequent reduction in data rate. Several worker have looked at regular hexagonal ampling of image which produce optimum data rate for a given information content. Previou work on hexagonal ampling by the author and other, ha hown that image proceing operator are computationally more efficient, and a accurate, a their quare counterpart. Hitorically, one factor which ha lead to the predominance of quare ampling in viion ytem, i that thi produce image which are more viually pleaing to human oberver. Thi paper decribe an invetigation of machine viion ytem performing indutrial inpection tak, which ugget that in uch application, hexagonal ytem outperform quare ytem. In particular hexagonal operator can follow tight curve more accurately, allowing better urface defect detection. A urpriing obervation of thi work wa that with uch image, hexagonal ampling alo gave image which were more viually pleaing to human operator. The paper preent a tudy of ampling point geometry and operator deign. Detail are given of an implementation of a et of hexagonal, grey-cale operator for ue in pipeline or other image proceing ytem, and a comparion of quare and hexagonal technique ha been made. Reult of operation on real and imulated urface defect image are given for both ampling ytem and the requirement for a defect detection figure of merit identified.. INTRODUCTION Conventionally, image are digitied and tored a a quare or rectangular array of value. The image i ampled at each point on a two dimenional grid of ampling point and a quare or rectangular tiling of pixel applied to the grid uch that a ampling point i located at the centre of each pixel. Before ampling, the image data i band limited in two dimenion in accordance with the ampling theorem. Thi i uually achieved through the averaging caued by the finite ize of the ening element. The area repreented by the pixel i conidered to have a uniform intenity of the value a meaured at the ampling point at it centre. Such a grid of ampling point and pixel i hown in figure. Many other ampling cheme are poible, ome are lited by Whitehoue, but Merereau 2 ha hown that a regular hexagonal cheme i optimum for ampling circularly band limited two dimenional ignal, in that the leat number of ampling point are required to maintain equal high frequency information. 3.4% fewer point are required than for quare ampling. On a regular hexagonal grid, ampling point are equiditant from each of their ix nearet neighbour. The grid geometry i hown in figure 2. Variou pixel hape can be conidered to tile the hexagonal grid, hexagonal and rectangular being the mot widely ued. Rectangular pixel have the advantage for image diplayed on rater canned output device, uch a TV, monitor a the pixel height can be an integral number of can line. A rectangular pixel hape i hown in figure 2. If each pixel ha unit height, then it length will be 2/(3 /2 ).

2 L L L L =.0, L2 = 2.0 / (3.0 /2 ). L2 Figure. Square Pixel on a Square Grid. Figure 2. Rectangular Pixel on a Hexagonal Grid. Local operator are eaier to deign for the hexagonal grid a the geometry contrain ampling point in the neighbourhood of a central point to be in fewer equiditant group from the central point than in the quare cae. Ditance from the centre i an important factor in local operator deign. Figure 3 how a local operator approximation to a Gauian moothing filter for ue in the two ytem. The area covered by the two operator i nearly the ame, but the four corner point in the quare operator are 2 /2 further from the centre than the other four neighbouring point. Thi ha reulted in three weighting factor needing to be calculated, wherea for the hexagonal cae, the ix neighbour are equiditant from the centre and only two weighting factor needed to be calculated. Connectivity within binary image ha been hown by Golay 3 to be more eaily defined for hexagonally tiled image. The central pixel connect to each of it ix neighbour along an equal length edge, wherea in the quare ytem, connectivity can be edge to edge with the nearet four neighbour or additionally include corner to corner connectivity with the next nearet four. With a rectangular tiling on a hexagonal grid, connectivity can be viualied a end to end or half edge to half edge, but the computer connectivity model can revert to Golay original definition (a) (b) Figure 3. Gauian Approximation Filter. (a) Square Grid. (b) Hexagonal Grid. The cla of image operator conidered in thi paper are all local operator, a thee are eaily realiable with a pipeline proceor. Thee operator take a input the pixel value urrounding a particular pixel and output a new value for

3 that pixel. For completene, we note that global operation are alo poible on hexagonally ampled image. Merereau 2 ha developed a hexagonal f.f.t. and f.i.r. filter and hown thee to be more computationally efficient than their quare counterpart. We have deigned and imulated a pipeline architecture that will operate in real-time on a tream of video data input directly from a rater-can device. All procee performed are local and o operator can be performed with only a few can line delay. The ytem and architecture are decribed in detail elewhere 4,5, but eentially the pipeline proceor ytem appear a in figure 4. The input i digitied and fed into the firt proceor element (PE) in a tring of hardware identical PE. Each PE i programmed to perform an operation from a et including edge detection, filtering, line thinning, inverion and caling. Special feature of the ytem include adaptive control and VLSI implementation of the PE. We enviage it ue in high peed low cot application. Output i either to a high level proceor or a TV monitor. The baic PE architecture i hown in figure 5. Digitied video i input into a line length delay which feed a econd line delay. The leftmot three pixel of each line together with the firt three pixel of the third can line are acceible by the operator proceor. Thi i a ytolic proceor which modifie the value of the central pixel of the nine pixel input to it. Thu the overall PE delay i one can line plu a few extra pixel. Thi paper dicue how the architecture can accommodate hexagonally ampled image. ync in video in video ource ync video a.d.c p.e p.e p.e d.a.c monitor line delay line delay pipeline control high level image proceor control operator proceor ync out video out Figure 4. The Image Proceor Pipeline. Figure 5. The Proceor Element Architecture. Thi paper conider the advantage and diadvantage of hexagonal ampling with repect to indutrial inpection tak and how uch ampled image can be proceed by a pipeline proceor. Generally, the advantage include :- (a) That 3.4% fewer ampling point are required for equal high frequency information 2. (b) More efficient global 2 and local 6 computation i poible. (c) The eae of defining connectivity 3. (d) The eae of deigning operator 7. (a) (b) The diadvantage include :- The large number of publihed technique pecific to quare ampled ytem, although it i eay to deign operator for the hexagonal ytem, time may be needed to convert publihed quare ytem operator. Merereau 2 note that quare two dimenional operator can eaily be developed from one dimenional operator and that hitorically thi led to the adoption of the quare ytem. Preton 8 indicate that the main reaon for the unpopularity of hexagonal ytem would appear to be that it i perceptually more atifactory for human to oberve traight vertical feature on a quare grid ytem, and that in

4 the real world, uch feature predominate. With a machine viion ytem, thi will not be a concern a human involvement i limited to problem evaluation and ytem etup checking. We alo note in thi paper that under ome circumtance hexagonal pixel image can be more pleaing to human. 2. Sytem Modification 2. MODIFICATIONS TO THE PIPELINE FOR HEXAGONAL SAMPLING Conidering the non-interlaced, analogue ignal, TV camera a being the mot likely input device to the pipeline, generally few ytem modification are required to enable hexagonal digitiation of the ignal. The ignal mut be circularly band limited at the ame frequency a for an equal reolution quare ytem, and a with quare ampling, the vertical ampling point pacing i et to an integral number of can line. The horizontal ampling pacing i increaed by a factor of 2/(3 /2 ), and the firt ampling point on alternate line delayed by half a point pacing. Thi reduce the number of point per can line by the 2/(3 /2 ) factor. It i noted that the type of CCD camera that output a digital byte for each cell of it CCD array would require a modification to it array layout to enable hexagonal ampling. The pipeline architecture i a ytolic ytem and data input mut be ynchronied with the clock that pule the line of proceor that operate on the tream of data a it pae through. Fewer modification to the PE reult if the input data i arranged to ynchronie with the PE a deigned for quare ampling. Thi i achieved if an A to D converter that convert in half the hexagonal ampling point pacing time i ued. The converter ample at time o a to effect hexagonal ampling, but data i read at time to effect a rectangular array. Thi i illutrated in figure 6. Simple circuitry i ued to re-ynchronie the pipeline output data for diplay on a TV monitor. read read read line line 2 read read read - ample Figure 6. A to D Converter Sample and Data Ready Timing. 2.2 PE Modification Change to the PE architecture are minimal, extra tap are added to the line delay to reflect the reduced number of pixel per line. The operator proceor mut alo be modified. For quare ampled data, ome operation performed by the proceor require the convolution of the nine image pixel input to the proceor with one or more 3x3 array of contant tored within the proceor. The equivalent for hexagonally ampled data require convolution with a even element array, ie. a central element together with ix urrounding element. With the above ytem modification, for hexagonal data, the poition of the central pixel with repect to the ix neighbour change within the grid of nine input pixel on alternate can

5 line. Thi i illutrated in figure Z U Y A V X W (a) (b) (c) (d) Figure 7. The Relationhip Between the 9 Input Pixel and the 6 Hexagonal Nearet Neighbour on Alternate Scan Line. (a) The 9 Input Pixel, (b) Logical Central Pixel A and 6 Nearet Neighbour, (c) Phyical Set of 7 Pixel, Scan Line, (d) Phyical Set of 7 Pixel, Scan Line 2. It i neceary to tore an extra et of convolution contant array within the PE operator proceor and to toggle between et on alternate line. The two unwanted element in each array are et to zero. The extra area on a VLSI chip for thee modification i mall in comparion to the area required by, for example, the line delay. 2.3 Comment With a pipeline proceor, the proceing time for an image will be unaltered for a particular operation whether quare or hexagonal digitiation i employed. One image i proceed in one frame time, although a latency delay period i introduced by the tring of PE in the pipeline. The only advantage of hexagonal digitiation in pipelined ytem i that the line delay can be reduced in length by 3.4% and that the PE mater clock can be reduced by the ame factor. Thee advantage will not greatly affect the final cot of the VLSI device, the argument ha to be that a quare only or a dual tandard device look likely for fabrication. It will only be worth building an optional hexagonal proceing capability into the PE if certain procee can be demontrated to give better reult with hexagonal data. The detection of mall urface defect in indutrial component i poibly uch a proce. It i noted that with a conventional ingle CPU computer, computation time aving of over 40% have been achieved with hexagonally ampled data INSPECTION OF SURFACES FOR SMALL DEFECTS To be ued for indutrial inpection tak, machine viion ytem mut be cot effective, fat enough to proce one object before the next one i preented and reliable. Pipeline ytem employing VLSI - PE are likely to be both cot effective and fat enough. Reliability of defect detection will now be conidered. For a typical object accept-reject inpection application, it will be required to know the ize of defect at which rejection hould occur, then knowing the ize of the complete object and the reolution of the ytem, the number of image required to cover the object can be calculated. If the number of image i minimied, the cot per object i minimied. The quetion of reolution mut therefore be analyed. In a noie free environment, even a ub-pixel ize defect cating a hadow may have a detectable effect on the value of a pixel, but in a noiy environment fale accept-reject deciion will occur. Slightly larger defect may produce effect in a group of neighbouring pixel, thi group may till be unconnected, but their proximity can be ued to decreae the number

6 of incorrect accept-reject deciion. Still larger defect may be detected a bounded region and meaurement of area and hape made to aid the accept-reject deciion and provide diagnotic information for the indutrial proce. Theoretically we expect the hexagonal ytem to how advantage with mall defect detection. Firtly, a the local operator ued are more compact, they hould follow tightly curved edge more eaily, and econdly, the connectivity definition hould enable eaier outlining. Figure 8. Square Sampled Image Showing Defect. Figure 9. Hexagonal Sampled Image Showing Defect. Figure 0. Edge Detected and Thinned Square Image. Figure. Edge Detected and Thinned Hexagonal Image. To tet defect detectability we began by proceing ome image of defect in a and core ued for automobile engine cating. The defect conit of long cratche and roughly circular indentation of variou ize, the mallet being about mm. diameter. The entire object wa 400mm. long. Figure 8 how a zoom in on a part of the object reulting in a 64x64

7 quare ampled image, the entire object wa digitied a a 52x52 image. Figure 9 how the equivalent hexagonally ampled ection. Thee image were formed by averaging from an image of twice the reolution, thi image conited of rectangular pixel with a 4:3 apect ratio, and it hould be noted figure 8 i not a quare ampled image, and that figure 9 i not a true regular hexagonal ampled image. They can only give pointer to the poible advantage and diadvantage of the two ytem. Figure 0 and how the defect image after after edge detection and thinning. Three defect are viible in the grey cale image of the and core, two circular defect are viible in the left finger of the core and a long cratch in the right finger. A trong light ource wa placed on the left ide of the object, and a "fill in" ource on the right ide during imaging. Surface indentation are revealed by hadow on their left ide and bright area on their right ide. Thi i not too eaily een in the PMT image publihed here, but i more eaily dicernible in the hexagonally ampled image. In the hexagonal image, the circular hape of the left finger defect are eaier to dicern a i the continuou cratch in the right finger, which i about three centimetre long. We alo oberve that, at thi reolution, the near vertical object edge are more eaily viewed in the hexagonal image. When the image are edge detected and thinned, the reulting hexagonal image how a well connected and parallel ided repreentation of the long cratch, wherea connectivity i incomplete and the hape not o eay to determined in the quare image. For the large circular defect, connectivity i equivalent in the two image, but the indication of hape appear better in the hexagonal image. In both cae the third defect i indicated only by a collection of point. The image were proceed by a computer imulation of the pipeline 4 which wa modified to enable comparable quare and hexagonal proceing. An equivalent et of quare and hexagonal local operator wa aembled. Operator included:- (a) (b) (c) (d) Edge detector Smoothing filter Line thinner Image inverter The Sobel edge detector wa extenively ued a it ha been demontrated by Davie 8 deign principle to be nearly optimum for the quare grid ytem, and an equivalent hexagonal operator ha been deigned by one of the author 6,7 to the ame principle. A quare ytem Gauian approximation filter 0 and it hexagonal equivalent 7 were ued for image moothing and equivalent line thinning operator were developed. The reult from the and core image appear to how ome advantage for hexagonal proceing, but the advantage mut be hown on true quare and regular hexagonally ampled image a the edge detector and other operator have been optimally deigned for thee. We have continued to imulate uch defect from mathematical model. 4. DEFECT MODELING AND DETECTION 4. The Model The model i implemented on a computer. It aim i to produce two output image, one on a quare 6x6 grid, and the other on a 3x6 hexagonal grid that are repreentation of a preciely poitioned and ized hape that ha been input at the tart of the computer programme. At preent, two defect hape are implemented, a circle and a rectangle. Thee can be of arbitrary ize and poition in the output image to within an accuracy of 0.03 of the length of a quare pixel ide. Thee hape were choen becaue they cloely repreent the defect found in the and core image, a cratch being a long thin rectangle. Other hape will be implemented a required. Within the computer model the outline of the requeted hape i plotted on a 52x52 quare pixel grid and then each pixel i modified by other parameter uch a background and defect brightne. When complete, thi defect image i

8 averaged to produce the output image. A local area of 32x32 pixel i averaged to produce one quare output pixel and an area of 37x32 pixel to produce one hexagonal output pixel. With image of thi reolution, averaging of a integral number of pixel produce no ditortion to the quare and only very little to the hexagonal grid. Thi method produce a pair of, almot exactly, comparable image with which to tet the quare and hexagonal ytem, although the total area covered by the hexagonal image i lightly le than that covered by the quare image. A imple model will contain the pecified hape with the background brightne et at one value and the defect brightne et to another. Where a pixel traddle the background-defect border it will take an intermediate brightne value. A 3D plot of uch a defect i hown in figure 2. A cloe examination of actual defect revealed the defect edge to be pread over everal pixel in the tranition from defect to background brightne. Thi effect i recreated in the model by moothing the defect data before the actual output image are produced. Real image are noiy, and o noie of a predetermined variance can be added to the model data. A 3D plot of a defect, a in figure 2, but which ha had it edge moothed and noie added i hown in figure 3. A further ophitication can be added by dividing the defect into two egment, one which i brighter than the background and one darker. Thi model oblique illumination from one ide of the object. Figure 2. 3D Plot of Circular Defect. Figure 3. 3D Plot of Circular Defect with Smoothed Edge and Added Noie. 4.2 Detection Applying edge detector to variou modelled defect reveal reult in cloe agreement with thoe obtained from the and core image in ection 3. However, many variable in the detection proce have been identified and the overall quetion remain: Will a hexagonally ampled ytem enable quicker and more exact detection for a large cla of defect? Defect detection i dicued in ection 3, where three of many poible definition are given. The choice of definition i tak dependent and additionally many other variable are introduced by the procee that are ued to achieve them. Thee variable have to be conidered before a comparion can be made between defect detection on quare and hexagonal grid image. For a detection cheme that involve defect outlining, variable include :- (a) (b) (c) The edge detector. The quare and hexagonal detector ued in ection 3 are equivalent for ue in both ytem, but they were deigned to be optimal for traight tep edge. With defect, tightly curved non-tep edge, that may be conidered planar only over mall local area, are likely to predominate, and thi may lead to different detector being deigned. Step and planar edge are defined by Haralick. The defect poition and ize. The poition and ize of mall defect with repect to the digitiation grid make a large difference to the edge detector output. A defect at a particular poition may line up more optimally with, ay, the quare grid than the hexagonal grid of ampling point. The defect hape. Some hape will line up more optimally with a particular grid.

9 (d) (e) (f) (g) The connectivity definition. Two definition are available for the quare ytem. The environmental and ytem noie. Noie will add uncertainty to the detection procee. In ome cae, image filtering tage will need to be employed. The percentage of complete outline neceary for detection. It may be poible to detect a defect and etimate it area and poition without complete connectivity having been achieved. The threhold method and value. The proce may not be able to ue optimum value. To conider the effect of thee variable, a Defect Detection Figure of Merit algorithm (DDFOM) i currently being deigned. A figure of merit method ha been developed by van der Heyden 2 for edge detector, which involve applying detector to randomly generated image of traight edge and curve. We intend to derive our DDFOM from the detection of a erie of randomly generated defect image and to link thi with computation cot figure. We expect the information gained will enable u to identify defect detection a an application where hexagonal ampling can be advantageouly employed and that a pipelined proceing ytem i uitable. More widely, the DDFOM will provide general information for inpection ytem deigner uch a the detection probability of a certain ize of defect. 5. CONCLUSIONS For the indutrial inpection application tudied, the hexagonal grid proceing ytem wa able to outline defect at leat a reliably, and for all the cae tried, more accurately, than the equivalently engineered quare proceing ytem. However, the more indutrially ueful tak of defect typing and meaurement will require additional, higher level procee, that, again, will have to be equivalently deigned in order to compare the two ytem. Many variable mut be conidered in order to make a ueful comparion, and many proceing operator, defect type, and computer architecture are available. Work ha tarted to identify a defect figure of merit (DDFOM) algorithm and to link thi to a computation cot figure. The DDFOM will not only indicate if the hexagonal ytem ha an advantage for a wide cla of defect, but alo indicate to the deigner which operator perform optimally for individual defect detection application. Regarding the human perception of hexagonally ampled image, we have found that defect viewed on a hexagonal grid of pixel are repreented more exactly than on the quare. The eye tend to follow the vertical line formed by the pixel boundarie in the quare image and to divide the defect up into vertical trip egment. In the hexagonal image, the eye eem to integrate information from all the defect pixel and convey a better repreentation of ize and hape. Thee are only peronal obervation, and are not meant a a definitive comment on human perception. None of the image in thi tudy contained perfectly vertical edge, but ome long, near vertical edge were preent (figure 8 and 9). At the reolution of thee zoomed in image (64x64) thee edge become meared by the quare pixel ytem, wherea in the hexagonal pixel ytem, it i eaier for the eye to follow the edge. We conclude that hexagonally ampled indutrial defect image are more pleaing to human oberver. Preton 8 noted that for general image viewing, the revere i conidered to be true, a many actual world image contain a preponderance of vertical edge. The pipeline ytem can be eaily modified to accommodate hexagonally ampled data. Only minor modification, requiring little extra chip area, would be required to the VLSI proceor element (PE) to enable a dual hexagonal-quare image proceing capability. A hexagonal only ytem would require approximately 3% le chip area and a lower clock rate than an equivalent quare only ytem, but development cot will be minimied if only the one, dual purpoe, ytem i purued. The pipeline ytem imulation and all the hexagonal local operator deigned worked well, and thi together with the encouraging reult from the urface defect application have convinced u to continue with the hexagonal proceing option for our pipeline proceor deign. 6. REFERENCES. D.J.Whitehoue and M.J.Phillip, "Sampling in a two-dimenional plane," J. Phy. A. Math. Gen. vol. 8 pp , 985.

10 2. R.M.Merereau, "The proceing of hexagonally ampled two dimenional ignal," Proc. IEEE vol. 67 no. 6 pp , June M.J.E.Golay, "Hexagonal parallel pattern tranformation," IEEE Tran. Computer vol. C-8 pp , Aug N.Storey and R.C.Staunton, "A pipeline proceor employing hexagonal ampling for urface inpection," 3rd. Int. Conf. on Image Proceing and it Application, IEE Conference Publication 307, pp.56-60, July N.Storey and R.C.Staunton, "An adaptive pipeline proceor for real-time image proceing," SPIE Conference: Automated inpection and high peed viion architecture 3, Philadelphia, USA. Proc. SPIE, vol.97, Nov R.C.Staunton, "The deign of hexagonal ampling tructure for image digitiation and their ue with local operator," Image and viion computing, vol. 7, no. 3, pp , Aug R.C.Staunton, "Hexagonal image ampling, a practical propoition," Proc. SPIE vol. 008 pp , K.Preton et al. "Baic of cellular logic with ome application in medical image proceing," Proc. IEEE vol.65, no.5, pp , E.R.Davie, "Circularity a new principle underlying the deign of accurate edge orientation operator," Image and Viion Computing, vol.2, no.3, pp , Aug E.R.Davie, "Deign of optimal Gauian operator in mall neighbourhood," Image and Viion Computing,vol.5, no.3, pp , Aug R.M.Haralick, "Edge and region analyi for digital image data," Computer Graphic and Image Proceing vol.2, no., pp.60-73, F.van der Heyden, "Evaluation of Edge Detector Algorithm," 3rd. Int. Conf. on Image Proceing and it Application, IEE Conference Publication 307, pp , July 989.

11 Figure : Fig 8. Square ampled image howing defect Figure 2: Fig 9. Hexagonal ampled image howing defect.

12 Figure 3: Fig 0. Edge detected and thinned quare image Figure 4: Fig. Edge detected and thinned hexagonal image 2

13 Figure 5: Fig 2. 3D plot of circular defect. Figure 6: 3D plot of circular defect with moothed edge and added noie 3