Precision edge detection with bayer pattern sensors Prof.Dr.-Ing.habil. Gerhard Linß Dr.-Ing. Peter Brückner Dr.-Ing. Martin Correns Folie 1
Agenda 1. Introduction 2. State of the art 3. Key aspects 1. Circular Homogenity approximation demosaicing 2. Difference vector edge filter 3. Subpixel precision edge probing 4. Experimental results 5. Conclusion 29.08.2013 Folie 2
Introduction Image processing based edge probing : Subpixel precision for single points (up to σ = 1/40 pixel) Search line based approach (analog to touch probing methods) Ø 12.43 Interpretation of point cloud is dependent on the measurement task Folie 3
Introduction Why color image processing for CMM? 1. Unknown object colors and edge contrast make monochrome optimization impossible Limited spectral resolution (3 channels) improve the odds for sufficient edge contrast for measurement. Color image Intensity Source: www.asrock.com 29.08.2013 Folie 4
Introduction Why color image processing for CMM? 2. Utilization of all the provided data in the digital image if color cameras are applied for other reasons, i.e. Colored live image for the operator Other image processing tasks need color information Split RGB-Image Object Red channel Green channel Blue channel 29.08.2013 Folie 5
Sensitivity Introduction Cameras with Color Filter Array (Bayer-Pattern-CFA) Patent US3,971,065: Bryce E. Bayer, 1976, Color imaging array Advantages: Monolithic geometric standard for all channels Mass production, low cost Disadvantages: holes when sampling single channels ( Demosaicing necessary) Wavelength [nm] Quelle: Datenblatt; Sony ICX424AQ Folie 6
State of the art Image processing used in CMM: Monochromatic camera sensor Edge probing in single channel or: CFA camera sensor Demosaicing Color spacetransformation i.e. HSI Edge probing in one of the channels (most often intensity) Folie 7
State of the art Image processing used in CMM: Monochromatic camera sensor Edge probing in single channel or: CFA camera sensor Demosaicing Color spacetransformation i.e. HSI Edge probing in one of channels (most often intensity) New Concept: CFA camera sensor Demosaicing Multi channel edge filter Edge probing in filtered image Folie 8
State of the art CFA camera sensor Demosaicing Multi channel edge filter Edge probing in filtered image Desired properties are specific to the application and hence very different Optimized for: a. Fast calculation (Live stream, preview, slow processing hardware) b. Pleasant appearance for human observer (Photography, Video) Disagreement on responsibility in industrial image processing: manufacturer of cameras user of cameras No CMM specific demosaicing algorithms Folie 9
Demosaicing Necessary step to fill the gaps Folie 10
Demosaicing Requirements: No edge position shift when approximating values Non adaptive behavior no image content dependent algorithm Identical affect to all pixel values not just filling the gaps Rotation symmetry analog to conventional optical systems influence on the image Name: Circular Homogeneity Approximation Demosaicing Circular shape of mask 5 distances to central pixel 5 coefficients needed Three equations based on homogeneity within the mask. i.e.: Folie 11
CHA-Demosaicing Missing two equations: Definition by rotation symmetric spread function Ratio of the volumes above the pixels leads to two equations: 2D-Gaussian-Function (with σ = 5/6) Folie 12
CHA-Demosaicing Four possible locations for interpolation: Central pixel blue Central pixel green in blue-line Central pixel red Central pixel green in red-line Different coefficients-matrices, but always the same coefficients Experiments with artificial mosaicing demosaicing: Subjective assessment of demosaicing results Reproducibility of the edge position in single channels Folie 13
State of the art CFA camera sensor Demosaicing Multi channel edge filter Edge probing in filtered image Creation of a signal at an objects edge Usually there are differential approaches: o Maximum of first derivate of intensity along a coordinate o Zero point of second derivate of intensity Most filters are designed for a single channel only Edge filter multi channel images: 1. Channel wise application of traditional filters and combination of the results 2. Vectorial approach Folie 14
Intensity Edge Value Difference vector edge filter Extraction of edge information Usage of all channels Responsive to all types of color edge contrasts (single color, hue, saturation, intensity) Signal in Original Image Edge Signal x k Position x k Position Folie 15
Difference vector edge filter Differential & vectorial approach Length of difference vector as a measure for differences of pixel neighbors B Euclidian vector norm x i G G x-x Xi j j -X i D n X j X i D X j, k X i, k k 1 2 x j X i RGB vector Pixel i RR X j RGB vector Pixel j X j X i Difference vector Folie 16
Difference vector edge filter Examples: Original image Edge image Standard-Testimage Lena Source: Playboy 11/1972 Folie 17
Edge probing Determination of the position of the maximum edge value Rough estimation of local maxima by standard methods (dynamic threshold, etc.) Fitting of a Gaussian function at a local maximum (splines and polynomials where tested as well) Several experiments with: Synthetic images Color Filter Array Cameras Monochromatic Cameras Folie 18
Experiments system comparison Object Multi channel system with CFA color camera in comparison with a monochromatic camera system CFA camera sensor Demosaicing Difference vector Edge filter Edge probing on filtered signal Monochromatic camera sensor Edge probing with established single channel methods Comparison Folie 19
Experimental results Test structures approx. 400 µm approx. 12 cm Folie 20
Experimental results Test structures: Approx. 1,3 µm Folie 21
Experimental results Projected pixel size Folie 22
Experimental results Procedure: Distance of two edges Edge detection for placement of the coordinate system Measurement with orthogonal search lines Calculation of the width of the structure Folie 23
Width in pixel (mean value of 100 images) Experimental results Monochromatic system System with cfa sensor an new color edge probing Search line position in pixel Pixel size approximately 6.45 µm Given value (calibration certificate): Position [pixel] 0 19 37 56 74 Value [µm] 399,40 399,33 399,25 398,12 398,21 step every 120 µm; U = 0,05 µm. Folie 24
Standard deviation in pixel Experimental results 1 40 Pixel Mean Standard deviation for i = 100 images u = 100 search line positions 1 70 Pixel Approx. 40% better X 1 n u X i, u n i 100 S u 1 n 1 n X i, u X u i 100 2 Monochromatic system System with cfa sensor S 1 n n S u u 100 Folie 25
Conclusion Reliable probing of edges that could have been problematic with monochromatic systems Equal or better measurement deviations compared to established methods Operator gains colored live image without compromising the performance of the coordinate measurement machine Application of cfa sensors in image processing systems for measurement of geometric features is of advantage, even at highest precision requirements. Thank you for your attention! Folie 26