-Todd Wittman, IMA Industrial Problems Seminar, 10/29/04

Transcription

1 Decreasing Blur and Increasing Resolution in Barcode Scanning What is a Barcode? A barcode encodes information in the relative idths of the bars. Todd Wittman IMA Industrial Problems Seminar October 9, 4 Supervised by Fadil Santosa, University of Minnesota Miroslav Trajković, Symbol Technologies All bars are measured relative to the narroest bar, called the module. The module is the first thing to disappear! The UPC Symbology A symbology is a coding scheme. The UPC is the familiar barcode in the grocery store. All bars idths are integer multiples (1,,3,4) of the module in the middle. Delta code Self-clocking Backards reading Error-checking Digit Manufacturing Product Other Symbologies Code 39 PDF Code 18A Maxicode Datamatrix 1

2 Types of Barcode Scanners Laser Scanner Imaging Scanner Laser Scanner Barcode Optics Detector Amplifier Filter 13 PS3STD Industrial Phaser 1D laser scanline 48x64 image LS348FZ Fuzzy Logic Decoder Digitizer Decoder To Terminal Cheap, established technology. High resolution signal. Developing technology. Better for D barcodes. 13 Imaging Scanner Handheld Scanner Performance Barcode 13 Optics Detector Amplifier Filter 13 Digitizer Decoder To Terminal Barcode Size (mils) Field of Vie Blur Resolution (PPM) 13 I am trying to improve the filter, not the optics or decoder. Scanning Distance (inches)

3 Problems I orked on problems in improving the filter performance. Decreasing Blur in Laser Scanners Partially blind deconvolution by TV minimization Interesting mathematics, not very practical Summer 3, UMN Increasing Resolution in Imaging Scanners Geometric projection of D image to 1D signal Very promising, not very exciting mathematically Summer 4, Symbol The Deblurring Problem The signal obtained by the scanner is a blurred, noisy version of the -1 step signal. Barcode Ideal signal Blurred signal Standard Deblurring Methods Standard approaches are based on local edge detectors Otsu: Peak clustering Marr: Zero crossings of second derivative Symbol: Analysis of peak histogram These methods are fast, but they use only local information. Blur is generally applied to the hole signal. We ant a global approach. Warning: This ill make our computation much sloer! The Noise Model u G = Ga, * a, = ae u Ideal -1 barcode signal u Observed (noisy) signal n Additive noise G Gaussian blur kernel amplitude a, idth u + n ( x / ) a 3

4 Noise u = Ga, * u + n The blur factor depends on the laser spot size relative to the barcode size. As the scanner moves aay from the 8 inch scanning distance, ill increase (defocus). Also, roughness in the paper adds speckle noise. The amplitude a depends on the scanner intensity and ambient light. The Inverse Problem Forard Model: Noise model resembles actual data. G a, u u Inverse Problem: Reconstruct u from u.??? The additive noise n is electrical noise or source defects, such as shados or stray marks. u u The TV Norm We approach the inverse problem by minimizing the Total Variation (TV) norm, given by the folloing energy: E [ u u ] = u Ga u dx + β u, * dx Recover u from u by solving the minimization problem: min E u u, a, (Rudin, Osher, & Fatemi, 1) [ u ] Interpretation of TV Norm E [ u u ] = u Ga u dx + β u, * dx Fidelity (Matching) Term Measures size of noise n Total Variation Term Measures total jumps The parameter β balances the to terms. If β is too small, our signal ill match u exactly. If β is too big, u ill be a horizontal line. We need to set β experimentally. Fortunately, a value that orks ell for one barcode should ork ell for all barcodes. We found β=.1 orks ell. 4

5 Blind Deconvolution Generally, recovering u from u ith an unknon point spread function (PSF) K is called a blind deconvolution problem: u = K * u + n In our case, e don t kno the exact size of our blur kernel, but e kno it s Gaussian in shape. u = G a, * u+ n, G a, = ae So e have a partially blind deconvolution problem. In addition, e kno that the recovered signal u should be a binary -1 signal. ( x/ ) Ill-Posedness In general, the blind deconvolution problem is ill-posed. u = K * u + n So e cannot hope to correctly recover u from u. But our problem is only partially blind... Well-Posedness Selim Esedoglu (4) shoed that under certain assumptions, the TV minimization is ell-posed. If, a lies in a compact set S, and u {,1} satisfies u= a.e. outside the barcode region, then inf u, a S, is attained. [ u ] E u So e have a mathematical theorem supporting the use of the TV norm for barcodes! The Staircasing Effect It is ell-documented that TV minimization produces pieceise constant or blocky results. This is called staircasing or terracing. This is because the jump term does not control smoothness. u dx = But this is a desirable feature for recovering barcodes. 5

6 Computation We minimize over u E [ u u ] = u Ga u dx + β u, * dx using Neton s method. The update step is ( n+ 1) ( n) 1 u = u λh ue Note u is not differentiable for u =. We use the corner approximation u u + ε Numerical Results a=.5 =.13 Runtime 1.6 min The initialization can be taken as u () = identically. Limits of Blurring We can handle blur kernels up to =.13. This corresponds to a Gaussian idth roughly 3 times the idth of the module. Symbol s blur limit is 1.3 times the module. Finding a, We can simultaneously solve for a, hile finding the barcode u. We can recover the correct barcode u provided the initializations for a and are ithin 1% of their actual value. E[ u u ] = u Ga, * u dx + β u dx The energy E is convex in u, but not in a &. So e need a good prior estimate of the PSF. Symbol has a technique for estimating the size of the blur kernel using relative peak locations (Joseph- Pavlidis, 1994). 6

7 Deblurring Summary Using a global deblurring scheme allos us to handle barcodes ith higher blur, but......e need a good estimate of the PSF parameters....computing the Hessian matrix requires a lot of memory....it s far too slo to be practical. Still, it may prove useful to explore it further. Can e......estimate the PSF accurately enough?...incorporate the binary nature of the signal or the fact that the bar idths are integer multiples?...speed it up? Symbol s Imaging Scanner Scanlines are dran on the image until one decodes. Averaging scanlines improve performance, but e are basically using a laser scanner approach on an image. Barcode Super-Resolution The main advantage of laser scanners over imaging scanners is that scanlines from a laser scanner have higher resolution. But in an image, e have a lot of information beyond just a single scanline. The larger the image, the higher the signal resolution should be. Super-resolution: Can e use the hole D image to get a single high-resolution 1D signal? Image Size Resolution What Is Super-Resolution? Super-resolution means e increase the amount of information, not just the number of data points. Unfortunately, there is no ay to quantify the effective resolution (Sapiro 1). Original Super-resolution Doubling the resolution 7

8 Super-Resolution in 1D Scanline Signal Super-resolution signal Speed is an Issue The scanner needs to decode in milliseconds, so standard image processing techniques are too costly. Super-resolution methods Anisotropic diffusion Hough transform D Fourier transform Also, the scanner has limited memory, so a practical technique ould require only integer arithmetic. So hatever approach e use must be fast and simple. The Projection Concept A scanline only uses a small percentage of the image. We project all pixels on the Bresenham line to the scanline. The Projection Concept What if e project pixels off the scanline? Super-resolution: We can increase the effective resolution of the signal! Advantages 1. Fast to compute Disadvantages 1. Sensitive to local noise. Lo resolution (Pixel level) Advantages 1. Higher resolution (Subpixel level). More robust to noise Disadvantages 1. Sloer to compute. Projection unclear 8

9 U(x,y) Ho Should We Project? We should trace don each bar to the base of the barcode image. The pixels should be similar along the bar vectors. We are exploiting the unique symmetry of barcode images to convert a D image U(x,y) to a 1D signal u(t). u(t) (Does not apply to other images. Except pictures of zebras.) Why Is This Super-Resolution? If the image as perfect black and hite, then the projection ould not increase the resolution at all. Scanline: Projection: Perfect Image Scanline: Projection: Imperfect Image We increase the resolution because images are imperfect! We make use of the gray values! Projection by Angle θ Suppose our barcode is rotated by a knon angle θ. y y=xtanθ U(x,y) Project each pixel along a vector orthogonal to the u(t) projection line y=xtanθ. θ x A little trigonometry shos the projection U(x,y) u(t) is t = x secθ + z sinθ, z = y x tanθ Degenerate Sampling Trivially, the projection ill not help hen the rotation angle is or 9. Pixels are projected on top of each other -- degenerate sampling. θ= θ=9 We ill only have degenerate sampling hen tanθ is a rational number (θ=,45,9 ). 9

10 Non-Uniform Sampling Signal Filtering But hen tanθ is irrational, the sampling along the t-axis ill be non-uniform t Due to noise and discretization, the resulting projection signal ill be rather choppy. These spikes are easily cleaned up ith a mean or median filter. Projection Also, projecting ro by ro (raster order) ill result in a signal that needs to be sorted in ascending t. After Median Filter Example: Code 18-A Code 18-A barcode not decoded by current softare. PPM=1.1 Example: Code 18-A Projection Average Scanline Ideal 1

11 Example: Code 18-A Close-up of position 78. Example: Interleaved of 5 Non-decoded Interleaved of 5 ith rotation angle -4.6 Example: Interleaved of 5 Projection Average Scanline Ideal Example: Interleaved of 5 Close-up of position 43 11

12 Roll-Pitch-Ya y YAW ROLL Roll-Pitch-Ya We ve already taken care of of the 3 rotations. Roll: We ve shon ho to handle roll rotation. PITCH x PITCH Ya: We don t care about ya. 1. Ya is corrected for by self-clocking.. Ya ill not improve the resolution. ROLL YAW z Occur hen e rotate the barcode or the camera. But hat about pitch? Projecting under constant rotation angle θ ill not ork in the presence of pitch distortion. Pitch & The Focal Point Under pitch, all lines should converge to a single vanishing or focal point F. F We can project each pixel U(x,y) don a ray from F to a point on the base u(t). So ill F be too large for us to calculate? t The more pitch in the image, the closer F ill be to the barcode. If the bars are parallel, then F ill be at infinity. Focal Point Illusions Focal points are closer than they appear. F=(49.5,-3) F=(49.5,-3)

13 Ho Do We Calculate F? Focal Point Projection F A = Theoretically, all lines should intersect at F. So e calculate F by finding the point that minimizes the sum of the distance from the lines l 1, l,..., l n in the leastsquares sense. F n F = arg min d ( F, li ) F R i= 1 Writing each line in y=mx+b form, e calculate directly m CD BE AE BC Fx =, Fy = AD B AD B m m b n n n n n i i i i i, B =, C =, D =, E = i= 1 mi + 1 i= 1 mi + 1 i= 1 mi + 1 i= 1 mi + 1 i= 1 mi + 1 b 1 Suppose e have base points P 1 and P and e kno the focal point F. We can calculate the projection U(x,y) u(t) by d = t = P ( x Fx )( P1 y Fy ) ( y Fy )( P1 x Fx ) ( x F )( P P ) ( y F )( P P ) 1x + d x ( P P ) x 1y 1x y U(x,y) P 1 P t y u(t) 1x x Line Tracing To calculate the focal point F, e need to accurately trace the lines in the image. We track maximum contrast. This tracks lines accurately, but often it doesn t generate enough lines (>4) on lo resolution images. Needs improvement! Bounding Box We can also use the traced results to find a bounding box for our barcode. We find the linear regression through each set of points, using iterative pruning. UPC, 59 lines found This is important so e kno hich pixels to project. 13

14 Example: Code 39 Focal point projection on Code 39 image ith pitch angle 7. δ-projection If e project every pixel in the image, e ll end up ith a very long signal. We can simply project onto a fixed number of points along the base. If all points are a distance δ apart, then e our increasing the effective resolution by a factor of 1/δ. This approach controls the signal size, obtains a uniform sampling, and gives the points in sorted order. Scanline Projection δ-projection F x What should e do ith the pixels beteen the dotted lines? B Projection Line δ P Mean δ-projection We take the mean of all pixels hose center is ithin a distance δ/ of the focal line. u ( t) = u( nδ ) = ( x, y) x < δ / # x < δ / So e have combined the mean filter and focal point projection into a single step. U P 1 δ x -δ/< x< < x δ/ FB 14

15 Preliminary Results Early results for this method are promising. Test set: 71 misdecodes of a lo PPM Code 39 image. Example: Decoding the Misdecoded The projection uncovers small features in lo PPM images. Current Softare: 71 misdecodes (1%) no-decodes Average Time: 1ms Mean δ Projection: 8 correct decodes (39%) 43 no-decodes Average Time: 3ms PPM=1.3 Projection Scanline 5 Lines Traced Manually! Limitations The projection is not appropriate for Severely damaged images Short barcodes (PDF) Removing global noise, blur Non-projective distortion (printed on curved surface) Future Super-Resolution Work Improve line tracing algorithm crucial for both bounding box and focal point location. Improve running time: narro bounding box, detect offset in pixels, make independent of scanline. Adjust method for tough cases (damaged barcodes, printed on curved surfaces, high blur, etc.). Extend to D barcodes. Streamline the code: decrease computation, integer artihmetic, integrate ith current digitizer/decoder. Ho do e quantify resolution and signal quality? 15

16 That s All Folks! Thanks for listening! Send comments, questions, and criticisms to Todd Wittman ittman@math.umn.edu 16