An Improved Method for the Binarization in Structured Light 3D Scanning Systems

An Improved Method for the Binarization in Structured Liht 3D Scannin Systems An Improved Method for the Binarization in Structured Liht 3D Scannin Systems Chih-Hun Huan 1 1 Department of Information Manaement Lunhwa University of Science and Technoloy, Taoyuan 33306, Taiwan mikeh@ms10.hinet.net Abstract The binarization process for each pixel on the imae taken from a pattern-illuminated object plays an important role in structured liht 3D scannin systems that use Gray-coded stripe patterns to construct 3D spatial coordinates of the object. The process determines every scanned point on the object is illuminated by a briht or a dark stripe of the liht patterns. This paper proposes an improved threshold-based method for the binarization on 3D scanners. Compared with the traditional threshold-based scheme, the proposed method not only takes the noise enerated by hardware into account, but also resolves the difficulty in settin threshold. Moreover, compared with the widely adopted method usin complementary patterns, the proposed method reduces the scannin time. Keywords: Gray code, Structured liht 3D Scanner, Binarization 1. Introduction Conventional structured liht 3D scannin systems [1], [2] (or called rane sensors) consistin of a laser projector and one or more cameras have been developed for acquirin 3D information of objects in the last few years. Recently, to fulfill the oal of usin off-the-shelf consumer products and to resolve human s psycholoical antipathy to the laser liht, most of the newly developed 3D scanners adopt a white liht LCD projector [3], [4], replacin the laser projector. However, in the binarization process, usin an LCD projector makes the stripe number be more difficult to identify than usin a laser projector. This problem will be addressed in detail in the followin context. Theoretically, while performin a 3D scan, the emitter plane of the projector sequentially enerates a set of encoded patterns and projects them onto an object. There are many ways to encode the patterns. One of the most widely used encodin schemes, also used in this paper, is the Gray code encodin [5]. Every pattern enerated usin the Gray code encodin is made by a number of vertical stripes, which virtually divides the projector s emitter plane into parallel lonitudinal briht-and-dark stripes with equal width. Each stripe, labeled with a Gray code, on the emitter plane projects a virtual lihtin plane and illuminates the scanned surface. Then, the imaes of the illuminated object are captured by the

龍華科技大學學報第二十四期,2007.12 camera and stored in the system s memory. Since a line intersects an unparallel plane with an unique point, the intersection of the virtual lihtin plane and a ray, which is from a pixel on the camera s imain plane throuh the camera s focus, could be determined uniquely. If the extrinsic and intrinsic parameters of the camera and the projector have been known, a pixel on the imain plane toether with the associated virtual lihtin plane could determine the 3D spatial coordinates of the intersection on the measurin surface. This is the basic principle of structured liht 3D scanners, as shown in Fi. 1. Fi. 1. The basic principal of structured liht 3D scanners with stripe patterns Practically, the extrinsic and intrinsic parameters of the camera and the projector can be found by a calibration procedure, thus a mathematical model of the structured liht 3D scanner can be built. The only problem left to calculate the 3D spatial coordinates of the taret point is how to determine the virtual stripe number, which is an ordered number of every virtual stripe on the emitter plane. The stripe number can be determined by decodin the binarization result of each pixel on the imae taken, i.e. to determine whether or not the pixel associated to the scanned point is illuminated by a briht stripe. As a consequence, when every pixel on the captured imaes is correctly binarized, the 3D information of the taret surface can be acquired. Binarization of the illuminated imae is not so obvious. First of all, the variation of the ray values of a pixel, under the full illumination or the ambient liht, depends on the color of the correspondin taret point on the scanned surface. In addition, the ray value on the ede between illuminated and non-illuminated reions may be ray instead of pure white or black. Yet another reason that may cause the ray value chane is the illumination condition, such as the width of the projected briht stripes. In addition, there are more problems that make binarization difficult will be discussed in Section 2. Several methods have been proposed for the binarization. In the intuitive threshold-based method [6], every captured imae is compared with two reference imaes, one taken under the full uniform illumination (the pure-briht pattern) and the other taken just under the ambient liht (the pure-dark pattern). Due to the variability of the object bein scanned and the scannin system, to determine an optimum settin for the threshold-based method is not easy at all. One alternative called the threshold-free method, which is one of the most widely used methods recently [7], [8], could resolve the problem. The threshold-free method illuminates the object by both the coded patterns and their complementary patterns (the positions of the briht and the dark stripes are swapped). Then, the pixels of the imaes are

An Improved Method for the Binarization in Structured Liht 3D Scannin Systems compared while in the binarization. This method is very robust, however, due to the number of the patterns is doubled, it also increases the scannin time by a factor of two. The other alternative is the intensity ratio scheme [5], [9], which uses ray levels as the labels. One of its drawbacks is the system is quite sensitive to noise. In this paper, a fast and robust scheme for binarization is proposed. The proposed method is based on the intuitive threshold-based method with addition of an extra reference pattern, which is the complement of the finest pattern. Furthermore, the proposed method not only takes the noise enerated by the hardware into account, but also resolves the difficulty of threshold settin in the traditional threshold-based methods. Compared with the method usin a complete set of complementary patterns, the proposed method takes much less scannin time because only one complementary pattern is used. The remainder of this paper is oranized as follows. Section 2 introduces the encodin and decodin of the Gray-coded stripe patterns used in the 3D scannin systems. Some factors that cause difficulties in the binarization process are also investiated. Section 3 describes methods for binarization includin conventional methods and the proposed method. Section 4 shows some experimental results to demonstrate the practicability and feasibility of the proposed method. Finally, Section 5 summarizes the conclusions. 2. Pattern Encodin and Decodin Virtually dividin the emitter plane into parallel lonitudinal stripes with equal width and identifyin the associated stripe label on each pixel of imain plane can be achieved by the spatial-temporal modulation of the illumination, i.e., by sequentially projectin several patterns. In this paper, a set of widely used Gray-coded patterns consistin of vertical stripes with different width are used. To analyze a structured liht, projectin a Gray-coded pattern is superior to a binary code projection in ways. On one hand, successive numbers of the Gray code vary exactly in one bit, a wron decodin, which is most likely to occur at locations where one bit switches, introduces only a misplacement of at most one resolution unit. On the other hand, the width of the briht and the dark stripes in the pattern with finest resolution is twice as wide as that of the binary code. Theoretically, N, decoded the number of standard Gray-coded patterns, virtually divides the N projector s emitter plane into 2 parallel stripes with equal width. The n -th pattern contains 2 n 2 ( n 2 ) parallel dark stripes. For example, Fi. 2 shows 4 structured liht patterns (i.e., N = 4 ), which consist of a set of vertical stripes with different width. The emitter plane is divided into 16 parallel, lonitudinal, equal width stripes, each of which is labeled with a Gray code. Thouh the fully briht illumination is not included in the stripe decodin process, it provides a reference imae for decodin the stripes. In order to identify the label of the virtual stripe for each taret point, the

龍華科技大學學報第二十四期,2007.12 imaes of the scanned surface, which is sequentially illuminated by the Gray-coded patterns, are captured into the camera s imain plane and converted into bit planes accordin to the Gray code labelin. The bit plane of each projected pattern is obtained from the binarization process of the imae captured, i.e. to determine a binary ray level of each pixel on the bit plane accordin to the taret point is illuminated by either a briht or a dark stripe. When all bit planes are processed, the same N th-bits taken from the bit planes form an identification number, or the virtual stripe number, which is associated a taret point on the scanned surface. Based on the principle of structured liht 3D scanners stated in the above, the 3D surface information can be calculated. Thus, the binarization of the captured imaes plays an important role in the 3D reconstruction. In the binarization process, ettin the correct stripe number can be affected by several factors, includin the cross talk of the LCD projector, the resolution of the cameras, the reflectivity rate of the measurin surface, the taret point is out of focus of the camera, and so on. The cross talk of the LCD projector happens when voltae from an enerized element, or on element, leaks to an adjacent off element to turn it partially on. This makes the off element slihtly visible, as shown in Fi. 3. This is a common problem of TFT-based displays. Consequently, the pixel on the dark stripe but near the ede is brihter than the pixel on the center of the dark stripe. Furthermore, when the resolution of the imain plane is lower than the emitter plane, the pixel on the ede will have a partial ray level value dependin upon the ratio of briht and dark components, as shown in Fi. 4. A measurin surface projected by a hiher resolution pattern (finer stripe width) and taken by a low-resolution camera will make the imaes captured have a small brihtness variation under different illumination condition. The reflectivity rate of the scanned surface evidently affects the binarization process. For instance, a surface with a dark or briht color will make the decision threshold value different. Finally, both the projector and the camera in a 3D scannin system are both based upon the pin-hole theory, i.e., the object on the focus will have the sharpest imain result. However, only a part of measurin surface is riht on the focus in practice. Therefore, if the taret point is out of focus, the illuminated scene and the captured imae blur. If the foreoin influences are all taken into account, constructin a bit plane for each pattern becomes crucial in the 3D reconstruction. Fi. 2. Gray code stripe patterns ( N =4) Fi. 3. Cross talk of an LCD projector (enlared for observation)

An Improved Method for the Binarization in Structured Liht 3D Scannin Systems Fi. 4. Pixels on the ede between briht and dark stripes 3. Identifyin Stripe Number In this section, the alorithms of the traditional methods to identify stripe number are described in the Section 1. Then, an improved method is proposed. 3.1 Traditional Methods The intuitive threshold-based method compares the imaes captured with two reference imaes; one is taken under the full, uniform, and briht illumination and the other is taken just under the ambient liht. Let i denote the ray level value of the pixel on the imae when the related taret point is illuminated by the i -th pattern, 0 and 0( denote the ray level value of the pixel on the reference imaes taken under the full illumination and the ambient liht, respectively, and t v, 0(,i) denote the varyin decision threshold. The binarization alorithm is to classify a taret point between two classes accordin to the ray level value i of the associated pixel. The two classes are labeled by W or B, where class W (or B) denotes the taret point is illuminated by a briht (dark) stripe. In the intuitive threshold-based method, the taret point associated to the pixel on the imain plane is valid and is labeled by W for the i -th pattern if 0 0 > 0 and i 0 t( v, (,, i). Obviously, 0 u this alorithm is straihtforward. However, if takin the factors that affect the ray level value of the pixel into account, the settin of threshold is difficult. Alternatively, the most widely used and robust method for binarization is the threshold-free method that uses the coded patterns as well as their complementary patterns. Thus, if i i( 0, then the taret point will be classified into class W for the i -th pattern, where i denotes the ray level value of the pixel on the imae taken when the i -th complementary pattern is projected to the taret. This threshold-free method is robust, but it almost doubles the scannin time while compared to the threshold-based method. 3.2 Improved Method The proposed method is a combination of a threshold-based method and a threshold-free method. Like the threshold-based method, two reference imaes are taken under the full illumination and the ambient liht. In addition, an extra imae taken under the illumination of the complement of the pattern that has the finest stripes. Usin this scheme, the required 3D object scannin time is slihtly loner than that of the threshold-based method, and is about a half of the threshold-free method. Furthermore, the proposed method is better than the threshold-based methods in threshold settin. It automatically enerates a feasible threshold from the data clusters of the imaes. Thouh it is not really a threshold-free method, it can be considered as a threshold-settin-free method. Similar to the definition in Section 3.1,

龍華科技大學學報第二十四期,2007.12 i ( i = 1, L, N) denote the ray level value of the pixel on the imae taken when the i -th (i = 1, L, N) pattern is projected onto the associated taret point and N as the ray level value of the pixel on the imae taken when the complement of the N -th pattern is projected onto the associated taret point. At the end of a round of pattern projections, the followin parameters could be determined. A = max( 1(, 2, L,,, 1 N N v A = min( 1(, 2, L, B 1 = max( N, N ) B = min(, )) 2 N N v 2 N N v t i = ( i N) ( β 1 B1 + β 2 B2 α1a1 α 2 A2 ) + α1a1 + α 2 A ) t i = ( i N) ( B1 A1 ) + A1 ( t i = ( i N) ( B A + A 2 2 ) where α 1, α 2, β 1, and β 2 satisfyin α 1 +α 2 = 1 and β 1 + β2 = 1 are tunable parameters for the threshold. Generally, α1 = α 2 = β1 = β 2 = 0. 5 are chosen as defaults. Then, the binarization alorithm in this method is modified as ) follows. If i ( ti, then the taret point is illuminated by a briht stripe and ) labeled by W. If t i ( v ) > i ( ti, then the taret point is mostly illuminated by a briht stripe and is also labeled by W. On the other hand, if i < t i, then the taret point is rearded as bein illuminated by a dark stripe and is classified as B. Consequently, the bit plane correspondin to the imae of the i -th pattern could be determined. In this method, N is used as a reference to eliminate the effects caused by the reflectivity of the measurin object and the out-of-focus problem. The cross talk of the TFT-LCD and part of the reflectivity of 2, )) )) 2 the taret points are compensated by an approximated linear ratio i / N in the threshold settin. In addition, the effect of the low-resolution of the camera is considered by measurin the location of ) ( i between and. t i 4. Experiments The proposed binarization alorithm has been implemented in a 3D scannin system developed. To verify the proposed method, a set of 10 lonitudinal ray-coded stripe patterns is used ( N = 10 ) to label the emitter plane of the projector. This set of patterns divides the emitter plane into 1024 virtual stripes. Pattern 10 contains 256 parallel briht stripes and 256 parallel dark stripes with equal width. (The two half-width briht stripes at the both sides of each pattern can be combined and rearded as a full briht stripe.) That is, every stripe, either briht or dark, in pattern 10 spans 2 pixels in a 1024 768 XGA projector. To compare our results with that obtained from the method usin complementary patterns, the other 9 complementary patterns are also used for that method. A female statuette with about the same size as a real woman is used as the measurin object. The set of coded liht patterns, includin the 9 additional complementary patterns, is sequentially projected onto the object. When each pattern illuminates, a still diital camera in the 3D scannin system takes a picture. The imaes captured for the 10 ray-coded stripe patterns are shown, from Pattern 1 to 10, in Fi. 5(a)14(a). For each pattern, the binarization results of the method usin complementary patterns and the proposed t i

An Improved Method for the Binarization in Structured Liht 3D Scannin Systems method are shown, in the bit plane form, in Fi. 5(b)14(b) and Fi. 5(c)14(c), respectively. In those bit planes, a black (white) pixel indicates that the associated taret point is classified into class B (W). A ray pixel represents that the taret point is invalid because its ray level values, under both the full illumination and the ambient liht, are identical. For convenience of comparison, the difference between the two binarization results are shown in Fi. 5(d)14(d). From these fiures, it can be found that the difference exists only on the edes of briht and dark stripes. The causes of the difference are the factors that have been discussed in section 2, especially by the effects of the cross talk and the low-resolution of the camera. If takin a closer look, no continuous neihbor points can be found in Fi. 5(d)14(d). This confirms the accuracy of the proposed method as comparin to the method usin complementary patterns. The binarization results from the two methods were further used in an alorithm developed for the 3D reconstruction. Fi. 15(a) and (b) are the 3D recovered output o o (view anles 20 and 20 ) from the binarization results of the method usin complementary patterns, and Fi. 15(c) and (d) are the outputs from the proposed method. It can be seen that the quality of the recovered 3D imaes is maintained by usin the proposed method. 5. Conclusion In this paper, an improved threshold-based method for the binarization of scanned points in structured liht scannin systems has been proposed. Compared with the intuitive threshold-based scheme, the proposed method not only takes the hardware influences into account, but also resolves the difficulty of threshold settins. Compared with the method usin complementary patterns, the proposed method reduces the scannin time. Experimental results show that the quality of the recovered 3D imaes by the proposed method is ood. References 1.Petrov, M., Talapov, A., Robertson, T., Lebedev, A., Zhilyaev, A., Polonskiy, L.: Optical 3D Diitizers: Brinin Life to the Virtual World. IEEE Computer Graphics and Applications, Vol. 18. (1998) 28-37. 2.Borhese, N. A., Ferrino, G., Baroni, G., Pedotti, A., Ferrari, S., Savare, R.: Autoscan: A Flexible and Portable 3D Scanner. IEEE Computer Graphics and Applications, Vol. 18. (1998) 38-41. 3.Rocchini, C., Cinoni, P. Montani, C. Pini, P. Scopino, R., A low cost 3D scanner based on structured liht. Computer Graphics Forum, Vol. 20. (2001) 299-308. 4.Wen, C., Chan, K-C., Yeh, M-F.: A low-cost, user-friendly, and real-time operatin 3D camera. IEEE Transactions on Consumer Electronics, Vol. 51. (2005) 1085 1091. 5.Inokuchi, S., Sato, K., Matsuda, F.: Rane-iamin System for 3D Object Reconition. International Conference on Pattern Reconition, Vol. 2. (1984) 806-808. 6.Horn, E., Kiryati, N.: Toward optimal structured liht patterns. Imae and

龍華科技大學學報第二十四期,2007.12 Vision Computin, Vol. 17. (1999) 87-97. 7.Sato, K., Yamamoto, H., Inokuchi, S.: Tuned Rane Finder for Hih Precision 3D Data. International Conference on Pattern Reconition, (1986) 1168-1171. 8.Sato, K., Inokuchi, S.: Three-dimensional Surface Measurement by Space Encodin Rane Imain. Journal of Robotic Systems, Vol. 2. Sprin (1985) 27-39. 9.Carrihill, B., Hummel, R.: Experiments with the Intensity Ratio Depth Sensor. Computer Vision, Graphics, and Imae Processin, Vol. 32. New York (1985) 337-358. Fi. 5(a) Fi. 5(b) Fi. 5(c) Fi. 5(d) Fi. 6(a) Fi. 6(b) Fi. 6(c) Fi. 6(d) Fi. 7(a) Fi. 7(b) Fi. 7(c) Fi. 7(d) Fi. 8(a) Fi. 8(b) Fi. 8(c) Fi. 8(d)

An Improved Method for the Binarization in Structured Liht 3D Scannin Systems Fi. 9(a) Fi. 9(b) Fi. 9(c) Fi. 9(d) Fi. 10(a) Fi. 10(b) Fi. 10(c) Fi. 10(d) Fi. 11(a) Fi. 11(b) Fi. 11(c) Fi. 11(d) Fi. 12(a) Fi. 12(b) Fi. 12(c) Fi. 12(d) Fi. 13(a) Fi. 13(b) Fi. 13(c) Fi. 13(d)

龍華科技大學學報第二十四期,2007.12 Fi. 14(a) Fi. 14(b) Fi. 14(c) Fi. 14(d) Fi. 5(a)14(a): The rabbed imaes for patterns from 1 to 10. Fi. 5(b)14(b): Binarization results of Fi. 5(a)14(a) by usin complementary patterns. Fi. 5(c)14(c): Binarization results of Fi. 5(a)14(a) by usin the proposed method. Fi. 5(d)14(d): Difference between the two binarization results in Fi. 5(b)14(b) and Fi.5(c)14(c). Fi. 15(a) Fi. 15(b) Fi. 15(c) Fi. 15(d) Fi. 15. (a)(b): 3D recovered output by usin the binarization result from the method usin complementary patterns. Fi. 15. (c)(d): 3D recovered output by usin the binarization result from the method proposed in this paper.

An Improved Method for the Binarization in Structured Liht 3D Scannin Systems 結構光三維掃描系統之二值化程序的改進方法黃志宏龍華科技大學資訊管理系桃園縣龜山鄉萬壽路一段 300 號電子郵件 :mikeh@ms10.hinet.net 摘要對於以格雷編碼條紋結構光來建立被掃瞄物件之三維座標的掃描系統而言, 被掃瞄物件受結構光照射後所產生的影像, 其每一個畫素的二值化程序是非常重要的此二值化程序乃是決定每一畫素所對應的掃描點是被結構光圖案中之亮條紋或是暗條紋所投射本論文針對三維掃描器的二值化程序, 提出一個以臨界值方法為基礎的改進方法與傳統的臨界值方法比較, 本論文所提出的方法不僅將三維掃描器硬體的雜訊納入考量, 同時也解決了傳統上臨界值選擇上的困難此外, 與普遍使用互補結構光的三維掃描器相較, 本論文所提出的方法亦減少了三維掃描所需的掃描時間關鍵字 : 格雷碼, 結構光三維掃描器, 二值化