Scanning Fringe Projection for Fast 3D Inspection

Transcription

1 Scanning Fringe Projection for Fast 3D Inspection Marc Honegger a, Michael Kahl a, Sandra Trunz a, Stefan Rinner a, Andreas Ettemeyer a, Patrick Lambelet b a NTB University of Applied Sciences, Buchs, Switzerland, b Heliotis AG, Root Längenbold, Switzerland ABSTRACT In an earlier paper we have described a concept for high speed 3D inspection using fringe projection techniques. We use a special CMOS camera with 300 x 300 px which can calculate the phase on board. The focus of the first step of development had been a fringe projector, which was able to modulate the projected fringes with up to 250 khz. In the second step the image acquisition part of the system was developed. In case of 3D measurement with a matrix camera, the camera resp. the measuring object has to be moved stepwise in the lateral direction to cover multiple acquisition areas of the measurement object. Between each image the lateral movement has to correspond to the field of view of the camera. At the intended very high image acquisition rates the high acceleration of the system between each image will lead to inacceptable mechanical forces. In order to obtain a continuous scanning procedure and at the same time to use the performance of a matrix camera, a special lens system was developed. A measurement field 120 mm long and 3 mm wide is imaged onto the camera. The width of the measuring field is imaged onto the 300 rows of the camera, giving a lateral resolution of 10 m. In the longitudinal direction the 120 mm object length is divided into lines to reach the same resolution of 10 m. Due to technical reasons that will be described later only 240 of the 300 pixel rows on the chip were used. Consequently, with each camera shot 240 separated lines are imaged onto the chip. Between each of these 240 lines there is a large empty space, which is not imaged by the camera. In principle, the camera is operating as 240 single line cameras. Therefore, if the camera is shifted in an inclined direction to the camera orientation over the object, the empty spaces can be recorded as well. In an optimum alignment, the complete measuring object can be scanned in a continuous movement, covering the total length of 120 mm. In this paper we will describe this image acquisition system and give first measuring results. Keywords: Fringe projection, high speed measurement, 3D measurement, lock-in principle, 3D inspection 1. INTRODUCTION In an earlier paper we have described a concept for high speed 3D inspection using fringe projection techniques. We use a special CMOS camera with 300 x 300 px which can calculate the phase on board. The focus of the first step of development had been a fringe projector, which was able to modulate the projected fringes with up to 250 khz. The camera uses a lock-in principle, to directly determine the phase on board from 4 images with a well-defined phase shift. Therefore the projected fringes have to be shifted by /2 between each image. An appropriate projection technique has been developed and presented in [1]. In a second step the image acquisition part of the system has been developed. In case of 3D measurement with a matrix camera resp. the measuring object has to be moved stepwise in lateral direction to cover multiple acquisition areas of the measurement object. Between each image the lateral movement has to correspond to the field of view of the camera. At the intended very high image acquisition rates the high acceleration of the system between each image will lead to inacceptable mechanical forces.

2 2.1 Line Scan Principle 2. GENERAL DESIGN OF THE SETUP In order to achieve a continuous scanning process, the application of a line camera would be ideal. A single line camera typically has a higher lateral resolution than a matrix camera and a smooth movement can be obtained for scanning the complete surface. The scanning speed is mostly limited by the data transfer rate between camera and computer. In our case, the goal was to increase the inspection speed by using a special camera, which directly provides amplitude and phase of projected fringes and therefore allows some pre-calculation and data reduction directly in the camera. However such cameras are only available as matrix cameras. The required lateral and vertical resolution of the measuring system is 10 x 10 x 10 m 3. Therefore, the observation of a complete printed circuit board or wafer requires scanning the surface. Very high scanning rates with matrix cameras are limited by the high required mechanical forces to accelerate the camera (or respective object) mass to the next measuring position. With continuous movement, most of the time would be required waiting for scanning the areas, which have already been observed with the matrix camera. Therefore, we developed a concept to change the optical configuration from a matrix camera to a multi-line camera, Figure 1. Figure 1. Left: Principle of projecting multiple lines onto a matrix camera. Right: If the multi-line image is moved in an inclined direction, the complete surface can be covered with continuous scanning. If we project only single lines of the object surface onto single lines of the camera pixels and keep empty space in between those lines we open the opportunity to move the camera laterally by one line and get a complete new surface, generated by the new object lines. In our case the camera has 300 x 300 px and we need a lateral resolution of 10 m. In order to obtain a continuous scan we have to move the object by 10 m in both directions. In this way the complete object surface can be scanned in a continuous movement without overlapping areas. 2.2 Optical Design of the Imaging System The task of the imaging optics is to image separated lines with 10 m width and 3 mm length onto the camera chip with 300 x 300 px and a x- and y-pitch of 40 m resulting in a chip size of 12 mm by 12 mm. This requires a very strong anamorphic behavior of the optic system. Therefore, the complete optical system was designed with cylinder lenses. The

3 setup is shown schematically in Figure 2. The cylinder lenses are shown as convex lenses in the direction where they have optical power and as rectangles in the direction where they act like plane-parallel plates and are optically inactive. The upper drawing shows the x-direction. The optically active lenses f 3 and f 4 image the object (3 mm, right side) onto the camera chip (12 mm) on the left side. So in this direction the objects has a magnification by 4. The reason for using two lenses as a pair is simply that the system was to be built with elements that could easily be purchased from manufacturers catalogues: lenses with the desired focal length couldn t be found and so a combination of two was chosen instead. At the position of the intermediate image a stop A is mounted. The lower drawing in figure 2 shows the y-direction. In this direction, lines with 10 m width have to be imaged. In between the lines a gap of 500 m is required. Therefore a microlens array (MLA) f 1 with cylindrical microlenses was installed in front of the measurement surface. The imaging optics collects the light of each microlens with an afocal system (lens pair f 2 and single lens f 5 ) and images it to the chip through a stop B. The cylindrical MLA has a pitch of 500 m and dimension 120 mm x 12 mm, in other words there are 240 microlenses on the MLA. Since each microlens of the array is to be imaged onto one of the pixels of the camera chip, only 240 of the available 300 chip pixels are used. Again, this has no technical reason but is due to the fact that MLAs with the relatively large dimension of 120 mm in length are difficult to find and the one that suited our requirements best was one with pitch of 500 m. Figure 2. Design of the line scan optics: Top: x-direction magnification by factor 4 (yellow elements). Bottom: y-direction magnification by factor 1/12.5; the cylindrical microlens array collects only light from separated lines (green elements). The functionality of this optical system can be shown in the example of Figure 3. Top of the figure shows a surface with a pattern. The marked rectangular area is imaged onto the chip. The camera sees only separated lines as marked in the enlarged image area. Since each separated line is imaged adjacent on the chip the camera image looks as shown in the bottom of the figure. This camera image can hardly be interpreted by eye. It is a compression of all 240 single separated lines. If the camera is now moved horizontally in 10 m steps and images are recorded, the recombination of all separate images provides the original structure of the surface in the combination image, Figure 4.

4 Figure 3. Top: sample surface with pattern; Center: enlarged surface area with indicated lines, which are imaged; Bottom: camera image (compressed lines). Figure 4. Recombination of many individual line images in one combination image.

5 2.3 Design of the Projection system As described in [1] a fast fringe projector was developed, which allows shifting the fringes laterally with up to 250 khz. The concept uses 4 identical fringe patterns which are projected by individual LEDs onto the measuring surface, Figure 5. During the experimental tests of this principle it became apparent that the limiting factor of the system, with respect to measuring speed, is the amount of light on the surface. The design with four beam splitters results in a nominal efficiency of the system less than 6%. Therefore an improved design was required. Figure 5. Principle of a fast fringe projector with 4 individual LEDs and fringe patterns. The principal reason for low efficiency is the combination of many beam splitters. Therefore the new design reduces the number of beam splitters to 1 for each LED projection path, giving a theoretical maximum efficiency of 50% (not taking into account losses at lenses, apertures, etc.). Figure 6 shows this design. The fringes are projected from different angles onto the surface. A change of the illumination direction causes an error in height measurement of the system. As a compromise the angle is limited to ±4 to the theoretical optical axes. The projecting of fringes with identical grating constants under different angles would cause different grating periods on the surface. In order to compensate for this error the grating periods of the grating slides have been adjusted. For practical reasons (size of microbench optics, etc.) it was not possible in our setup to further reduce this angle and to project each fringe pattern individually. Therefore, two of each fringe projection systems were combined with one beam splitter and both pairs were arranged symmetrically around the optimum axis. The mode of operation of the projection system is as follows. There are four LED light sources used to illuminate the grating in a homogeneous manner. Therefore, two lenses serve as light collecting optics. The light of two LEDs is combined by a beam splitter and constitutes one of the two projection paths. In each of the two paths at a distance from the fringe pattern a lens images the pattern onto the surface ensuring the right number of lines per mm in order to reach the desired resolution. In order to comply with the condition of maximum amount of light on the surface one would prefer short focal lengths and short distances. Contrary to that are the restrictions of the mechanical setup. As a consequence, in order to reach the full measurement speed the projection system is required to be optimized in the future. There are different ideas of how this could be achieved, such as e.g. designing an integrated approach with multiple light sources or to select other than LED light sources.

6 Figure 6. Modified fringe projection system with two pairs of projectors. 3. EXPERIMENTAL SETUP Figure 7 shows the complete setup of the optical system. The top part is the LED fringe projection system, the bottom part the line scan optics (imaging system) with the fast lock-in camera (Heliotis AG, [2]). Just in front of the measuring system the microlens array is positioned at focal distance. The setup is built up symmetrically to obtain as much light as possible. Both, the projector and the line scan optics are aligned with an angle of 22.5 to the surface normal of the measuring object. Hence, the triangulation angle between projector and camera is 45. Figure 7. Experimental setup of the complete scanning system. The projection system was built up with standard microbench components. For the line scan optics we designed a similar system with larger dimensions to carry the relatively large optical elements. All optical components and lenses were standard commercial systems, Figure 8.

7 The fringe projection system covers only a 3 mm section of the 120 mm line length. In order to measure the complete lines in one image, several such systems should be mounted on each other. This does not seem practical with the experimental setup as shown in Figure 8. However, for industrial application powerful projection systems with multiple LEDs can be realized [3]. The triggering of camera and projector is realized by an FPGA. This device allows controlling the timing with an accurateness of a microsecond. A constant current source is used to supply each LED. Nevertheless, because of manufacturing tolerances of the LEDs, a perfectly uniform light distribution cannot be guaranteed. These distinctions result in an increased measurement uncertainty of the overall system. Figure 8. Optical setup Left: Line scan optics; Right: Projection system 4. MEASUREMENT RESULTS With this system first test results could be realized. The camera delivers phase and amplitude values for each measurement simultaneously. Only the phase values are needed to determine the height of an object. But the amplitude values offer an important indication for the quality of the measurement. Only measurements with minimal amplitude of 30 are processed and displayed in our experiments. 4.1 Calibration Phase shifting theory assumes constant background intensity and constant contrast while the phase is shifted. In our case, the phase shifted fringes are produced by independent fringe projection systems including separate LEDs. Therefore this basic condition will certainly not be completely fulfilled and a calibration procedure was carried out to characterize the behavior of the system and compensate systematic errors. As a calibration object a flat surface was positioned in front of the scanning system and moved in well-defined steps into the z-direction. In each position the phase was calculated for each point on the surface. The uncalibrated measured phase values in the range [-π, π] are shown in Figure 9 left. With these given supporting points a polynomial fit was calculated for each pixel to convert any phase measurements into a height value. For further measurements, the coefficients of the polynomial fit have to be recorded. In our experiments, a polynomial of degree three was chosen and the coefficients for each pixel were calculated with a least square algorithm from seven supporting points. In this way a compensation algorithm could be applied to real measurements. Figure 9 shows the position of the plane surface before calibration and after calibration.

8 3 2 1 Height Comparison 5.20mm 5.25mm 5.30mm 5.35mm 5.40mm 5.45mm 5.50mm mm 5.25mm 5.30mm 5.35mm 5.40mm 5.45mm 5.50mm Phase [rad] 0-1 Height [mm] Position [Pixel] Position [Pixel] Figure 9. Different positions of the calibration plane before calibration (left) and after calibration (right) 4.2 Measurement First test measurements were carried out. Due to the ealier described limitations of this first experimental setup only a demonstration of the functioning principle can be expected. Figure 10 shows a piece of a wafer with a 95 m deep structure. The wafer was sprayed with a white powder to avoid reflections. A diffusely reflecting surface is optimal for triangulation. On the other hand this results in little light efficiency. Therefore the image acquisition speed is limited, too. The fringe modulation frequency was 2.5 khz but each image was integrated over 10 modulation cycles. This results in a measuring frequency of 250 Hz. Because of the uniform colouration of the measurement piece, the amplitude should show a more or less constant value over the whole surface of the object. As can be seen, the projected grid is still visible in the amplitude image (Figure 10 top right). This is a result of the improper illumination conditions and imperfect calibration. Of course these errors also show up in the 3D presentation of the height measurements which are shown in Figure 10 bottom left. The graph shows a ripple along the projected grid of ca. 10 m. The cross section in Figure 10 bottom right gives a step height of m which nicely corresponds with the actual dimension. If the specular surface is not sprayed, much more light is collected by the camera due to the symmetric setup. In comparison a wafer structure was measured in this state with a modulation frequency of the projector of 10 khz, and just 1 image acquisition cycle resulting in 10 khz measuring frequency. Of course direct reflections lead to additional and unwanted phase information (reflective surfaces should be analysed with deflectometry algorithms, [4]). However this example demonstrates that the line scan optics and the projector principle can be used for fast measurement frequencies, too.

9 Amplitude Y [Pixel] X [Pixel] Position [Pixel] Figure 10. Measurement of a white sprayed wafer structure: Top left: Image of the wafer and definition of the field of view; Top right: Measured amplitude; Bottom left: 3D presentation of the structure; Bottom right: Cross-section along the indicated line Height [mm]

10 Figure 11 shows the image and the result of a wafer with a specular surface, which is also structured with a depth of 95 m. In the 3D presentation of the height, the phase ripples can again be identified. Furthermore, there are holes in the measurement where direct reflections leads to saturation of the camera or wrong phase information. Figure 11. Measurement of a PCB: Left: image and definition of the field of view; Right: height measurement A second example is a printed circuit board (PCB) with solder pads, Figure 12. Again the sample was sprayed to avoid direct reflections. The height difference between the solder resist mask and the pads is about 30 m. In the 3D presentation the ripple effect of the improper phase illumination is visible again, but the solder points are easily visible. Figure 12. Measurement of a PCB: Left: image and definition of the field of view; Right: height measurement

11 5. RESUME An optical system for fast fringe projection was developed. The principle could be verified and tested. The working principle of the imaging optics (line scan optics) exceeded all expectations and opens the possibility to use matrix cameras as multi line scan cameras. Even with a simple setup with standard optical components good image quality and sufficient resolution could be achieved. The main problem of fast scanning is the lack of light available. Therefore, the main focus for industrialization has to be laid on the projection system. An integrated approach with multiple LEDs should be used. Test results showed that the intended resolution could be achieved. Measuring speed is mostly dependent on the amount of light available. ACKNOWLEDGEMENT This project was generously sponsored by the Swiss Commission for Technology and Innovation CTI (proj. no PFNM-NM). We wish to thank CTI for this support and our project partners Heliotis AG and Essemtec AG for their support and constructive discussions. REFERENCES [1] Caspar, S., Honegger, M., Rinner, S., Lambelet, P., Bach, C., Ettemeyer, A., "High speed fringe projection for fast 3D inspection", Proc. SPIE 8082, Optical Measurement Systems for Industrial Inspection VII, 80820Y (May 26, 2011). [2] Lambelet, P., "Parallel optical coherence tomography (poct) for industrial 3D inspection", Proc. SPIE 8082, Optical Measurement Systems for Industrial Inspection VII, 80820X (May 26, 2011). [3] Sieler, M., Schreiber, P., Dannberg, P., Bräuer, A., "Array projection optics: multi-channel design for ultra slim projectors", Proc. SPIE 7716, Micro-Optics 2010, 77161A (May 13, 2010). [4] Faber, C., Olesch, E., Krobot, R., Häusler, G., "Deflectometry challenges interferometry: the competition gets tougher!", Proc. SPIE 8493, Interferometry XVI: Techniques and Analysis, 84930R (September 13, 2012). [5] Beer, S., "Real-Time Photon-Noise Limited Optical Coherence Tomography Based on Pixel-Level Analog Signal Processing", Dissertation, University Neuchatel, Switzerland, (2006).