Intelligent Measurement Processes in 3D Optical Metrology: Producing More Accurate Point Clouds

Transcription

1 Intelligent Measurement Proesses in 3D Optial Metrology: Produing More Aurate Point Clouds Charles Mony, Ph.D. 1 President Creaform in. mony@reaform3d.om Daniel Brown, Eng. 1 Produt Manager Creaform in. dbrown@reaform3d.om Patrik Hébert, Ph.D. 1 3D Optial Metrology Advaned Researh Creaform in. patrik.hebert@reaform3d.om 1 Creaform 5825, rue St-Georges Lévis (Québe) Canada G6V 4L2 Word Count: 4488 Number of Images Submitted: 6 Abstrat This paper introdues the new paradigm of intelligent measurement proess in the ontext of 3D optial metrology. Sine new 3D sensor tehnologies an apture 3D points at very high rates on the surfae of objets, it is now possible to quikly apture very dense sets of points on an objet s surfae. This very large quantity of 3D point observations allows one to examine the loal distribution of these points on setion areas before validating their onsisteny with the expeted error model developed at the alibration stage. From these dense point observations, higher quality surfae point measurements an be produed for metrology. An intelligent measurement proess will integrate these steps in real time during apture. We desribe a framework adapted for implementing suh a proess. The result setion presents some appliation ases in metrology. The paper onludes with prospetive omments on intelligent measurement systems. 1. Introdution It is interesting to note that sientifi and tehnologial advanes have always been tightly oupled with the improvement of measuring systems. The 21 st entury has already witnessed the explosion of the ability to measure in several domains. Probably the most signifiant examples are the experiments with the CERN s Large Hadron Collider, eah generating several terabytes of observation data. Closer to our domain, existing 3D optial sensing tehnologies allow quik generation of a tremendous amount of dense points on the surfae of an objet. Indeed, apturing to more than a million 3D points per seond is no longer exoti. This ever growing apability to ollet observations ombined with the improved omputational tehnologies not only paves the way to, but leads us inevitably towards the new paradigm of intelligent measurement. Intelligent measurement is the proess that produes onsistent measurement obtained from a set of basi observations, and whih onsisteny is verified against a model of the measuring proess (the aquisition model) during apture. Reading an ambient temperature of 60 C in a onferene room from a thermometer, or reading a diameter of 100 mm for a oin with a aliper, obviously makes no sense. One would at least make one or several additional readings before providing a reliable measurement or reporting a problem with the devie. Indeed, the observation is not onsistent with the expeted range of values in the urrent ontext of measurement. Several

2 observations (readings) would be neessary to be able to onfirm the onsisteny of eah observation. Consisteny is verified with respet to the aquisition model whih integrates knowledge on the measuring devie and hypotheses on the objet to be measured. These knowledge and hypotheses may be as simple as the typial range of values and the resolution, repeatability and auray of the measurement that an be produed by the devie. When observations are onsistent, improved measurements an also be produed after reduing error due to noise, from several observations. Let s now onsider that the aquisition model is integrated within the measurement proess; that would yield an intelligent measuring system. In the preeding examples, the human is the intelligent agent in the measurement proess. Preferably, intelligene would be integrated into the omputerized system when the quantity of observations inreases and beomes intratable for the human. Integration of intelligene then beomes partiularly valuable with the enormous amount of observations that an be aquired in real time with 3D optial sensors. In this paper, it is shown how to build an intelligent measurement proess in 3D optial metrology. After explaining why optial sensing tehnologies are well suited for the new paradigm in setion 2, we will revisit the whole paradigm of raw 3D points in setion 3. Atually, in any measuring system, 3D point oordinates are the results of several alulations on the observations. Moreover, their noise error distribution is not stritly idential. Setion 4 develops on the priniples for developing an intelligent measuring system. In setion 5, we present the bases of a framework developed for implementing an intelligent measurement proess. Setion 6 shows various results following the appliation of the framework. Finally setion 7 disusses the prospetive in this new paradigm. 2. Capturing a large number of 3D points with optial sensors In metrology, it is ommon pratie to inspet parts by extrating measured harateristis from geometri entities suh as spheres or planes, for instane. The harateristis are based on a limited number of alulated points from touh probe or high density loud of points from an optial 3D sensor. It is also more and more ommon to evaluate the geometri onformity after omparing a 3D point loud with a referene CAD model. During this inspetion, statistis are omputed on the distane between 3D points and their putative mathing points that are the losest on the aligned surfae model. Besides measuring points at higher density on the surfae, optial sensors make it possible to avoid ompensation for the radius of a tatile probe during alignment. Due to the higher point density, it is also possible to apture a dense set of points that are very lose to the boundaries and rease edges. One must keep in mind that the primary objetive while inspeting a part is not to measure 3D points, but the onformity with geometri dimension and tolerane model (GD&T) in order to respet funtional speifiations and assembly of the produt using surfae geometri harateristis. Typially, 3D points measurement is a preliminary step beause it is a simple and flexible way to extrat surfae harateristis and ompare two surfaes. For instane, one may onsider measuring the distane or the angle between two planes. It is lear that 3D points onsidered solely are of no utility sine they annot be ompared. It is the underlying surfae shape that is of interest. Measuring a large number of points makes it possible to better define the surfaes and entities for measuring harateristis but, furthermore, a better sampling will also allow one to haraterize the error loally. The very large number of observations that is possible with a 3D optial sensor allows the appliation of the new intelligent measurement paradigm desribed below.

3 3. Revisiting the paradigm of raw 3D points In metrology, it is ommon pratie to measure harateristis or to ompare a 3D point loud arising from a sensor to the referene surfae. These observations are onsidered raw points, whih errors are independent and identially distributed. Unfortunately, raw points do not exist, and it is erroneous to onsider these measurements as raw observations. They are rather the result of several alulation steps. For instane, in the ase of a CMM-mounted touh probe, basi observations are olleted from the linear or angular enoders of the system. A 3D point is then alulated from the parameters of the mehanial model of the devie and the probe. These parameters are alibrated beforehand. Any error in these parameters will affet several 3D points differently and will not be independent from the measured geometry (e.g.: starting from a simple radius orretion of the probe). A similar situation arises with optial sensors. Let s onsider the ase of a triangulation laserbased profilometer omposed of a amera and a laser projetor. The amera aptures light and produes a 2D pixel image of the laser trae. Figure 1 illustrates a typial image setion aptured on an objet. From this image, subpixel positions of the laser trae will be extrated. This image proessing alulation involves a set of neighboring pixel values. One these 2D positions are alulated, the system s alibration parameters are exploited to alulate the set of 3D points. These alulations will involve image distortion orretion and projetion of a ray in spae from the amera s intrinsi parameters. Then, the extrinsi parameters desribing the spatial relationship between the amera and the laser projetor will also intervene in the omputation before produing a 3D point. It is thus worth mentioning that 3D points output from sensors are the result of several omputations. Their distributions are stritly neither idential nor independent. One example is shown in Figure 1, enter and right. Figure 1. Image setion of a laser trae. From this image, 2D subpixel positions of the laser trae will be extrated before omputing 3D points based on triangulation. Center) Cross-setion aptured on a orner objet. Right) Distribution of the 3D points along the ross-setion. The inset shows how the distribution of the 3D points differs near the orner. Another example is shown in Figure 2 where 3D points were aptured on a sphere from one viewpoint in the first ase and from several viewpoints in the seond ase. The amount of observations and the overed setions on the sphere are equivalent. It is well known that the inident angle of observation will affet the error distribution of 3D points. For better results, one will make sure to apture observations from various diretions.

4 Figure 2 Measuring the diameter of a sphere from a single viewpoint of from several viewpoints with approximately the same amount of observations in the same area. This example shows that the distribution error of 3D point observations is affeted by the angle of inidene. The error is not identially distributed. The measurements were aptured using the EXAsan TM tehnology 1. In onlusion, raw observations for 3D optial sensors are pixel intensities. In the urrent generation of 3D optial sensors, 3D points generated should be onsidered at least - as a seond level of observation. One annot state that these points are raw data; they are in fat the results of a hain of alulations. In pratie, to make sure inspetion will be valid, one will manually validate data aquisition before applying the inspetion proedure. 4. Towards the new paradigm of intelligent measurements The new intelligent measurement systems are also produing 3D point louds or verties. However, the 3D point loud is generated and validated by the intelligent measurement system. For this purpose, the system will hek onsisteny of the olleted 3D observations with respet to an aquisition model. From a set of onsistent 3D observations, more aurate 3D points, namely 3D measurements, will be produed. Atually, if one onsiders a single 3D point, i.e. a set of three oordinates (x,y,z) in 3D spae, nothing an be said unless the position or the orientation of the sensor is provided. One may also provide the expeted error distribution assoiated with the point. That would be helpful for identifying the orresponding point on the surfae of the objet during inspetion. That would also imply that the distribution is idential for every observation. Nevertheless, for any type of sensor, the error may differ loally due to unmodelled phenomena. The ase shown in Figure 1 (right) is one example. For instane, the error distribution for optial sensors an be affeted by albedo (refletane) variations on the objet s surfae, by the angle of inidene between the measurement ray and the surfae normal or even by inter-refletions in onavities. One important harateristi for an intelligent measurement system is to identify these situations and avoid produing measurements suh as these in the examples of the temperature or oin diameter measurements in the introdution.

5 To reah this goal, the system must gather several observations, model the loal distribution of these observations and provide, from these observations, a set of measurements where the distribution is onsistent with the aquisition model. This requires olleting redundant observations and testing the onsisteny after a suffiient number of observations are gathered. Final measurements will be alulated from these observations. Besides testing the number of aumulated observations on a surfae setion, the system an also monitor the variety of viewpoints from whih observations have aptured. The high measurement rate of optial sensors is well-suited for this task. In order to maximize the benefit of monitoring the observations, it needs to be done in real time simultaneously to the aquisition, instead of during the postproessing of the resulting point loud. 4.1 Building an aquisition and alibration model The aquisition model defines the expeted observation in a given appliation ontext. For instane, while observing a planar opaque surfae in the diretion normal to the surfae, at a given distane, the average and ovariane matrix of 3D observations (x,y,z) an be modeled beforehand, either empirially or analytially. This refers to the repeatability of the measurement. One will also evaluate the auray of the system using an independent referene for omparison. Although the performanes of a system an be assessed independently, the haraterization is also performed during alibration with a referene objet. Of ourse, the aquisition model an be more sophistiated while onsidering a wider spetrum of aquisition onditions or, onversely, modeling very speifi onditions for a dediated appliation (e.g. measuring the skin surfae geometry of a human body). Following the ase of a triangulation-based optial sensor, the proedure enompasses the geometri alibration of the amera and optis as well as the alibration of the laser projetor. In the first ase, a photogrammetri amera model may be used. The intrinsi (internal) parameters inlude the prinipal point, pixel sale and eventually skew as well as distortion typially modeled with radial and tangential parameters. One must also alibrate the extrinsi (external) parameters of the struture assembly with the amera and the laser projetor. These parameters are neessary for applying 3D triangulation. Sine the laser projetor might not be perfet, its distortion must also be modeled. An alternative to the expliit alibration parameters would be building an impliit model in the form of lookup tables. In this ase, the sensor is moved preisely relatively to a alibration target, and the positions of the laser trae in the image are extrated and registered at eah inremental step. These are lassial approahes and the quality of the measurement will depend on the quality of hardware omponents, the alibration target as well as the proessing algorithms. One an go one step further with the intelligent measurement system. Atually, one an apply different alibration models depending on the environment. For instane, it is possible to adapt the aquisition model based on the distane between the sensor and the surfae of the objet or based on environmental measurements suh as temperature that will affet the physial arrangement of the sensor. This provides the sensor with basi intelligene sine it adapts to the onditions. One an still go further and haraterize the influene of the measured geometry on the noise level by onsidering the surfae angle of inidene or the loal urvature. This reahes a higher level of intelligene sine the system must apture observations and estimate the environmental onditions inluding the observed shape before refining the parameters of the aquisition model. In the end, measurements of better quality an be obtained with a given set of hardware omponents.

6 5. Data struture and surfae representation for intelligent measurements In addition to providing an aquisition model, a system apable of intelligent measurements should be suited for aumulating an unlimited quantity of 3D observations along with their assoiated sensor pose. It should also permit loal assessment of the onsisteny of these aumulated 3D observations with respet to the aquisition model before produing 3D measurements. These requirements impose a data struture for managing all the 3D information. Sine the 3D observations are aptured on the surfae of an objet, it is expeted that the data struture will enode a representation of the observed surfae. While expliit surfaes suh as triangulated meshes are the most ommonly used surfae representations for 3D modeling and omputer graphis, they are ill-adapted for effiiently managing the aumulation of 3D observations in a ontinuous proess. For these reasons, volumetri representations of impliit surfaes have been developed in the last 15 years 2,3,4. As opposed to a surfae mesh, the impliit surfae representation defines a value for all points in a 3D volume, thus not only on the surfae. For instane, this value an be the signed distane between a point in 3D spae and its losest point on the surfae. The distane is signed to disriminate between points that are inside or outside the objet surfae. This type of surfae representation is ommonly referred to as signed distane field (SDF) whih is naturally enoded in a 3D volume omposed of voxels. Figure 3 depits a partial ross-setion of an SDF. Eah voxel is olored with a grey level that indiates its signed distane to the surfae. In the figure, vetors aiming towards the surfae have been superimposed. From an SDF, an expliit representation suh as a mesh an be reovered using methods similar to the marhing ubes 5. Atually, the surfae orresponds to the zero set of the SDF, whih is where the SDF takes 0 as value. In a volumetri representation, the 3D spae enlosing an objet is divided into a regular lattie of volumetri elements, namely voxels. Eah of these voxels may aumulate information about the loal distribution of the aptured 3D observations in its neighborhood. This loal distribution enodes the parameters of a loal surfae model, whih may be more or less sophistiated. One example is the aumulation of the first two moments of the statistial distribution of the 3D points in the form of a enter of gravity and a ovariane matrix over x, y, and z omponents. Figure 3 Cross-setion of a signed distane field (SDF) on whih are superimposed arrow vetors aiming at the surfae (voxels with distane values equal to 0). From this matrix, a loal tangent plane an be extrated and validated with respet to the aquisition model. Indeed, loally on a surfae, the two larger eigenvalues of the ovariane matrix will be signifiantly larger than the third value, whih haraterizes the noise level omponent perpendiular to the tangent plane. Otherwise, unexpeted (meaning unmodelled) phenomena ourred. This way, the onsisteny of the 3D observations an be ompared with the

7 expeted noise level from the sensor 3D observations and thus be validated. For instane, in Figure 1, the system an invalidate data lose to the intersetion of the two planes where interrefletions affet the distribution of observations. In the same way, the number of observed points and the distribution of sensor positions where the 3D points have been olleted are aumulated at eah voxel for further validation. From suh a volumetri representation, there is no limit on the quantity of observations that an be aptured and proessed. The online appliation of this paradigm was unoneivable only 10 years ago due to the required omputer resoures inluding memory and omputation. Today, it an be applied on ommodity hardware. In order to redue omputational osts, the field is omputed only in the neighborhood of the surfae, i.e. at points that are loser to the surfae than a predetermined onstant distane T > 0 where T will depend on the sensor s noise harateristi. This neighborhood is said to be enlosed by a volumetri envelope. Being defined as the zero-rossing of the norm (whih ours between neighboring voxels), the surfae is adequately represented as long as T is larger than the voxel's diagonal (the largest distane between two neighboring voxels). One representation provides an improvement over SDF. We atually build and enode the gradient diretion of the SDF along with its norm. In this way, eah voxel enodes a vetor towards the losest point on the surfae. This augmented representation is a vetor field as opposed to a salar distane field that would enode only the signed distane. It should also be noted that, unlike salar distane fields, the vetor field at eah point defines exatly a point on the surfae. The surfae itself is represented as the zero set of the vetor field norm. Figure 4 illustrates the vetor field in the neighborhood of a surfae setion. More formally, let a surfae S be defined as a differential map S: U R R. The vetor field f: V R R representing the surfae S is defined as f p p q p ( ) argq min S (1) and V p R 3 p p f ( ) n ( ), arg min q S p q (2), where n() denotes the unit normal on the surfae at point. It is now illustrated how the vetor field an be onstruted from observations in the form of unorganized sets of 3D points. At eah voxel, a ovariane matrix is built from all points that are within T, T > 0 from the voxel. As before, an envelope is defined for eah point, in this ase a sphere.

8 Figure 4 Example of the vetor field. a) The vetor field is omputed on a regular grid of points (voxel enters) so that the vetor at eah voxel enter points towards the losest surfae point. The field is defined only in the viinity of the surfae, whih is in the volumetri envelope (region bounded by two iso-distant surfaes, oloured in green). b) A ross-setion of the vetor field. The ovariane matrix is obtained as follows: C 1 N N i1 ( i )( i ) T 1 N N i1 i T i T (3) where N 1 N i 1 i (4), is the entroid of losest points i. Both and matrix C an be omputed inrementally. The normal is obtained as the eigenvetor assoiated with the smallest eigenvalue. The distribution of the sensor orientation is also easily umulated in the voxels. Finally, those voxels that aumulated a signifiant number of onsistent observations in their neighborhood ontribute to produing a set of output 3D point measurements. The onsisteny test desribed beforehand is based on the eigenvalues of the ovariane matrix. It an be further validated using the distribution of sensor diretions. The output measurements are obtained after applying a modified version of the marhing ubes algorithm that is atually adapted to vetor fields. 6. Appliation Based on this data and surfae representation, aquisition model and alibration, a first generation of intelligent measuring systems with 3D optial sensors were implemented. These intelligent measuring systems introdue the new paradigm presented in this artile to the market plae for onrete appliations. One major and immediate result of this new intelligent measuring system that an be easily demonstrated is that inreasing the number of observations (multiple aquisition) inreases the auray of the measurement instead of reating noise in the point loud.

9 The first appliation that is shown in Figure 5 is a volumetri performane test using the ASME B standard 6 as a referene for the proedure. The volumetri performane test onsists of multiple measurements of a alibrated length artefat in various orientations (vertially, horizontally and at 45-degree angles) and various loations (20 loations) inside the mahine working volume. The experiment ompares touh probe (HandyPROBE) with optial sanning (MetraSCAN) implementing the intelligent measurement proess. The two measurement devies are positioned in spae (6 Degrees of Freedom) using the same positioning devie, the dual amera sensor C-Trak 780. The working volume is also the same; in this ase it is 7.8 ubi meters. The results show that the performane of both systems is in the same range. This is not surprising sine there is no fundamental reason to produe less aurate measurement with a 3D optial sensor than with a touh probe sensor. Figure 5 Comparison between probing (HandyPROBE tehnology 7 ) and using optial 3D measurements (MetraSCAN tehnology 8 ) exploiting intelligent measurement proess. The two systems use the same positioning system in spae. Tests follow the proedure ditated by the ASME B standard. When applied to the 3D optial metrology, this new paradigm of intelligent measurement omes with great benefits, partiularly when it omes to auray. That opens up new measurement possibilities for a wide range of appliations in various industries: manufaturing, medial, automotive, energy and aerospae. Among others, they are used for inspetion on projets with high-standard requirements. Figure 6 illustrates inspetion examples where these 3D optial sensors have been used as the measurement tool for inspeting an inlet ase of an airraft engine (left) or a landing gear (right).

10 Figure 6 Examples of typial appliations for the intelligent measurements systems. a) Example of the quality of the point loud generated. b) Example of a typial appliation in the aerospae industry: inspetion of a landing gear omponent. 7. Conlusion and prospetive Optial sensors pave the way to intelligent measurement proesses. Indeed, the large amount of 3D observations that an be quikly aptured on the surfae of an objet makes it possible to provide measurements of the same quality as probing. Moreover, the aquisition proess is more effiient and the measurements may spread over the whole surfae instead of only gathering points on a few targeted features. It thus enlarges the possibilities of metrology. For any 3D measurement system, the output 3D points arise from a hain of alulations. A 3D point is not a raw measurement in the physial sense, and applying an intelligent measurement proess is no exeption. Initially, 3D points are 3D observations from whih 3D measurements are obtained after validation with an aquisition model of the sensor followed by the omputation of the best loal estimates for eliminating noise. To implement this paradigm, we have proposed a volumetri struture for aumulating the 3D observations and for produing the 3D measurements. Currently, it is possible to implement an intelligent measurement proess on ommodity hardware (e.g. laptop omputers). Interestingly, given a rigorous alibration proedure and aquisition model, it is possible to obtain high quality measurements that would normally require ultra-expensive hardware. This is a natural onsequene of intelligent measurement sine it is possible to exploit the information from multiple data olleted in various onditions of measurement. In that sense, the proess is more flexible. What an be expeted in the future? First, this paradigm is only in its infany. We have been progressively integrating this paradigm in our systems for the last six years. Currently, it is already possible to produe 3D measurements that an be exploited diretly in the existing inspetion software that are available ommerially. In the near future, aquisition models will beome more and more sophistiated and the level of intelligene integrated in the sensing systems will progressively inrease as well. The aquisition systems will progress in their flexibility while remaining affordable. In the mid-term, we an also expet that inspetion software will be losely integrated with the sensor. These more intelligent sensors ould diretly aomplish inspetion tasks.

11 Referenes 1. EXAsan tehnology, site onsulted on May 13, Hilton, Adrian and Illingworth, John, Geometri fusion for a hand-held 3D sensor. Mahine vision and appliations, 12:44 51, Curless, Brian and Levoy, Mar, A volumetri method for building omplex models from range images, In SIGGRAPH 96 Conferene Proeedings, pages , August Masuda, Takeshi, Registration and Integration of Multiple Range Images by Mathing Signed Distane Fields for Objet Shape Modeling, Computer Vision and Image Understanding, 87:51 65, January Lorensen, William and Cline, Harvey. Marhing ubes: A high resolution 3D surfae onstrution algorithm. SIGGRAPH 87 Conferene Proeedings, 21(4): , The Amerian Soiety of Mehanial Engineers, Amerian National Standard ASME B , Method for Performane Evaluation of Artiulated Arm Coordinate Measuring Mahine, HandyPROBE tehnology, site onsulted on May 13, MetraSCAN tehnology, site onsulted on May 13, 2011.