A Method to Segment a 3D Surface Point Cloud for Selective Sensing in Robotic Exploration

Transcription

1 A Method to Segment a 3D Surface Point Cloud for Selectie Sensing in Robotic Exloration Philli Curtis, Pierre Payeur School of Information Technology and Engineering Uniersity of Ottawa Ottawa, O, Canada [curtis, ayeur]@site.uottawa.ca Abstract Autonomous robotic exloration in a 3D enironment requires the acquisition of 3D data to create a consistent internal model of the enironment from which objects can be recognized for the robot to interact with. As the acquisition of 3D data with stereo ision or a laser range finder can be a relatiely long rocess, selectie sensing is desired to otimize the amount of data collected to accurately reresent the enironment in a minimal amount of time. In order to erform selectie sensing, a coarse acquisition of the enironment first needs to tae lace. Regions of interest, such as edges and other boundaries, can then be identified so that an acquisition with higher satial resolution can occur oer bounded regions. For that urose a segmentation method of the coarse data is roosed so that regions can be efficiently distinguished from each other. The method taes a raw 3D surface rofile oint cloud of arying oint densities, organizes it into a mesh, and then segments the surfaces resent in this oint cloud, roducing a segmented mesh, as well as an octree of labeled oxels corresonding to the segmentation. This mesh and octree may then be used for sensory selection to drie a robot exloration tas. The method is demonstrated on actual datasets collected in a laboratory enironment. Keywords: segmentation; selectie sensing; 3D modeling; octrees; robotic exloration. I. ITRODUCTIO Robotic naigation and object maniulation tass require that the arious sensor acquisitions be fitted into a model of the enironment. In ractice, a large fraction of data acquired by sensors such as laser range finders or stereo-based systems are heaily inter-correlated while requiring large amounts of time to be collected. In order to otimize the acquisition rocess, minimizing the redundancy of information from a single oint of iew is desired. This can be accomlished by erforming a coarse acquisition, and then selecting regions of interest within this acquisition to focus on for refinement. To erform this selectie sensing, regions of similar stochastic roerties and continuity must be searated from each other in order to determine what areas need to be enhanced. The current solutions for segmenting 3D range data mainly concentrate on recise modeling and regularly samled datasets as oosed to seed that is needed for autonomous robotic tass. The roosed method focuses on seed of segmentation as oosed to recision as the urose is for robotic naigation and tas lanning. Moreoer the method does not rely on regularly samled data, as many inexensie commercially aailable sensors, such as stereoscoic sensors, tend to roduce irregularly samled sarse 3D data acquisitions. A common rocedure for segmenting in 3D is to organize the oints, usually by a Delaunay triangulation algorithm, and then using those triangles to estimate surface normals at the acquisition oints, if not already roided by the sensor [1-4]. Woo et al. [1] use oint normal estimations to guide the subdiision rocess of the oints in an octree: as the oint normals in a cell start arying by more than a certain threshold, the cell is subdiided, leading to smaller cells where there is large ariation, and large cells where there is small ariation. Cells that are small are considered edge cells, and the remainder are merged using a satial adjacency relationshi, and are labeled as belonging to the same segment. This technique relies on a reasonable uniformity on samling density. Schall et al. [] cluster similar data for the urose of filtering, and use the oint normals as well as the satial locations to determine if oints in a local neighbourhood belong to the same lane. The lielihood function that is used taes into account distance from the oint in question when determining what lane it belongs to. When all local lielihoods are calculated, all oints are moed to locations of higher robability using an adatie gradient function. As the robabilities are maximized, the oints are merged into a reresentatie oint, which reresents the merged cluster of oints and its corresonding surface. The highly iteratie nature of this technique does not lead itself well to automated tass, but is interesting in that it addresses the non-uniform oint density roblem. Hubeli and Gross [3] roose a multi-resolution technique which uses seeral difference algorithms to achiee flexibility in segmentation of a mesh. The difference algorithms used are normal-based or olynomial fitting-based in order to calculate weights for indiidual edges. The imortance of an edge is determined by using its weight as well as its neighbours weight, and a seletonizing algorithm extracts features based on the imortance of a edge. This technique is flexible in its aroach, but the classification asect of the method assumes reasonably uniform oint density along the surface of the object.

2 Lee et al. [4] describe a technique for edge detection on 3D datasets. The first ste is to erform a triangulation and then calculate surface normals. An edge is determined by finding areas where the surface normals ary beyond a threshold. eighbouring surfaces with similar normals are groued together in regions. To reent missing gradual edges (e.g. smooth cures), seed faces for the region growing are used. They are also used for the region comarison to detect ariation in the normals for finding edges. This algorithm aears to deal well with non-uniform surface oint densities, as face size is a arameter used in edge detection. It shows good results when ¾ of the oints are remoed from the source data. Mederos et al. [5] introduce a technique which creates a surface mesh based uon regions of similarity. The method clusters oints in local regions together, and decides whether to subdiide the region into smaller regions based uon the ratio of eigenalues of the oint coariance matrix and the number of oints in the local region. After local clustering is determined, a moing least-squares technique hels find a reresentatie oint for the region. A triangulation is determined from those reresentatie oints, and is continuously refined until certain criteria are met. The clustering technique is interesting as it effectiely segments local regions of data with similar stochastic roerties without haing to erform a triangulation first, and minimizes the amount of data for mesh creation, but unfortunately many iterations seem to be required to roduce reasonable results. Payeur and Chen [6] deelo a segmentation method which creates lanar surface atches for the urose of erforming registration between differing 3D iews. Since a line-based laser range finder is used, a straight line fitting is alied to each 3D line rofile acquired, effectiely segmenting a line rofile into seeral straight line segments. ormals to the line segments, as well as lengths and centroids are calculated, and then used to combine similar neighbouring line segments of successie acquisitions together to form lanar atches. This method is relatiely quic, but relies on densely samled structured data collected oer relatiely flat surfaces, which counteracts the goal of selectie sensing. II. OVERVIEW OF PROPOSED TECHIQUE The roosed technique deals with 3D data acquired from an in-house structured light stereoscoic system [7] which oerates under many different conditions, and can acquire data oer seeral tyes of surfaces. Delaunay triangulation is initially alied on the raw 3D oint cloud to roduce a surface rofile. From this surface rofile, the normals of the oints are calculated from the weighted aerage of the normals of the surrounding triangles, with the weights being the inerse of the area of each triangle inoled. By using the inerse of the triangle area, more local and densely samled regions, which are reresented by smaller triangles, are gien larger weights than sarse regions with less locality, which are reresented by larger triangles. The oints are then inserted into a multiresolution octree, which is subdiided based uon the standard deiation of the oint normals contained in each oxel. Eery oxel in the octree is considered as a seed segment. The latter are merged based uon neighbours determined from the connectiity of the original Delaunay triangulation, and deending on how much deiation there is between the normals of the segments to be merged. Fig. 1 resents the flow of rocessing to segment a 3D dataset. Figure 1. Flow of data for the roosed method. III. IMPLEMETATIO A transformation of the data from Cartesian coordinates to a sace more suitable to the sensor s method of acquisition is first erformed to ensure that rojectie occlusions in the triangulation do not influence the maing. A sherical sace defined by 3 arameters is used. The arameters resectiely reresent the angular distance from the Z-Axis to the X-Axis (θ), the angular distance from the Z-Axis to the Y-Axis (φ), and the Euclidean distance from the origin to the oint in question (r), as shown in Fig. a. The transformations to and from this sace are shown in eq. (1). Fig. b shows an examle of an occlusion (shown as the thinner segment on object 1) that is created by object when obsering from the X-Axis in Cartesian coordinates. This occlusion does not occur in the sherical sace where the entire length of object 1 can be obsered from the origin, which corresonds to the range sensor s oint of iew. Figure. a) Grahical reresentation of Cartesian and sherical sace maings, and b) grahical reresentation of occlusion in Cartesian sace that does not occur in the sherical sace. ( x z) θ at an, z r ( θ ) + t ( ϕ) 1+ tan an ϕ at an( y, z) x z tan( θ ) x + y z y z tan( ϕ) r + (1)

3 A. Triangulation While any triangulation method would be accetable for forming the surface rofile mesh, the common Delaunay triangulation was chosen since reasonable erformance can be obtained, and it guarantees that there will not be too many long thin triangles [8], and thereby maintaining a strong locality relationshi between oints. The incremental Delaunay algorithm is used with a triangle data structure that ees trac of neighbouring triangles. B. Point ormal Calculation Since the oint clouds are not guaranteed to hae a regular samling density, the method taes this fact into consideration. The normal of each triangle, t, is calculated as well as the area of the triangle, A t. For eery 3D oint,, the oint normal,, is calculated by a weighted aerage of all triangles, t, from which the oint is a ertex, as shown in eq. (). Since regions of higher oint densities are reresented by smaller triangles, and lower density areas are reresented by larger triangles, the inerse of triangle area is used as the weight to ensure that neighbour oints that are more closely related to the oint for which the normal is currently calculated receie a higher weight than oints that are located further away and are reresented by larger triangles. The final result of the weighted aerage is normalized. t A 1 t C. Octree Generation The oints are inserted into a multi-resolution octree which is subdiided based uon the standard deiation of the oxel s,, normal. The first ste in this rocess is to determine the area which a oint is associated with, A P. It is calculated in eq. (3) as the sum of the areas of the triangles to which the oint is a ertex. The inerse of A is used as the weight alied to the corresonding oint normals for calculating the oxel normal,, as shown in eq. (4). Again this ensures that sarsely samled areas do not dominate oer denser samled areas in the calculation of the oxel normal. A t t t () A (3) A (4) The standard deiation of the oint normals, σ, contained in a oxel is then calculated as shown in eq. (5). A threshold is alied to the standard deiation along each axis, σ, θ for the θ-axis and σ, ϕ for the φ-axis. If either one exceeds this threshold, the oxel is slit into eight different oxels. A threshold of 7.6 deiation in both directions was determined to wor well through trial and error. A (, θ, θ ) ( ) σ, θ σ (5) σ, ϕ A, ϕ, ϕ A D. Merging Octree Voxels The octree reresentation of sarse and irregularly samled datasets will ineitably contain gas between oxels, and the oxels will hae neighbours that are statistically similar. In order to reent the fragmentation of the enironment into many oxel sized segments, and to maximise the area of statistically similar regions, a merging of oxels into larger segments needs to occur. The initial ste is to label the oints contained in each oxel of the octree as belonging to their own segment corresonding to that oxel. These initial segments are used as seed segments, which are then merged together to form larger segments of stochastically similar contiguous regions. This merging relies on the information of adjacency contained in each triangle from the Delaunay triangulation and the information already calculated when determining whether or not to subdiide the octree, which corresonds to surface orientation. The reresentatie area, A, of the oints contained by the oxel is calculated, as shown in eq. (6), by summation of the areas associated with oints. The normal for each segment is then calculated using a weighted mean, eq. (7). The inerse of the area is used as the weighting factor to ensure that oxels that contain a denser contribution of oints hae more weight than a less dense oxel. A A A (6) S S A S S S Each oxel contains a collection of oints, j, and these oints belong to triangles, t l,. Triangles can be considered as constructs that connect together different oints that may or may not be in the same segment. Triangles that hae all three of their oints belonging to the same segment, S, are considered as interior triangles belonging to that segment, T I,.. Triangles that hae less than three oints belonging to a articular segment are considered as edge triangles as they lie (7)

4 on a boundary between two segments. Furthermore, two classifications of edge triangles are made. If a triangle s normal is within a certain range of the segment s normal then the triangle is considered a alid one to consider for searching along its ertices for segments to merge with the current segment (T VE, ). If it is not, then the triangle robably lies on a boundary between different regions, and is not a alid triangle to use to search for ossible segment merging (T IE, ), as is shown in eq. (8). TI, iff j tl,, j S tl, TVE, iff tl, TI,, l, S 5 TIE, otherwise Merging segments together relies on searching the ertices of alid edge triangles for other segments, and then determining if these segments are statistically similar to the current segment. If the difference between the segment normals is less than a threshold, as shown in eq. (9), then the segments are merged together, as shown in eq. (10). A threshold of 5 was found to wor well for our uroses. Fig. 3 illustrates this classification: triangle d is entirely in segment A, so it is considered an internal triangle. Triangle e is an edge triangle, as it has a ertex in both segments A and B. Triangle f is also an edge triangle, since it has a ertex in segments A and C, but since it also crosses a transition region (an edge or other discontinuity) and hence its normal is significantly different than either segments, it is considered an inalid triangle to use to search for regions to merge. (8) S Sj 5 (9) otimal erformance from the Delaunay algorithm. The datasets used include an acquisition of a cube (Fig. 4), a moc u chair (Fig. 5), and a moc u of a car door with a ding in it (Fig. 6). Figure 4. Image of cube used in cube dataset. Figure 5. Image of chair used in chair dataset. S S S j (10) Figure 3. Merging rocess between seed segments. IV. EXPERIMETAL RESULTS Our reliminary datasets include data acquired from an inhouse 3D stereo acquisition system described in [7]. When reading the oint data, all information other than ositional data was remoed and the oints order randomized to ensure Figure 6. Image of car door used in door dataset. The segments shown in Fig. 7-9 are coloured according segment labels maed to unique hues of HSV sace to better show searation between segments. White triangles reresent edge triangles, and may aear as shades of grey in the images due to lighting and shading effects. The segmented cube dataset (Fig. 7) shows fie significant segments, one for each isible face coloured with in, urle

5 and blue, and two segments at the bottom in orange and green which constitute bacground surface (refer to Fig. 4). Each of these are a searate lanar surface, and are labeled as different segments as exected. The edges of the cube contain ery small segments as well as edge triangles. The chair (Fig. 8) shows two large segments, namely the front of the chair in light blue, the bac rest ortion in dar blue, and smaller segments on the inner right armrest in orange and red. These constitute different lanar surfaces of the chair, and are correctly labeled as different segments. The door with the ding (Fig. 9) shows seeral distinct segments. One for the main ortion of the door in light green, one for the window frame segment in light blue, one for the handle in dar blue, a small one for the ding in urle, seeral for where the door cures towards the window segment, and seeral segments for the bacground wall, with the redominant segments in in and orange. There are also seeral smaller segments within the main region of the door due to noise resent in the data. The multi-resolution octree isualizations (Fig. 10-1) show the same coloured segmentation methodology as for the meshes. As can be obsered, a region where there is large surface ariation is reresented by smaller oxels, while regions with low ariation are reresented by larger oxels similar to [1]. This effect is most isible in the chair (Fig. 11) where the bac rest has large oxels in the middle and smaller oxels near the edges, and the cube (Fig. 10) where the faces hae larger oxels than the edges. Table 1 shows the number of oints for each dataset, as well as the execution times for each ste. These tests were run on a Pentium IV 3.4 GHz Windows XP based comuter with.5 GB of memory. It should be noted that a full segmentation of the dataset of the door containing slightly more than oints needs about 5.0 seconds, which corresonds to a maximum segmentation time of the order of 0.1 ms er 3D oint in a dataset. Gien that measurements sarsely collected in a tyical robotic exloration tas would contain a few thousands of oints, the roosed segmentation method is suitable to erform real-time selectie sensing to imroe autonomous robot naigation and ath lanning. Figure 7. Mesh of segmented cube dataset. Figure 8. Mesh of segmented chair dataset. TABLE I. SEGMETATIO EXECUTIO TIMES Data Set # Points Delaunay (s) Segmentation Results ormal Octree Calc. Gen. (s) (s) Merge Voxels (s) Total (s) Door Chair Cube Figure 9. Mesh of segmented door dataset.

6 Figure 10. Octree isualisation of segmented cube dataset. COCLUSIO This aer introduced a quic and efficient segmentation technique of 3D oint clouds for the use in robotic exloration tass. Regions of the surface of objects that are searated by a discontinuity or an edge are automatically labeled as different segments. A surface mesh and an octree are both roduced in the segmentation rocess, and can be refined and reused to aid in the naigation and tas lanning rocesses. The method can handle data with non-uniform samling density, as the technique taes into account local densities of samling oints. As such a robotic sensory system can erform a coarse acquisition oer the enironment and then, based on the tas, erform a finer acquisition only oer regions of interest that corresond to secific segments or transition regions determined by the roosed segmentation algorithm. Future wor will integrate an automatic mechanism to select regions of interest for finer scanning around the candidate segments and refine the aroach for the segmentation to rely on arameters that go beyond relatie surface orientation. ACKOWLEDGEMETS The authors wish to acnowledge the suort from Communications and Information Technology Ontario Centre of Excellence, and from the atural Sciences and Engineering Research Council of Canada to this research wor. Figure 11. Octree isualisation of segmented chair dataset. REFERECES [1] H. Woo, E. Kang, S. Wang, K.H. Lee, A new segmentation method for oint cloud data, International Journal of Machine Tools & Manufacture, ol. 4, , 00. [] O. Schall, A. Belyae, H.-P. Seidel, Robust filtering of noisy scattered oint data, Proc. of the IEEE Eurograhics Symosium on Point-Based Grahics, , Long Island, ew Yor, USA, 0-1 June, 005. [3] A. Hubeli, M. Gross, Multiresolution feature extraction from unstructured meshes, Proc. of the IEEE Conference on Visualization, , San Diego, California, USA, 001. [4] Y. Lee, S. Par, Y. Jun, W.C. Choi, A robust aroach to edge detection of scanned oint data, International Journal of Adanced Manufacturing Technology, Sringer London, ol. 3, no. 3-4, , 004. [5] B. Mederos, L. Velho, L. H. de Figueiredo, Moing least squares multiresolution surface aroximation, Proc. of the IEEE Brazilian Symosium on Comuter Grahics and Image,. 19-6, São Carlos, Brazil, 1-15 Oct., 003. [6] P. Payeur, C. Chen, "Registration of range measurements with comact surface reresentation", IEEE Transactions on Instrumentation and Measurement, ol. 5, no. 5, , Oct [7] A. Boyer, P. Curtis, P. Payeur, 3D modeling from multile iews with integrated registration and data fusion:, Proc. of the Canadian Conference on Comuter and Robot Vision,. 5-59, Kelowna, BC, 5-7 May 009. [8] Ø. Hjelle, M. Dæhlen, Delaunay triangulations and Voronoi diagrams, in Triangulations and Alications, Sringer Berlin Heidelberg, , 006. Figure 1. Octree isualisation of segmented door dataset.