Automatic 3D Reconstruction and Modeling

Transcription

1 Automatic 3D Reconstruction and Modeling Title Slide Nice Picture of 3D Models Nice Picture of Segway Automatic 3D Reconstruction and Modeling 1

2 Automatic 3D Reconstruction and Modeling Spot the difference! 2

3 3D Model inspection 3

4 Goal and Motivation Growing demand for digitizing indoor and outdoor environments Applications 3D content generation for mapping service applications, e.g. google maps, bing maps video games based on real locations interactive museum exhibits Pre-visualization and virtual sets for the film industry Virtual Reality Systems Realistic simulators for robotic algorithms (e.g. ROS/gazebo) Goal: autonomous system for efficient creation of photo realistic 3D models of indoor and outdoor environments 4

5 Overview Data Acquisition Registration Surface Reconstruction Texture Reconstruction 5

6 3D Object Representations Raw Data Point Cloud Rang Image Polygon Soup Solids Voxels BSP Trees CSG Surfaces Meshes Parametric Implicit High Level Structures Scene Graphs Skeleton etc. 6

7 Data Acquisition (1) Robot hardware including range and imaging sensors Frontier based exploration for active environment exploration 2D mapping and localization (Grisetti et. al, 2005) 5MP Camera Stereo Cameras prev. 3D scans Scanning Lidar SLR Camera next scan trajectory Current location Pan-tilt unit Tilting Lidar frontier Exploration Lidar Base Lidar Bosch s 3D Mapping Robot Willow Garage s PR2 7

8 Data Acquisition (2) 8

9 Data Acquisition (3) 9

10 Pairwise Registration Iterative Closest Point (Besl & McKay 92) Optimization loop Finding point correspondences (distance, normal, color) Minimizing error metric Apply transform and iterate Fast convergence Easy to implement Errors accumulate for multiple scans 10

11 Global Registration Measurement links (scan matching) SLAM: estimating the robot s pose and a map at the same time. Full SLAM posterior: Map Prior Motion Model Measurement Model Finding the most probable solution: Motion links (wheel odometry) Maximum a-posteriori estimation! A vector of features representing the environment. The robot s position and orientation. A feature observation. The control vector applied a time t-1. 11

12 Global Registration Example 2D Floorplan 3D Map 12

13 Global Registration with Map Prior Full SLAM posterior: p ) ( m, x u, z) = η p( x0 ) p( m) p( xi xi 1, ui ) p( zi xi, mi i Map Prior Motion Model Sensor Model Use a local surface model to approximate global map prior: Intuition: observation of surfaces belong to smooth manifolds Result: Probabilistic Non-rigid Registration (Pitzer et al. ICRA 2010) 13

14 Global Registration with Map Prior Example Registration without map priors Registration with map priors 14

15 Surface Reconstruction 15

16 Volumetric Surface Reconstruction Surface reconstruction using a indicator function χ oriented points indicator function iso-surface Gradient of the smoothed indicator function is equal to the smoothed surface normal field (Kazhdan et al., 2006) Point Cloud Formulation as a Poisson problem: indicator gradient χ Bilateral filter as smoothing function. Surface Model 16

17 Appearance Reconstruction Reconstruction of material properties (e.g. color, specularity, temperature) Inference of material types (e.g. wood, plastic, vegetation) Enhancing visual impression by adding more realism Surface Model Textured Surface Model 17

18 Color Reconstruction (1) 1. Surface partitioning Surface segmentation into nearly planar regions Merging of similar regions using adjacency graphs 2. Unfolding Mapping of surface into texture domain Finding a conformal mapping which preserves the angular distortion Minimizing the complex conformal energy (Levy et al., 2002): 3. Color Reconstruction Unprojection of 3D points into camera images 18

19 Color Reconstruction (2) Global optimization of texture color to remove discontinuities. Multi-view blending algorithm: before blending averaging Poisson formulation with Dirichlet boundary conditions (Perez et al., 2003) our algorithm 19

20 20

21 Handling Large 3D Models (1) Office model (50m x 140m) 28M Vertices, 55M Faces 772 MByte Geometry 600 MByte Texture Generation of scene graphs by iteratively dividing 3D model Model simplification of scene graph nodes to limit number of elements per node View-dependent culling and loading 21

22 22

23 23

24 24

25 Failure cases: Reconstruction of fine structures (plants, furniture, etc.) Texture reconstruction failures due to registration errors Future work: Overcome limitation to static environments Online integration of new data vs. batch processing 25

26 Thank you for your attention! 26