MeshLab and Arc3D: Photo-Reconstruction and Processing of 3D meshes P. Cignoni, M Corsini, M. Dellepiane, G. Ranzuglia, (Visual Computing Lab, ISTI - CNR, Italy) M. Vergauven, L. Van Gool (K.U.Leuven ESAT-PSI - ETH Zürich D-ITET-BIWI)
Intro Arc3d + MeshLab a complete free software pipeline for the 3D digital acquisition based on standard photographic equipment Arc3D A free web based 3D reconstruction service, you upload photos and you get sequences of aligned depth maps MeshLab An open source mesh processing system for cleaning, aligning and merging meshes and range maps.
Arc3D: Architecture 1. Record a sequence of images of a scene or object 2. Upload the images to the ARC server 3. The server computes the 3D reconstruction 4. Download the results from the ARC website 5. Process and Visualize the results with MeshLab
Arc3D: How it works QuickTime and a TIFF (Uncompressed) decompressor are needed to see this picture. The entire process is based on finding matches between images
Arc3D: the output For each submitted image a depth image is reconstructed an image with the distance from the camera for each pixel a quality estimation for each pixel All these depth images must be cleaned and filtered integrated into a full model. QuickTime and a TIFF (Uncompressed) decompressor are needed to see this picture. QuickTime and a TIFF (Uncompressed) decompressor are needed to see this picture.
A typical Epoch - CI Tool MeshLab - an epoch result Developed at ISTI - CNR Just an example of the kind of tools that we are targeting for building the pieces a common infrastructure tool chain Domain Digital representations of 3D shapes Objectives Linking together input and output of different CH apps Adapting data from one CH app to another (mesh processing!)
MeshLab Generic mesh processing tool Open source GPL avail for win, mac and linux (scheduled for inclusion inside debian-science) The system relies on a large GPL library for mesh processing Aimed to the (almost) automatic processing of large unstructured 3d models acquired 3D models are different from human built models No structure, no particular need of a scene graph Large datasets : millions of primitives
MeshLab Able to manage many different kind of mesh formats Input format PLY, STL, OFF, OBJ, 3DS, COLLADA, PTX, X3D, VRML Output format PLY, STL, OFF, OBJ, 3DS, COLLADA, X3D, VRML, DXF
MeshLab dissemination License GPL Code size: 35k + 65k LOC (>20 py estimated ) V1.0.0 (march 2007) Downloaded >30k times, Web site served more than 400k pages more then 2000 faithful users (users that opened more than 50 meshes), hundreds of university and research centers using it all over the world V1.1.0 (feb 2008) Downloaded 5k times in three weeks
MeshLab typical tasks Filtering Removal of outliers according to various heuristics Smoothing out noise Various fairing algorithms Editing Selection of parts and removal of non interesting portions of the acquired data
Remeshing Subdivision Surface (loop and butterfly) Reducing complexity through simplification Both fast approximate and slower high quality algorithms 500k 50k 5k
Checking Visual inspection See through filters Quality evaluation Curvature dependent, Automatic marking of mesh inconsistencies topological: non manifoldness Geometric: self intersection
Repairing Hole Filling Automatically interpolate small missing mesh portions filling small gaps
MeshLab: Aligning and Merging 3D Scanning Pipeline Tools ICP based alignment Including global alignment for error redistribution Surface reconstruction algorithms for range map merging
MeshLab Features Aligning functionalities needed to bring together models reconstructed from different sequences:
MeshLab Features Merging tools can be used for blending all the reconstructed range maps into a single (eventually watertight) mesh Various surface reconstruction algorithms Poisson Surface Reconstruction Watertight mesh, sometimes can invent too much Volumetric distance field Preserve and blend color information Do not fill large holes
MeshLab Recon. Sample Arc du triomphe >100 photos ~1M triagles reconstructed model No texture here to better show the geom quality. Reconstructed with the distance field approach
MeshLab Recon. Sample One of the two lions in the Loggia della Signoria in Florence >30 photos 200k triangles reconstructed model Hi quality color reconstructed Reconstructed with the distance field approach
MeshLab Recon. Sample A Lion statue from the Portalada in Ripoll, a large romanic portal, near Barcelona >60 photos 600k triangles reconstructed model No color but watertight Reconstructed with the Poisson surface reconstruction alg.
PhotoCloud Prototype A new way to use the results of Arc3D Do not aim to a clean 3D mesh model, but retain and exploit all the original photos information
PhotoCloud Prototype Basic ideas Mix a rough 3D with (many) High Res photos Allow free navigation in 3D space and in the space of the images Allow to to take measure on photos Work in remote, everything (3D and photos) is stored on a standard web server You can show your reconstruction results without requiring that people download giga sized datasets
Conclusions Download and play by yourself! Both tools are ready to use and tested. http://www.arc3d.be http://meshlab.sourceforge.net Thanks for your attention!