ARC 3D Webservice How to transform your images into 3D models Maarten Vergauwen info@arc3d.be
Overview What is it? How does it work? How do you use it? How to record images? Problems, tips and tricks
Overview What is it? How does it work? How do you use it? How to record images? Problems, tips & tricks
What does the 3D Webservice do? The ARC 3D Webservice allows you to transform your images into 3D models automatically. You only need a digital camera, a PC and access to the Internet. images Maurizio Forte, CNR Rome
What are the different steps? Step 1: Record a sequence of images of a scene or object Step 2: Upload the images to the ARC server Step 3: The server computes the 3D reconstruction Step 4: Download the results from the ARC website Step 5: Visualize the results
Overview What is it? How does it work? How do you use it? How to record images? Problems, tips & tricks
How does it work? The 3D Webservice only uses images as input. Computer vision techniques are used to compute the results. Automatic matching of images Automatic computation of camera calibration Automatic dense matching for depth map generation
Image Matching The entire process is based on finding matches between images. Record your pictures not too far apart, so the computer can match them easily!
Camera Calibration No prior knowledge on camera calibration is available, so all information must be recovered from the images It is therefore important that enough information is present in the images! General motion of the camera General structure in the scene Enough overlap (only points that are visible in at least 3 images are useful) See tips and tricks for more info
Dense Matching After recovery of the camera calibration, dense depth maps are computed These contain the depth of every pixel and a quality measure (how confident we are of each particular pixel)
Overview What is it? How does it work? How do you use it? How to record images? Problems, tips & tricks
How do you use it? Obtain an account from the website Download and install the programs Use the upload program Load your images Upload them to the server Have a coffee Download the results from the server Visualize the results
Obtain an account Surf to www.arc3d.be and navigate to Request Login
Obtain an account (2) Fill in name, email address and the captcha form
Obtain an account (3) Your login information will be sent to your email address
Download and install Install as Administrator in the default place or as normal user, specifying a writeable folder
Upload Client
Upload Client (2) * If ARC decides to use your images in a publication, it will add this reference. See conditions on the ARC homepage. Images selected by user Subsample to smaller size for faster upload and computation Authentication with server @ ARC Organization into scenes and sequences Select or create a new scene Supply a sequence label Add academic reference if desired*
Have a coffee
Download the results The ARC service reports its result to you by email If the reconstruction has succeeded, a URL is specified from where you can download a zip file containing the results. If the reconstruction failed, some more information is given, including tips on how to improve the result.
Visualize the results Two tools are available for inspecting the results The ARC team recommends MeshLab, a tool developed by CNR-ISTI in Pisa, Italy A simple model viewer comes with the ARC installer
Visualize with MeshLab Download MeshLab from meshlab.sf.net and read the documentation
Visualize with Model Viewer Download results from server Unzip file with images, cameras, depth and count maps Select one image on the left. Reconstruct 3D model for this specific depth map Parameters to tune: Minimum quality (count) Subsample for smaller models Minimum angle w.r.t. viewing direction. This ensures that shearing angles are removed Median filter size: larger filter for smoother (less noisy) models
Visualize with Model Viewer (2) Select the model tab and view the result in a 3D widget Texture comes from the original image Export to different formats (X3D, PLY, OBJ, VRML) is possible
Some Examples
Temple of the Masks, Edzna Maurizio Forte, CNR Rome
Temple of the Masks, Edzna Movie available at http://www.arc3d.be/edzna.mpg
Arc de Triomphe Movie available at http://www.arc3d.be/arc.mpg
Overview What is it? How does it work? How do you use it? How to record images? Problems, tips & tricks
How To Record images? The ARC 3D Webservice has been developed with robustness in mind. However to minimize frustration it is best to take heed of some simple guidelines! We will explain some dos and don ts
A Good Sequence Shoot a picture of the same location for every step made in the shooting sequence. This results in multiple pictures of the same scene, but viewed from slightly different sides. Walk with the camera in an arc around the scene, while keeping the scene in frame at all times.
Bad Sequences not pan from the same location, as if you were recording a panorama. It is not possible to determine enough 3D information from such a sequence. Do
Bad Sequences (2) Also don t shoot multiple panorama s from different viewpoints.
Bad Sequences (3) Do not walk in a straight line towards or inside the scene you want to reconstruct.
Bad Sequences (4) It is better to shoot too many pictures than not enough. A minimum of five or six images are required for a good reconstruction. Less than four and the reconstruction will fail. The viewing angle between images should not be too small, i.e. adjacent images should not be too far apart
Bad Sequences (5) Do not use a turntable with a static camera and a scene that rotates on a perfect circle in front of the camera. Because of this very specific motion, there is not enough information in the images to allow camera calibration.
Bad Sequences (6) Do not take pictures of a planar (or almost planar) scene These scenes do not contain enough information for camera calibration
Bad Sequences (7) Do not zoom in or out during recording!
Recording Equipment What kind of camera should I use? More pixels = more 3D points = longer upload and processing time 5MP (2500x2000) is typically good enough Good lens = less distortion = better result Good lens = more light = better result
Overview What is it? How does it work? How do you use it? How to record images? Problems, tips & tricks
Problems, Tips and Tricks Two major steps in the pipeline Camera calibration Dense matching Both vulnerable to problems Most important rule to realize All information is retrieved from the images, so take good care when you shoot them! Better to spend 2 more minutes than have to go back and re-shoot!
Image Content Information is found by matching images Make it possible to do so automatically! The texture (color, intensity) of the scene object is critical! Enough texture must be available on the object Appearance of object must stay the same!
Not Enough Texture
No Constant Appearance
No Constant Appearance
No Constant Appearance
No Constant Appearance The first thing people try, is to upload their baby pictures This will not work! Human skin is notorious for changing appearance when viewed from a different angle
Occlusions and Self-occlusions Objects blocking the scene (e.g. trees) make the reconstruction of hidden parts impossible. Sometimes the object is self-occluding: parts sticking out hide parts behind it
Quality: Beware Blurry Pictures When the camera moves during recording, the image becomes blurred This causes problems for the reconstruction or degrades the result Hence avoid it!
Dynamic Scenes = No No The scene must be static: it should not change during the recording!
Lighting Conditions Perfect Overcast sky Constant illumination Worse Direct sunlight Bad Changing conditions
More Info? More information about the ARC webservice can be found here Read the MVA publication: M. Vergauwen and L. Van Gool, Web-based 3D Reconstruction Service, in Machine Vision Applications (MVA), 17, pp. 411-426, 2006. Send an email to info@arc3d.be