3D Data Management From Point Cloud to City Model GeoSmart Africa 2016, Cape Town

3D Data Management From Point Cloud to City Model GeoSmart Africa 2016, Cape Town Albert Godfrind Spatial Solutions Architect ORACLE Corporation April 13, 2016 Copyright 2014 Oracle and/or its affiliates. All rights reserved. Screenshot courtesy of: IQsoft Spatial Day 2015

Managing 3D Data at National Level Large user community Image courtesy of: IQsoft, Austria Open Standards Screenshot courtesy of Rico Richter, HPI Image courtesy of: PDOK, NL Large data sets Spatial DB Screenshot courtesy of Rico Richter, HPI

Point Cloud Challenges Data Volumes increasing densities with technology Billions of points Storage Where do I put all this data? Archive? Security? Compress? Optimal format for analysis? LAS, LAZ, CSV, Proprietary Analysis Derivative product generation (TINs, Contours, DEMs, Co elate ith othe data e to, aste, et o ks, add esses

Why a Database? Seamless coverage Separation in individual files disappears Extract any subset irrespective of capture Integration with other data Addresses, utilities, land cover, land use and ownership, DEMs a d g ids, i age Information lifecycle management Migrate aging data to cost-efficient storage (SSD, disks, cheaper disks, off-li e a d also ala ilit, a ess o t ols, o ti uous ope atio, disaste e o e, 5

Manage All your Geospatial Data e1 f1 e2 n2 f2 n1 e4 Rasters Networks e3 Topology 3D Oracle Spatial and Graph Geocoding and Routing Locations, track and trace Cables and pipes Land Use

Point Clouds Workflow for Object Type LiDAR Files LiDAR Flat Files Files Load flat ASCII files into point tables then build the point cloud from the point tables Point loader Point tables Build from point tables Query and Clip Direct load from LAS files Flat files Flat filesfiles LiDAR Point Cloud tables Generate Derived Products LiDAR loader Convert to Geometries

Point Clouds Workflow for Relational Type LiDAR Files LiDAR LiDARFiles Files Direct load from LAS files Copy Query and Clip Point tables Copy Generate Derived Products Direct load from flat ASCII files Flat files Flat Flatfiles files Convert to Geometries

Dutch escience research project on Massive Point Clouds Project Consortium, led by Peter v. Oosterom, TU Delft 1. 2. 3. 4. Image courtesy of: PDOK, NL TU Delft: GIS technology TU Delft Library 3TU.Datacentrum TU Delft Shared Service Center ICT NLeSC (Netherlands escience Center) Oracle Corporation (NEDC) Rijkswaterstaat Fugro B.V.

File and Datasets Actual Height Model of the Netherlands (AHN2) Covering surface of the entire country 6-10 pts/m2 640 billion pts 60,185 LAZ files, 987 GB in total, 11.64 TB uncompressed (X, Y, Z) only Future plans AHN3 at even higher resolution Cyclorama-based photogrammetric datasets (50x AHN2, and with RGB)

Benchmark results (I) Performance and best practices Data loading Use e te al ta le e ha is i o ju tio ith eate ta le as sele t to stream LAZ files directly into database Loading all 640bn points from LAZ files in 4:39h on Exadata X4-2 (Full Rack) Compression o p ess fo ue high setti g delie e s est o p o ise et ee sto age reduction and query performance Requires 2.24TB of storage in database, compared to 0.97TB in LAZ files or 12TB (uncompressed) LAS 11

Benchmark results (II) Performance and best practices Data management Use partitioning for better performance and improved parallelism 20655 partitions with between 10,000,000 and 50,000,000 rows Also helps administration and information lifecycle management Scalable hardware and database Oracle 12c Spatial and Graph on Oracle Exadata Database Machine Flat table storage model No indexes, making use of Exadata internal indexing on storage cells 12

ÖBB-Infrastruktur AG, R&D Objectives Optimize railway planning, construction and maintenance Enable efficient routing of railway lines Solution [This te h olog ] is i dispe sa le to p o ess geospatial data with high efficiency at low ost, Dr. Michaela Haberler-Weber ÖBB-Infrastruktur AG, R & D Stores and processes more than 8 billion points of objects along railway tracks Enables LiDAR data to be viewed with existing infrastructure vector data Provides comprehensive metadata about railway tracks through CSW Delivers data through open WebServices Image courtesy of: IQsoft, Austria

Visualization 14

Creating a 3D model constructing 3D objects Simple Surfaces Face = 3D Polygon Composite Surface Extrusion Generating solids from 2D polygons Multiple connected faces Simple Solid Closed composite surface Composite Solid Multiple connected simple solids

Data models for City Modeling 3DCityDB (open source) is widely used LOD1 Building LOD2 Building LOD3 Building LOD4 Building Semantically structured model Structures at multiple levels of detail Textures and facades Orthophotos Versioning Source: Research Center Karlsruhe

City of Berlin 3D City Model Implemented by TU Berlin 550000 buildings, reconstructed from 2D cadastre and LIDAR data Textures extracted from oblique aerial photography Stored in 3DCityDB 2012 Oracle Spatial Excellence Award Images courtesy of: TU Berlin, Institute for Geodesy and Geoinformation

Capital Region of Brussels UrbIS 3D see: Van Welsenaere, R., Goud Jutla, G., Das, D., LiDAR Magazine Vol. 4, No. 3 2014 Screenshot courtesy of: Avineon

Point Cloud Classification Screenshot courtesy of Rico Richter, HPI 19

Point Cloud Change Detection Screenshot courtesy of Rico Richter, HPI 23

Point Cloud Change Detection Screenshot courtesy of Rico Richter, HPI 24

Change detection process Using massively parallel algorithms in graphics cards 25 Graphics courtesy of Rico Richter, HPI

Maintenance of 3D City Models Screenshot courtesy of Rico Richter, HPI 26

Maintenance of 3D City Models Screenshot courtesy of Rico Richter, HPI 27

Maintaining City Models Cooperation with Hasso Plattner Institute 28 Graphics courtesy of Rico Richter, HPI

Where do we go from here? Derivation of 3D models Classification, conflation with data from other sources Web-based or service-based rendering Visual inspection, etc. Using the full resolution of the dataset or parts thereof (pyramiding) Selective data dissemination Extract subsets for analysis by external tools In-database processing and analytics Change detection in multi-temporal point clouds (buildings, vegetation,...) 29

Resources Further information on oracle.com www.oracle.com/goto/spatial Blogs https://blogs.oracle.com/oraclespatial Developer forums on OTN https://community.oracle.com/community/database/oracle-database-options/spatial LinkedIn community O a le patial a d G aph g oup Google+ community O a le patial a d G aph IG 30