OIL & GAS Analysing Big Data in ArcGIS AIS based risk modelling Esri European Petroleum GIS Conference 2014 Karl John Pedersen 7th November 2014 1 DNV GL 2014 7th November 2014 SAFER, SMARTER, GREENER
Introduction Large amounts of data are produced from monitoring systems Processing and analysing this data provides a challenge How does DNV GL use Big data AIS vessel tracking Background and challenges Solution Implementation examples Q & A 2
Background Environmental Modelling and GIS unit within Oil & Gas, DNV GL, Oslo DNV GL - International company headquartered in Norway Global classification, certification, technical assurance and advisory company GIS used in risk modelling and visualisation for customers Many projects use AIS data How have we adapted to increasing amounts of AIS data Examples from Oil & Gas and Maritime sectors 150 400 100 16,000 years offices countries employees 3
What is AIS AIS - Automatic Identification System - vessel tracking data Create lines append lines spanning multiple days Join to ship database to get attributes e.g Vessel type & size, engine & fuel type 4
Analysis examples Examples where AIS data are used in risk modelling 5
View vessel traffic around an offshore installation 6
Calculate vessel density around installation collision risk 7
Risk to pipelines from fishing activity Installation 8
Number of fishing vessels per grid cell quantifies risk 9
Fishing activity vs. depth risk from vessel anchors 10
Developing risk models in ArcGIS AIS data used in further analysis AIS data from ArcGIS often used as input to DNV GL risk models Results visualised/mapped in ArcGIS Risk models increasingly developed in ArcGIS: ArcGIS AIS Database Process AIS data in ArcGIS Run risk model Map/ Visualisation Data export for users Excel/Access 11
ArcGIS modelling example Input accident frequencies from Excel Run risk model - Python Arctic Shipping Risk 12
How we previously handled AIS data Data in SQL Server accessed via IBM Cognos Imported text files from Cognos into ArcGIS Challenges: Large text files, lacking data structure Many steps - possibility of errors Limited geographic selection Time consuming Increasing data quantities Historical data Global data Need to improve access to AIS Big data 13
What is Big data? Big data a collection of data sets so large and complex difficult to process using traditional data processing applications (Wikipedia) Example from DNV GL: Database of > 2,5 billion AIS points 14
Solution for handling AIS data IBM Netezza data warehouse implemented to improve data access Implemented in existing Cognos environment What is Netezza: Data warehouse appliance from IBM Provides rapid access and analysis of Big data Includes a Spatial Esri package AIS data readable directly in ArcGIS 15
Making data available to ArcMap from Netezza Netezza Spatial Esri package AIS data imported from supplier Spatial data created in Netezza Polygon data loaded via WKT format 16
Accessing data in Netezza from ArcGIS Install IBM Netezza ODBC driver Add data as normal geographic data has own icon Data access via: ArcToolbox Query Layer Python 17
ArcToolbox Query Layer Processing carried out on server Select geographical area Points mapped directly in ArcGIS Data downloadable to ArcGIS Work reduced from days to minutes 18
Allows analysis of global AIS data 19
Serving maps to users - ArcGIS Server ArcGIS Server service for AIS data created Issues creating service for whole dataset 20
Serving maps to users - Cognos Cognos use spatial data from Netezza Esri Maps for Cognos Implemented simple mapping tool 21
Limitations ArcGIS cannot write to Netezza Issues creating ArcGIS service No support for raster Limited user base 22
Summary Advantages of implementing solution: Enables significantly improved access to AIS data Allows analysis which was not previously possible Part of existing data warehouse Ability for existing non-gis users to analyse data using familiar tools 23
That s all! Questions? Contact details: Karl John Pedersen Principal Specialist, Environmental Modelling and GIS, Environmental Risk Management, Høvik, Norway E-mail karl.john.pedersen@dnvgl.com Mobile +47 950 02 061 www.dnvgl.com SAFER, SMARTER, GREENER 24