Latest Developments in Oceanographic Applications of GIS including!

Transcription

1 Latest Developments in Oceanographic Applications of GIS including! Near-Real-Time Interactive Map Exemplars & Scientific Empowerment Through Storytelling Dawn Wright Esri Chief Scientist Affiliate Professor, Oregon State U. Schmidt Ocean Institute Research Planning Workshop, August 26, 2015

2

3 Analytics Dashboards Cool Maps Storytelling Apps

4 Analytics!.

5 Web & Real-Time GIS We b Device Desktop! Integration, analysis of highvelocity & -volume data Stream Layer Apps! Updates, alerts sent out Visualization Access Ingestion Stream Service Analytics A GeoEvent GeoAnalytics Extension Extension Services Storage High velocity and volume spatiotemporal data ArcGIS Server ArcGIS Spatiotemporal Big Data Store

6 Types of (Geo)Analytics Aggregate by Polygons or Cells

7 Thanks to Adam Mollenkopf, Esri Real-Time GIS Capability Team Lead and Ricardo Trujillo, GeoEvent Developer, links.esri.com/geoevent-wiki We aim to register large data stores / data sets with ArcGIS Server, then distribute analysis across a cluster of machines for parallel processing Performance example: buffer 8.2 million points or thousands of polygons in a little over a minute, Coming: ~250,000 writes to disk per second across 5 nodes Many frameworks/technologies exist for distributing computation E.g., Hadoop, MapReduce, Spark Spark: processes distributed data in memory; Supports MapReduce programming model Includes additional framework level distributed algorithms ArcGIS Server integrates these technologies on a cluster to solve analytic problems 5

8 Parallelized batch analy0cs on tabular, vector, raster, and imagery datasets (big and standard data) Performance example: buffer 8.2 million points or thousands of polygons in a little over a minute, NetCDF output is supported on Aggregate Points Point in polygon aggregation (spatial join) Aggregate by Cell simple aggregation Summarize Nearby spatial join where left dataset and a distance are used to inflate the bounding box. A distance calculation is then employed (no drivetime/distance). Summarize Within a spatial join that is a generalization of Aggregate Points Find Existing Locations an attribute query followed by/with a spatial query (?); the attribute portion is SparkSQL, the spatial relationship is: intersections, within distance, contains, and within. Find Similar Locations non-spatial similarity search (SparkSQL). Calculate Density cell-based spatial aggregation by attribute with a weighted neighborhood/ distance-based summarization. Find Hot Spots very similar to Calculate Density but with the Getis-Ord Gi* statistic. This can be cell-based (point observations) or polygon-based (polygon source data). Create Buffers simple spatial buffer generation across the dataset (the Java geometry library supports Euclidean and Geodetic buffer generation). This is done. Extract Data a simple region query with a specified output format. Field (RASTER) Calculator apply a Scala or SQL expression to each feature, one expression per field. 6

9 Dimension Explorer Add-In (Desktop)

10 Interoperability / Crosswalking Out of the Box Simple Integration

11 r-arcgis.github.io

12 Interoperability / Crosswalking

13 Web & Real-Time GIS We b Device Desktop Apps Stream Layer Visualization Access Ingestion Stream Service Analytics A GeoEvent GeoAnalytics Extension Extension Services Storage High velocity and volume spatiotemporal data ArcGIS Server ArcGIS Spatiotemporal Big Data Store

14 Dashboards!

15

16 Wave Glider Sensor Dashboard: bit.ly/gcooswaveglider Shin Kobara, Texas A&M From Kobara et al., 2015, Ocean Solutions, Earth Solutions

17 Active Vessels Dashboard Prototype Shin Kobara, Texas A&M Esri Global Fishing Watch aimed at Illegal, Unreported, Unregulated Fishing

18 Cool Maps!.

19 coolmaps.esri.com/bigdata/shippingtracks

20 coolmaps.esri.com/bigdata/shippingtracks

21 coolmaps.esri.com/bigdata/shippingglobe

22 coolmaps.esri.com/bigdata/shippingglobe

23 coolmaps.esri.com/bigdata/shippingglobe

24 arcg.is/1pg7jlg

25 Once upon a time!

26 Stories are data with a soul,!persuasive and memorable. Nancy Baron, Director of Science Outreach, COMPASS

27 Licensed personally to Dawn Wright by The New Yorker Cartoon Bank, TCB-86966, Invoice L12480 By Charles Saxon Published March 21, 1983

28 Communicating with! Story Maps

29 esriurl.com/ocnstories

30 esriurl.com/ocnstories esriurl.com/ocnstories

31 Sensor dashboard as a story map

32 esriurl.com/ocnstories

33 esriurl.com/ocnstories

34 storymaps.arcgis.com

35 esriurl.com/ocnstories

36

37

38 Natural Resources Defense Council, esriurl.com/ocnstories

39 Natural Resources Defense Council, esriurl.com/ocnstories

40 Natural Resources Defense Council, esriurl.com/ocnstories

41 Final Chapter Consider potential of stories and scientific storytelling via maps, photos, videos, more Consider implications for education, training, certification? Consider implications for ocean science contributions to society Dawn esriurl.com/scicomm, esriurl.com/ocnstories, esriurl.com/ocnres

42 Extra Slides

43 Story map made into a conference poster with interactive version on table for discussion &questions, e.g., esriurl.com/agustories

44 esriurl.com/analyticalstories

45 How do I get real-time data into a GIS? ArcGIS example: Use an Input Connector Inputs GeoEvent Extension GeoEvent Services Outputs You can create your own connectors. Poll an ArcGIS Server for Features ActiveMQ CompassLDE Esri Out of the Box HTTP REST WS WS.csv Poll an external website for GeoJSON, JSON, or XML Features, GeoJSON, JSON, or XML on a REST endpoint Receive GeoJSON or JSON on a WebSocket Receive RSS Receive Text from a TCP or UDP Socket Subscribe to external WebSocket for GeoJSON or JSON Watch a Folder for new CSV, GeoJSON, or JSON Files Esri Gallery CAP Cursor-on-Target Flume * Instagram KML Kafka MQTT * * Partner Gallery envirocar exactearth AIS FAA (ASDI) GNIP Networkfleet OSIsoft Valarm * * * * NMEA 0183 RabbitMQ Zonar * Sierra Wireless (RAP) Trimble (TAIP) Twitter

46 Applying Real-Time Analytics ArcGIS example: Use a GeoEvent Processor Inputs GeoEvent Extension GeoEvent Services Outputs You can create your own processors. Aggregator * Geotagger Add XYZ Range Fan Esri Out of the Box Buffer Creator Convex Hull Creator Difference Creator Envelope Creator Field Calculator Field Enricher Field Mapper Incident Detector Intersector Projector Simplifier * Sliding Time Window Symmetric Difference Track Gap Detector Esri Gallery Bearing Ellipse Event Volume Control Extent Enricher Field Grouper GeoNames Lookup Motion Calculator Reverse Geocoder Service Area Creator Symbol Lookup Track Idle Detector Unit Converter Visibility Query Report Field Reducer Union Creator

47 How do I update and alert those who need it where they need it? Disseminate notifications, alerts, and updates using an Output Connector Inputs GeoEvent Extension GeoEvent Services Outputs You can create your own connectors. Add or Update a feature ActiveMQ Publish Text to a UDP Socket Cursor-on-Target Esri Out of the Box HTTP WS im Push GeoJSON or JSON to an external Website Push GeoJSON or JSON to an external WebSocket Push Text to an external TCP Socket Send a Text Message Send an Send an Instant Message Send Features to a Stream Service Esri Gallery Flume Hadoop (HDFS) Kafka MongoDB MQTT RabbitMQ Twitter * *.csv Write to a CSV, GeoJSON, or JSON File Write to a Spatiotemporal Data Store *

48 Getting Real-Time Data into Web Applications Two patterns! Feature layers pull from feature services -! Web apps poll to get periodic updates -! Must be backed by an enterprise geodatabase (EGDB)! Stream layers subscribe to stream services -! Web apps subscribe to immediately receive data -! Low latency and high throughput ArcGIS Server feature layers Polling (Pull) Your Applications GeoEvent Extension Update a Feature Add a Feature Map Services Feature Services Stream Services EGDB Send Features to a Stream Service Feature Layer Stream Layer