Web-based data management and data science with the OpenEarth community at Deltares Part 1: Decision support and Real-time forecasting Annette Zijderveld (with support of my colleagues) Outline Introduction Deltares Forecasting and Delft-FEWS Example projects Getting involved 1
Introduction Deltares Living in deltas, coastal areas and river basins 2
Living in deltas, coastal areas and river basins Living in deltas, coastal areas and river basins 3
Leading top institute Societal themes 4
Facts and figures Excellent expertise 1000039.005 5
Sharing knowledge Through serious gaming, (open source) software, publications, data and models, doctorate supervision, Deltares Academy, workshops, congresses and projects/research projects Open software System informatics Deltares software used in more than 100 countries Open software establishes sharing community 6
Sand Engine Hydro- & morphodynamics Ter Heijde peninsula: 128 hectares (256 soccer fields) Protection against rising sea levels and space for nature Extension of Rotterdam harbour Hydro- and morphodynamics/ Environmental sciences & eco-engineering / Hydraulics and geotechnical engineering Research and advice on port design Testing coastal defences using scale models Tidal flow predictions for ships during the construction phase Environmental impact assessments and cooling water discharges 7
Flood forecasting for USA Hydrological sciences / System informatics Support for National Weather Service and Forecasting Centres during implementation of their own system based on Delft-FEWS Thailand floods Hydrological sciences / Social sciences & policy analyses Advising thethai Prime Minister and crisis team on flood control measures (1 November 2011). 8
Forecasting & Delft FEWS Properties at Risk?!? 9
Response Forecasting, Warning & Response Measurements Warning Response Forecasting From data to information DELTARES Simulation Key Questions More lead time? More accurate? Risk of flooding due to levee failure? 10
Data Delft FEWS Open Shell Forecasting System Hydraulic Models Forecasts & warnings Delft FEWS Open Shell Forecasting System System for operational forecasting (resilience!) Flood, surge, water quality, structure stability Fully configurable by (super)users Platform for operational research Short cycle from research to operations Java, PostgreSQL/Oracle, Jboss, XML Operating system independent, very scalable Software free of charge central role for user community 11
Sharing Tools > 50 models of different nature DELFT-FEWS Model Type Supplier/Owner Country ISIS 1D Hydrodynamic HR/Halcrow UK Flux 1D Hydrodynamics Scietec Austria DELFT3D 2/3D Hydrodynamics/ Water quality Deltares Netherlands TWAM 2D Hydrodynamics PlanB UK DELFT-3D 2D-3D Hydrodynamics Deltares Netherlands Modflow96/VKD 3D groundwater Deltares/Adam Taylor Netherlands/UK BASEFLOW Baseflow Simulation NWS USA CHANLOSS Channel loss Simulation NWS USA CONSUSE Consumptive use of River Simulation NWS USA Grid2Grid Distributed Hydrologic Model CEH UK REW Distributed Rainfall-Runoff Deltares Netherlands Vflo Distributed Rainfall-Runoff Vieux & Assiciates USA PCRASTER Dynamic Modelling Software University Utrecht Netherlands ARMA Error Correction CEH UK PRTF Event Based RR PlanB UK GLACIER Glacier simulation NWS USA HEC-RAS Hydrodynamic USACE USA SynHP Hydrodynamics BfG Germany Mike11 Hydrodynamics DHI Denmark SOBEK Hydrodynamics, Water Quality, RR Deltares Netherlands SOBEK-2d Linked 1d/2d inundation modelling Deltares Netherlands RSNELEV Rain Snow Elevation Simulation NWS USA PDM Rainfall-Runoff CEH UK TCM Rainfall-Runoff CEH UK Sacramento Rainfall-Runoff Deltares Netherlands NAM Rainfall-Runoff DHI Denmark MCRM Rainfall-Runoff EA UK Wageningen model Rainfall-Runoff Haskoning Netherlands SAC-SMA Rainfall-Runoff NWS USA API-CONT Rainfall-Runoff NWS USA SACSMATH Rainfall-Runoff NWS USA TOPKAPI Rainfall-Runoff Univ. of Bologna Italy PRMS Rainfall-Runoff Univ. of Karlsruhre Germany HBV Rainfall-Runoff (inc snowmelt) SHMI Sweden URBS Rainfall-runoff and hydrological routing Don Caroll Australia rtcmodule Reservoir Simulation Deltares Netherlands SSARRESV Reservoir Simulation NWS USA RESSNGL Reservoir Simulation NWS USA HEC-ResSim Reservoir Simulation USACE USA LAG/K Routing (hydrological) NWS USA SARROUTE Routing (hydrological) NWS USA KW Routing (kinematic wave) CEH UK DODO Routing (layered Muskingum) EA UK LAYCOEF Routing Model NWS USA MUSKROUT Routing Model NWS USA LAYCOEF Routing model NWS USA TATUM Routing Model NWS USA PACK Snow Melt CEH UK Snow17 Snow Melt NWS USA TRITON Surge propagation/overtopping PlanB UK STF Transfer functions EA UK Unit-HG Unit-Hydrograph NWS USA SWMM Urban Rainfall-Runoff USGS USA RIBASIM Water distribution + Reservoir Deltares Netherlands Model Generic Module Type Supplier/Owner Country ISIS Adapter 1D Hydrodynamic HR/Halcrow UK Flux 1D Hydrodynamics Scietec Austria DELFT3D 2/3D Hydrodynamics/ Water quality Deltares Netherlands TWAM XML 2D Hydrodynamics PlanB UK DELFT-3D 2D-3D Hydrodynamics Deltares Netherlands Modflow96/VKD 3D groundwater Deltares/Adam Taylor Netherlands/UK BASEFLOW Baseflow Simulation NWS USA CHANLOSS Channel loss Simulation NWS USA Model CONSUSE Consumptive use of River Simulation NWS USA Grid2Grid Adapter Distributed Hydrologic Model CEH UK REW Distributed Rainfall-Runoff Deltares Netherlands Native Vflo Distributed Rainfall-Runoff Native Vieux & Assiciates USA format PCRASTER Model Dynamic Modelling format Software University Utrecht Netherlands ARMA Error Correction CEH UK PRTF Event Based RR PlanB UK GLACIER Glacier simulation NWS USA HEC-RAS Hydrodynamic USACE USA SynHP public.deltares.nl/display/fewsdoc/models+linked+to+delft-fews Hydrodynamics BfG Germany Mike11 Hydrodynamics DHI Denmark SOBEK Hydrodynamics, Water Quality, RR Deltares Netherlands SOBEK-2d Linked 1d/2d inundation modelling Deltares Netherlands RSNELEV Rain Snow Elevation Simulation NWS USA PDM Rainfall-Runoff CEH UK TCM Rainfall-Runoff CEH UK Sacramento Rainfall-Runoff Deltares Netherlands NAM Rainfall-Runoff DHI Denmark MCRM Rainfall-Runoff EA UK Wageningen model Rainfall-Runoff Haskoning Netherlands SAC-SMA Rainfall-Runoff NWS USA API-CONT Rainfall-Runoff NWS USA SACSMATH Rainfall-Runoff NWS USA TOPKAPI Rainfall-Runoff Univ. of Bologna Italy Examples of Models linked to FEWS PRMS Rainfall-Runoff Univ. of Karlsruhre Germany HBV Rainfall-Runoff (inc snowmelt) SHMI Sweden URBS Rainfall-runoff and hydrological routing Don Caroll Australia rtcmodule Reservoir Simulation Deltares Netherlands SSARRESV Reservoir Simulation NWS USA RESSNGL Distributed Reservoir hydrology Simulation NWS USA HEC-ResSim Reservoir Simulation USACE USA LAG/K REW Routing (Delft (hydrological) Hydraulics), G2G (CEH) NWS USA SARROUTE Routing (hydrological) NWS USA Topkapi (Univ of Bologna), Lisflood (JRC) KW Routing (kinematic wave) CEH UK DODO PCRaser Routing (Univ (layered of Muskingum) Utrecht) EA UK LAYCOEF Routing Model NWS USA MUSKROUT Routing Model NWS USA LAYCOEF Routing model NWS USA TATUM Hydrodynamics Routing Model NWS USA PACK 1D: Mike11 Snow Melt(DHI), ISIS (HRW), SOBEK (incl. CEH WQ), RAS (HEC) UK Snow17 Snow Melt NWS USA TRITON 2D/3D: Surge Delft3D propagation/overtopping (incl WQ) PlanB UK STF Transfer functions EA UK Unit-HG Unit-Hydrograph NWS USA SWMM Urban Rainfall-Runoff USGS USA Groundater RIBASIM Water distribution + Reservoir Deltares Netherlands MODFLOW (USGS) Water allocation Ribasim (Delft Hydraulics) 12
Open System Open to many data formats (XML, Grib, NetCDF etc.) Process configuration - Workflows HD Model Model 1 Flood Forecasting System Model Current 2 University Model 3 Services offered by Delft FEWS Import & Export Validation & Interpolation (temporal & spatial) Transformation Model execution (manual, automatic, ensemble) What-if scenarios & modifiers Error correction & updating Performance indicators Flood mapping Display (times series, longitudinal, spatial) Configuration management Web reporting Archiving (Matroos/OpenEarth) Calibration (OpenDA) Verification 13
Delft FEWS User Community Delft FEWS User Community Environment Agency for England & Wales National Flood Forecasting System (NFFS) Scottish Environmental Protection Agency FEWS Scotland Rijkswaterstaat & waterboards in The Netherlands Demonstrator for St Petersburg Flood Protection Barrier National Weather Service (NWS/NOAA) in United States Community Hydrologic Prediction System (CHPS) Bureau of Meteorology in Australia National Application 14
RWsOS Integrated forecasting system of Rijkswaterstaat for all areas: Rivers (Rhine, Meuse, IJssel, Vecht) IJssellake area Coast Rotterdam harbour area Channels Control barriers Purpose: RWS HMCN Flood forecasting/ Storm surge forecasting Drought forecasting Shipping guidance for the Rotterdam, Tide gates for huge carriers Swell data Data for Watermanagement RWsOS North Sea 15
DCSM v6 water levels SWAN North Sea: wave hight 16
Water level forecast Surge forecast 17
RWsOS North Sea concept ICT platform FSS 1 FEWS Database Clients: OC SVSD ICT platform FSS 2 Master Controller OC HMCN OC HMCZ ICT platform FSS 3 Archive Database (Matroos) OC Maeslant barrier MATROOS Matroos web interface Developed 2003/2004 Times series/ map data Still operational 18
MATROOS RWS waterportal Separation DB In archive and short-term cache Matroos goes OpenEarth MATROOS Deltares app on storm surge waterlevels Forecast Information right on my smartphone 19
Crisis management Lakes Coast line Coastal response group Lakes response group Rivers National Flood Committee Rivers response group Delta response Delta area group ISCRAM April 22-25 2012, Vancouver 39 National Exercise Coast line Coastal response group Lakes Lakes response group Rivers learning goal: cooperation Rivers response group Delta response Delta area group ISCRAM April 22-25 2012, Vancouver 40 20
Delft-FEWS systems Coastal response group ISCRAM April 22-25 2012, Vancouver 41 Delft-FEWS systems and the Water Coach ISCRAM April 22-25 2012, Vancouver 42 21
Global Flood Partnership http://portal.gdacs.org/. Global Flood Partnership Common data for large scale to local model building Standardization key issue!! Model forcing, as well as static datasets to build models rapidly Global flood models running on open architecture (operational/risk) Model nesting options Variable resolution across the domain Global Tide Model Example: Flexible Mesh 22
Philosophy for open data/tools: OpenEarth Agreements on how to store data so that it can be used for: Rapid visualisation Fast model building (e.g. DEMs, bathymetry, land use) Agreements how to make available data and tools Data preferably through OPeNDAP server. Advantage: each institute can host its own data (cloud) rather than a central server solution Tools open and under version control (svn). Agreements how to access tools and data All institutes make their own tools in matlab, C, fortran, python, and so on Let s keep on doing things the way we are used to Solution: model adapters/wrappers for models/tools Modelling storm surge impact Delft3D + Delft Dashboard 23
Tools: Delft-Dashboard Boundary of large scale model Nested model, taking boundary conditions from large scale model Works with NetCDF-CF data standard Extracts from SRTM+, and many other bathymetry data sources Extracts from tidal model data (TOPEX/POSEIDON) Not yet wrapped (now in matlab) not yet other codes than Delft-3D Future: expand to inundation models Getting involved 24
Topics and tasks for students Deltares can offer you 1) Internship 2) Master thesis project 3) Starting position as data science advisor/ researcher Example projects: Decision support system for gaining more insight of cascading effects of flooding on critical infrastructure (EU project CIPRnet: www.ciprnet.eu) Working on data support for global flood/ draught models (GFP) Working on visualisation/ apps/ dashboards based on Delft-FEWS systems (RWS or other clients) Working on serious gaming applications (Watercoach) Questions? 25