Delft-FEWS User Days 2010 New features in Delft-FEWS 2010.01, 2010.02 and future developments
Overview Organisational aspects; Projects around the globe; New features: 2010.01 2010.02 (end of November 2010) Future research and developments
Delft-FEWS organisation Delft-FEWS Product Management OWB Manager Developments Support & Maintenance simone gerben marcel
Delft-FEWS Developers deltares - operational water management jitka juzer erik deltares software centre andre onno arjen frank saskia frederik arno camiel
Delft-FEWS Project Leaders (OWB) arnejan marc micha dirk simone alex albrecht matthijs klaasjan martin willem yenory frederiek
Delft-FEWS Project Leaders (HYD, ZKS, BGS) paolo jaap bernhard ronald otto daniel annette karel martin joao simone regien rogier neeltje
Delft-FEWS Support marcel (online) support & analysis remote assistence patching remote installation(s) training simone alex gerben juzer erik matthijs andre
ICT-Operational Systems (ICT-OWB) Installation of operational water management (live) systems in Delft and at clients More than Delft-FEWS: Matroos/Nautboom deltares - ict information and communication technology arthur ivo eelco
Deltares-USA Office: Silver Spring NWS-CHPS project project management First contact NWS Site Support Activities in close cooperation with Delft Office peter edwin toon willem-jan
New (external) developers HKV (Lelystad, Johan Ansink) Imports/Exports Display (future?) VORtech (Delft, Hans Bos, Jeroen Gerrits) FEWS Training Module
FEWS Project NL (2010) 1 Deltares 5 HKV 1 7 Nelen & Schuurmans 9 7 8 1. FEWS Noorderzijlvest 2. ISA Hoekse Waard 3. Veluwe (extension) 2 4. FEWS Velt & Vecht (extension) 5. HDSR 6. WS Vallei & Eem 7. WS Zuiderzeeland 8. Rijnland 9. Hunze en Aa s 5 6 3
Use of Delft FEWS in NL FEWS-LIZARD Hollands Noorderkwartier BOS Noorderzijlvest FEWS-Rivieren, FEWS-Meren, FEWS Algen, FEWS Waterbeheer FEWS IWP Rijkswaterstaat BOS Hunze & Aa s WIS Zuiderzeeland WIS Velt & Vecht FEWS-LIZARD Waternet BOS Regge & Dinkel FEWS-LIZARD Hollandse Delta AIM Veluwe WIS Vallei en Eem FEWS-LIZARD Zeeuwse Eilanden WIS HDSR WIS Aa en Maas WIS Peel en Maasvallei WIS & BOS De Dommel WIS Roer en Overmaas
FEWS Projects Europe (2010) 1 Deltares 7 HKV 1. FEWS-EGLV 7 2. FEWS FFC 3. FEWS-Verbund 4. FEWS-Adriatico 5. FEWS-Mino Sil (KV Consultores) 6. FEWS-Tajo (OFITECO) 7. River Suir FFS 5 2 1 4 3 6
FEWS Projects World (2010) 1 Deltares 8 Deltares Inc. 10 CSIRO 9 8 1 1. FEWS New Brunswick 2. FEWS Sudan 3. FEWS Zambesi-basin 4. FEWS/HYMOS Indonesia 5. FEWS Seqwater 6. FEWS-Ovens 7. FEWS Guanabaro Bay 8. FEWS-CHPS (9x CATII) 9. Bonneville Power Forecasting System 3 2 10 7 11 4 6 5 Use of Delft-FEWS in: 10.Condamine Balonne 11. AWRA
New Developments 2010
Dockable GUI
Dockable = multiple
FEWS Explorer Dockable save layout } user specific Color definitions (color blindness) can also be saved
Configuration: Visibility Locations Visibility of locations in FEWS Explorer depenent on system time visibilityperiod > from > to > from/to also configurable through LocationAttributes ( DBF method )
TimeSeries Display: on-the-fly statistics moving average
TimeSeries Display: on-the-fly statistics cumulative per interval
TimeSeries Display: on-the-fly statistics frequency distribution
TimeSeries Display: on-the-fly statistics gaussian distribution
TimeSeries Display = mouse click adding values manually with clicks on graph
TimeSeries Display move values over time
TimeSeries Display box (and whisker) plot option
TimeSeries Display scatter plot option
TimeSeries Display falling dry inundation ice new quality flags
TimeSeries Display Excel -like calculation functionality
TimeSeries Display Excel -Layout possibilities of graphs
Export / Save Graphs Save as: PDF (adobe) SVG (scalable vector graphics)
Export animations Save as animated *gif Using in reports Publish on the Web
Schematic Status Display Some of the current FEWS Displays sometimes to complex for users that only use limited amount of data Special requirements for Real Time Control or Monitoring type modules (System) Diagram SVG file as background
Schematic Status Display What does it do? What can it do? Show values of time series (last, max) Change objects based on values (arrows) Use Thresholds definition for coloring of objects Open TimeSeriesDisplay Open TimeSeriesEditor Start a workflow
Schematic Status Display
Schematic Status Display
Schematic Status Display
Schematic Status Display
Schematic Status Display
Schematic Status Display
Schematic Status Display Other examples
Transformation Module Transformations transferred to new Transformations Module New Transformations: > low pass filter > interpolation of gaps in "360 degrees time series (wind, current) > Thiessen polygons in spatial interpolation
Interpolation Module Bilinear interpolation possible (linear interpolation in two directions) Regular grids and grids with varying column width and row height The four red dots show the data points and the green dot is the point at which we want to interpolate.
Secondary Validation Module Check within time series or based on other time series Generate log messages and adjust quality flags below: upstream level can t be lower than downstream unreliable
Module Adapter: SSARR published interface Delft-FEWS General Adapter input data (pi xml) output data (pi xml) SSARR Module Adapter module input data SSARR Module module output data
Module Adapters Twente Alarm Model (water quality model) WT42D Model (e.g. Brisbane basin, Australia) published interface input data (pi xml) Delft-FEWS General Adapter output data (pi xml) Module Adapter module input data Module module output data
Modifier timeshift Modifier to shift a time series timeshiftconstant Set the amount of time steps for shift positive value = past, negative value = future Compare results with original series
New Projections / Geodatums Indonesia TM-3 zone 48.2 geodatum Transverse Mercator projection based on WGS 1984 also implemented for zones 46.2 to 54.1 Australia GDA94 geodatum Germany Gauss Krueger Meridian2 geodatum Ireland Irish National Grid (TM65) Transverse Mercator projection with GCS_TM65 geodatum Spain ETRS89 geodatum ETRS89 for geographical coordinates UTM-ETRS89xx geodatum for projected coordinates Support projection files (.prj) with ESRI Shapefiles replaces the geodatum in layer or map
New Import / Export Routines Imports Obserview (special csv format) DWD Radar (PI, DXQ ) McIdas (binary radar) UVF (ascii text format) Exports Menyanthes (CSV ASCII format) UMAquo (IDsW standard) Improved imports WISKI 6 WISKI 7
Dino Import Webservice Dino Import Module communicates with SOAP webservice (SOAP = Simple Object Access Protocol) XML messages through HTTP / HTTPS / SMTP or FTP Configuration: <importtype>dinoservice</importtype> <serverurl>http://www.dinoservices.nl/gwservices/gws-v10?wsdl</serverurl>
F12 option database information show statistics of localdatastore total disk space total disk space scalar time series total number of scalar values total disk space grid time series total amount of grids average disk space values 02-11-2010 08:48:49 INFO - Statistics of time series in local data store Total time series disk space 319.4 kb External data disk space 319.4 kb Equidistant Scalars: number of values 626; disk space 7.1 kb; average disk space per scalar 11.6 B Grids: number of grids 10; number of grid cells 5,000,000; disk space 312.3 kb; average disk space per grid 31.2 kb; average disk space per grid cell 0.06 B 02-11-2010 08:48:48 INFO - Scan all time series in local datastore, please wait...
Other developments Improved FEWS JDBC server; Export to relational database (RDBMS export); Improved FEWS PI Webservice; Improved logging (several places); Improvement of user profiles/permissions; Direct log-on using default profile; Parameters properly ordered (alphabetically) in case of combination of individual parameters, parameter groups + child parameters; Child-locations properly ordered (alphabetically);
Live System: Master Controller Synchronisation between more than 2 MC s Order/priority becoming main system > all-mcs-aware (MC-MC synch) Also: 2 MCs as duty-standby + 1 test FSS MC FSS FSS FSS FSS FSS OCs MC MC OCs OCs
Live System: ensembles & workflows Distribution of ensemble runs over multiple FSSs Merging in central database viewing in OC Expiry times per workflow Default (global.properties DEFAULT_EXPIRY_DAYS ) New value configurable in WorkflowDescriptors > deviate from new value during scheduling
Live System: MC Recoverytool System Administrator tool Direct access to MC/central database Command line functionality (separate java-tool) > Removal of time series with a specific creatortaskrunid, moduleinstanceid, ensemblememberid; > Removal of non-active configuration files of the master controller database. > Removal of all configuration files of the master controller database. > Fix for the sequences table (Since 2010_02) multi-mc > Fix for repairing the expirytime of Tasks so that they expire after their taskrunid (Since 2010_02).
Live System: Admin Interface Monitoring of logged on users: User Administration Active Users Also possible on OC (via System Monitor, FewsSessions table) Admin interface query on date and time in logmessages
Live System: Central Database Performance improvements Indices on tables (queries) WarmStates (in case of many WarmStates) Repair of foreign key relations in configuration tables prevents incomplete uploads!! ColdStates Storage Application detects in which place ColdStates are stored: database or configuration Database Rollback Scripts Undoing the update scripts (rollback situation)
Binaries, Backend Systems and Tests (Delft) Compatibility tests run for Windows 7 SQL Server 2008 Clean-up script in /bin/ Removal of unnecessary files (OS libraries) reduction of size /bin/ directory
Documentation (WIKI) Upgrade of documentation: New developments; Reference guide (alphabetically): work in progress; Copyright and Credits
FEWS (System) Components Install instructions (protected) section on WIKI advanced users, intermediairies (N&S, HKV, Cap) training/reference Configurator tool (improved) Generation of relevant installation files Data Transfer Module Historically grown set of file copy/maintenance scripts (Delft) Formalised using subscriptions data supplier #1 data share #1 Delft-FEWS System #1 data supplier n FTP data backup data share n Delft-FEWS System n
FEWS Developers Community External contributions to developments Imports Exports (HKV: UMAquo, Menyanthes) Model Adapters How? FEWS developers: Wizards and IntelliJ / Eclipse projects ; Use of many Delft-FEWS standard code (e.g. PI xml); Result(s): jar file executable Next steps: Code review (Delft-FEWS developers) Inclusion of code in mainstream Delft-FEWS code: yes/no Code tests and documentation (WIKI)
Operator Client (2010.02) Release 2010.02 (planned end of November) Imports/Exports ibever (improved); ASCII export (flags); ModisSca; EasyQ; EnviromonCSV; Improvements Interactive Forecast Displays (IFD) Driving projects: Canada, Australia; All windows docked; Interactivity: > segment map; > segment filters; Modifiers switchoptionmodifier choice from a set of timeseries to be active; missingvaluemodifier set user defined expiry time (long);
Operator Client (2010.02) continued Grib import refactoring (grib1, grib2, netcdf); Expiry times for states can differ; Preconfigured plots with fixed T0 or fixed axis; Clipboard option to copy-paste grid timeseries (result of click in grid) into a scalar plot (and vice versa) Gates (Barrier) Display: interactive opening and closure rules Decision logic (Maeslant) barrier opening and closure;
Operator Client (2010.02) continued Training Module (in cooperation with VORtech): Service to adjust T0 from outside FEWS Hide external data after certain time (SA) Read only Operator Client with direct access to central database No synchronisation Local caching
Towards 2011.01 Related locations : feature to identify that certain locations have more than one relation to other locations (which can not be solved with child locations): gate 1 gate 2 flow Config Manager Improvements Implementation of WaterML2
Towards 2011.01: Delft tests & research Middleware and database upgrades PostgreSQL: 8.4 en 9.0 Oracle 11g JBOSS 5 en JBOSS 6 (already in testing phase) ARTIS system (Automatic Regression Testing Integration System) Automatic tests (daily) Hybrid configuration (e.g. all imports) O-system Isolated Master Controller environment Testing MC improvements/synchronisation VM Ware (parameter)settings & organisation of VM Ware for running Delft- FEWS New localdatastore format: Derby
Questions??