Delft-FEWS User Days 2010 New features in Delft-FEWS

Transcription

1 Delft-FEWS User Days 2010 New features in Delft-FEWS , and future developments

2 Overview Organisational aspects; Projects around the globe; New features: (end of November 2010) Future research and developments

3 Delft-FEWS organisation Delft-FEWS Product Management OWB Manager Developments Support & Maintenance simone gerben marcel

4 Delft-FEWS Developers deltares - operational water management jitka juzer erik deltares software centre andre onno arjen frank saskia frederik arno camiel

5 Delft-FEWS Project Leaders (OWB) arnejan marc micha dirk simone alex albrecht matthijs klaasjan martin willem yenory frederiek

6 Delft-FEWS Project Leaders (HYD, ZKS, BGS) paolo jaap bernhard ronald otto daniel annette karel martin joao simone regien rogier neeltje

7 Delft-FEWS Support marcel (online) support & analysis remote assistence patching remote installation(s) training simone alex gerben juzer erik matthijs andre

8 ICT-Operational Systems (ICT-OWB) Installation of operational water management (live) systems in Delft and at clients More than Delft-FEWS: Matroos/Nautboom deltares - ict information and communication technology arthur ivo eelco

9 Deltares-USA Office: Silver Spring NWS-CHPS project project management First contact NWS Site Support Activities in close cooperation with Delft Office peter edwin toon willem-jan

10 New (external) developers HKV (Lelystad, Johan Ansink) Imports/Exports Display (future?) VORtech (Delft, Hans Bos, Jeroen Gerrits) FEWS Training Module

11 FEWS Project NL (2010) 1 Deltares 5 HKV 1 7 Nelen & Schuurmans FEWS Noorderzijlvest 2. ISA Hoekse Waard 3. Veluwe (extension) 2 4. FEWS Velt & Vecht (extension) 5. HDSR 6. WS Vallei & Eem 7. WS Zuiderzeeland 8. Rijnland 9. Hunze en Aa s 5 6 3

12 Use of Delft FEWS in NL FEWS-LIZARD Hollands Noorderkwartier BOS Noorderzijlvest FEWS-Rivieren, FEWS-Meren, FEWS Algen, FEWS Waterbeheer FEWS IWP Rijkswaterstaat BOS Hunze & Aa s WIS Zuiderzeeland WIS Velt & Vecht FEWS-LIZARD Waternet BOS Regge & Dinkel FEWS-LIZARD Hollandse Delta AIM Veluwe WIS Vallei en Eem FEWS-LIZARD Zeeuwse Eilanden WIS HDSR WIS Aa en Maas WIS Peel en Maasvallei WIS & BOS De Dommel WIS Roer en Overmaas

13 FEWS Projects Europe (2010) 1 Deltares 7 HKV 1. FEWS-EGLV 7 2. FEWS FFC 3. FEWS-Verbund 4. FEWS-Adriatico 5. FEWS-Mino Sil (KV Consultores) 6. FEWS-Tajo (OFITECO) 7. River Suir FFS

14 FEWS Projects World (2010) 1 Deltares 8 Deltares Inc. 10 CSIRO FEWS New Brunswick 2. FEWS Sudan 3. FEWS Zambesi-basin 4. FEWS/HYMOS Indonesia 5. FEWS Seqwater 6. FEWS-Ovens 7. FEWS Guanabaro Bay 8. FEWS-CHPS (9x CATII) 9. Bonneville Power Forecasting System Use of Delft-FEWS in: 10.Condamine Balonne 11. AWRA

15 New Developments 2010

16 Dockable GUI

17 Dockable = multiple

18 FEWS Explorer Dockable save layout } user specific Color definitions (color blindness) can also be saved

19 Configuration: Visibility Locations Visibility of locations in FEWS Explorer depenent on system time visibilityperiod > from > to > from/to also configurable through LocationAttributes ( DBF method )

20 TimeSeries Display: on-the-fly statistics moving average

21 TimeSeries Display: on-the-fly statistics cumulative per interval

22 TimeSeries Display: on-the-fly statistics frequency distribution

23 TimeSeries Display: on-the-fly statistics gaussian distribution

24 TimeSeries Display = mouse click adding values manually with clicks on graph

25 TimeSeries Display move values over time

26 TimeSeries Display box (and whisker) plot option

27 TimeSeries Display scatter plot option

28 TimeSeries Display falling dry inundation ice new quality flags

29 TimeSeries Display Excel -like calculation functionality

30 TimeSeries Display Excel -Layout possibilities of graphs

31 Export / Save Graphs Save as: PDF (adobe) SVG (scalable vector graphics)

32 Export animations Save as animated *gif Using in reports Publish on the Web

33 Schematic Status Display Some of the current FEWS Displays sometimes to complex for users that only use limited amount of data Special requirements for Real Time Control or Monitoring type modules (System) Diagram SVG file as background

34 Schematic Status Display What does it do? What can it do? Show values of time series (last, max) Change objects based on values (arrows) Use Thresholds definition for coloring of objects Open TimeSeriesDisplay Open TimeSeriesEditor Start a workflow

35 Schematic Status Display

36 Schematic Status Display

37 Schematic Status Display

38 Schematic Status Display

39 Schematic Status Display

40 Schematic Status Display

41 Schematic Status Display Other examples

42 Transformation Module Transformations transferred to new Transformations Module New Transformations: > low pass filter > interpolation of gaps in "360 degrees time series (wind, current) > Thiessen polygons in spatial interpolation

43 Interpolation Module Bilinear interpolation possible (linear interpolation in two directions) Regular grids and grids with varying column width and row height The four red dots show the data points and the green dot is the point at which we want to interpolate.

44 Secondary Validation Module Check within time series or based on other time series Generate log messages and adjust quality flags below: upstream level can t be lower than downstream unreliable

45 Module Adapter: SSARR published interface Delft-FEWS General Adapter input data (pi xml) output data (pi xml) SSARR Module Adapter module input data SSARR Module module output data

46 Module Adapters Twente Alarm Model (water quality model) WT42D Model (e.g. Brisbane basin, Australia) published interface input data (pi xml) Delft-FEWS General Adapter output data (pi xml) Module Adapter module input data Module module output data

47 Modifier timeshift Modifier to shift a time series timeshiftconstant Set the amount of time steps for shift positive value = past, negative value = future Compare results with original series

48 New Projections / Geodatums Indonesia TM-3 zone 48.2 geodatum Transverse Mercator projection based on WGS 1984 also implemented for zones 46.2 to 54.1 Australia GDA94 geodatum Germany Gauss Krueger Meridian2 geodatum Ireland Irish National Grid (TM65) Transverse Mercator projection with GCS_TM65 geodatum Spain ETRS89 geodatum ETRS89 for geographical coordinates UTM-ETRS89xx geodatum for projected coordinates Support projection files (.prj) with ESRI Shapefiles replaces the geodatum in layer or map

49 New Import / Export Routines Imports Obserview (special csv format) DWD Radar (PI, DXQ ) McIdas (binary radar) UVF (ascii text format) Exports Menyanthes (CSV ASCII format) UMAquo (IDsW standard) Improved imports WISKI 6 WISKI 7

50 Dino Import Webservice Dino Import Module communicates with SOAP webservice (SOAP = Simple Object Access Protocol) XML messages through HTTP / HTTPS / SMTP or FTP Configuration: <importtype>dinoservice</importtype> <serverurl>

51 F12 option database information show statistics of localdatastore total disk space total disk space scalar time series total number of scalar values total disk space grid time series total amount of grids average disk space values :48:49 INFO - Statistics of time series in local data store Total time series disk space kb External data disk space kb Equidistant Scalars: number of values 626; disk space 7.1 kb; average disk space per scalar 11.6 B Grids: number of grids 10; number of grid cells 5,000,000; disk space kb; average disk space per grid 31.2 kb; average disk space per grid cell 0.06 B :48:48 INFO - Scan all time series in local datastore, please wait...

52 Other developments Improved FEWS JDBC server; Export to relational database (RDBMS export); Improved FEWS PI Webservice; Improved logging (several places); Improvement of user profiles/permissions; Direct log-on using default profile; Parameters properly ordered (alphabetically) in case of combination of individual parameters, parameter groups + child parameters; Child-locations properly ordered (alphabetically);

53 Live System: Master Controller Synchronisation between more than 2 MC s Order/priority becoming main system > all-mcs-aware (MC-MC synch) Also: 2 MCs as duty-standby + 1 test FSS MC FSS FSS FSS FSS FSS OCs MC MC OCs OCs

54 Live System: ensembles & workflows Distribution of ensemble runs over multiple FSSs Merging in central database viewing in OC Expiry times per workflow Default (global.properties DEFAULT_EXPIRY_DAYS ) New value configurable in WorkflowDescriptors > deviate from new value during scheduling

55 Live System: MC Recoverytool System Administrator tool Direct access to MC/central database Command line functionality (separate java-tool) > Removal of time series with a specific creatortaskrunid, moduleinstanceid, ensemblememberid; > Removal of non-active configuration files of the master controller database. > Removal of all configuration files of the master controller database. > Fix for the sequences table (Since 2010_02) multi-mc > Fix for repairing the expirytime of Tasks so that they expire after their taskrunid (Since 2010_02).

56 Live System: Admin Interface Monitoring of logged on users: User Administration Active Users Also possible on OC (via System Monitor, FewsSessions table) Admin interface query on date and time in logmessages

57 Live System: Central Database Performance improvements Indices on tables (queries) WarmStates (in case of many WarmStates) Repair of foreign key relations in configuration tables prevents incomplete uploads!! ColdStates Storage Application detects in which place ColdStates are stored: database or configuration Database Rollback Scripts Undoing the update scripts (rollback situation)

58 Binaries, Backend Systems and Tests (Delft) Compatibility tests run for Windows 7 SQL Server 2008 Clean-up script in /bin/ Removal of unnecessary files (OS libraries) reduction of size /bin/ directory

59 Documentation (WIKI) Upgrade of documentation: New developments; Reference guide (alphabetically): work in progress; Copyright and Credits

60 FEWS (System) Components Install instructions (protected) section on WIKI advanced users, intermediairies (N&S, HKV, Cap) training/reference Configurator tool (improved) Generation of relevant installation files Data Transfer Module Historically grown set of file copy/maintenance scripts (Delft) Formalised using subscriptions data supplier #1 data share #1 Delft-FEWS System #1 data supplier n FTP data backup data share n Delft-FEWS System n

61 FEWS Developers Community External contributions to developments Imports Exports (HKV: UMAquo, Menyanthes) Model Adapters How? FEWS developers: Wizards and IntelliJ / Eclipse projects ; Use of many Delft-FEWS standard code (e.g. PI xml); Result(s): jar file executable Next steps: Code review (Delft-FEWS developers) Inclusion of code in mainstream Delft-FEWS code: yes/no Code tests and documentation (WIKI)

62 Operator Client ( ) Release (planned end of November) Imports/Exports ibever (improved); ASCII export (flags); ModisSca; EasyQ; EnviromonCSV; Improvements Interactive Forecast Displays (IFD) Driving projects: Canada, Australia; All windows docked; Interactivity: > segment map; > segment filters; Modifiers switchoptionmodifier choice from a set of timeseries to be active; missingvaluemodifier set user defined expiry time (long);

63 Operator Client ( ) continued Grib import refactoring (grib1, grib2, netcdf); Expiry times for states can differ; Preconfigured plots with fixed T0 or fixed axis; Clipboard option to copy-paste grid timeseries (result of click in grid) into a scalar plot (and vice versa) Gates (Barrier) Display: interactive opening and closure rules Decision logic (Maeslant) barrier opening and closure;

64 Operator Client ( ) continued Training Module (in cooperation with VORtech): Service to adjust T0 from outside FEWS Hide external data after certain time (SA) Read only Operator Client with direct access to central database No synchronisation Local caching

65 Towards Related locations : feature to identify that certain locations have more than one relation to other locations (which can not be solved with child locations): gate 1 gate 2 flow Config Manager Improvements Implementation of WaterML2

66 Towards : Delft tests & research Middleware and database upgrades PostgreSQL: 8.4 en 9.0 Oracle 11g JBOSS 5 en JBOSS 6 (already in testing phase) ARTIS system (Automatic Regression Testing Integration System) Automatic tests (daily) Hybrid configuration (e.g. all imports) O-system Isolated Master Controller environment Testing MC improvements/synchronisation VM Ware (parameter)settings & organisation of VM Ware for running Delft- FEWS New localdatastore format: Derby

67 Questions??