Global Sensor Networks An open source data middleware built for data storage by researchers.

Transcription

1 Global Sensor Networks An open source data middleware built for data storage by researchers. Nicholas Dawes, Michael Lehning 1

2 GSN Requirement Most institutes do not have funding for specialist data managers for research data Most research data is stored as files on a file server Many researchers do not have the knowledge required to store their data in a basic database system especially not to stream it. Hours are wasted in reformatting and sending data according to requests Research data may be stored in a centrally accessible repository, or on a local disk 2

3 About GSN Provides easy, reliable streamed data storage and processing methods Multiple database types are supported Replaces many data acquisition scripts by providing configurable wrappers Web services and webbased interfaces Provides the data protection methods required to satisfy most researchers Researchers remain in control of access to their own data 3

4 Usage WSL/SLF Experimental Data 750 virtual-sensors' Set of parameters with a common time-stamp parameters Data for many research sites Short-term experiments Long-term reference stations Data present Weissfluhjoch - almost complete Stillberg - in progress 4

5 Implementation Database GSN Main Server Database Database NAS Database GSN Backup Server Database Database Backup NAS 5

6 Raw Data Types Meteo Stations and other automated sensors 6

7 Raw Data Types Mobile measurement networks 7

8 Raw Data Types Manual measurements Tree heights Understory composition Shoot lengths Water potential Biomass.. 8

9 Modelled data Significant datasets for re-use Alpine3D simulations for specific catchments over specific climate change scenarios 9

10 Basic configuration Sensor data in file Sensor data in database Virtual sensor 10

11 Advanced stream configuration Sensor / Data file Raw data VS1 Sensor data in database "Local virtual-sensor" Calibration function 1 (time 1 - time 2) VS2 Calibration function 2 (time 2 - time 3) Sensor data in database Calibration function 3 Parameter X = 0 11

12 Monitoring - alarms Processing and monitoring ( and SMS alerts) possible through Java scriptlets 12

13 R Integration Sensor data in file Raw data GSN virtual-sensor R-Script Sensor data in database 13

14 External Processing Sensor data in GSN REST query FTP Server Wrapper acquisition Sensor data in GSN 14

15 User administration User requests access Owner accepts/rejects request Admin accepts/rejects request 15

16 Administration Overhead Server updates GSN updates Correction of errors found in the data (if required) Changes to parameters names etc (if required) User admin Sensor rights admin! Semi-automated admin Monitoring to ensure that vs do not take up processing time if no longer live Monitoring to ensure that errors in data do not stop acquisition Monitoring to ensure that disk space is sufficient 16

17 Questions 17