Data Management Tool -

Transcription

1 Data Management Tool - Continuous Logger Data Uploading, Multisite Uploading, and Reporting Tools Created June 2013 Updated May 2015 Developed by: & Funded by: Research Office of Information Technology 1 P age

2 The 3RiversQUEST database was created to provide consistency among the various programs throughout the Upper Ohio River Basin. Realizing that needs of each volunteer monitoring program are unique, our hope is to have created a data uploading feature that can suit the needs of groups collecting continuous logger data, field samples, lab samples and any visual observations. This document was created to provide a step-by-step guide for volunteers to easily and efficiently enter and store their data. Please feel free to provide feedback, as we hope that by developing a framework, we will better serve to encourage standardization for the collection and dissemination of continuous logger data throughout the Upper Ohio River Basin. Note that this framework is a compilation of various meetings and discussions with several groups that are actively collecting continuous logger data. If we ve missed something, let us know! Thanks for your time and collaboration! 2 P age

3 Contents I. Data Logger Upload Feature... 4 Step 1: Setting up a Site... 4 Step 2: Setting up a Data Logger Profile... 4 Step 3: Uploading Data... 7 II. Multi-Site Upload Feature... 9 Step 1: Setting up a Site... 9 Step 2: Setting up a Multi-Site Profile Step 3: Uploading Data III. Manual Data Entry Feature Step 1: Setting up a Site Step 2: Entering Field Data IV. Reporting Features Exporting Data by Site Data Interpretation and Management Data Logger Analysis Field Data Analysis Lab Data Analysis Data Logger Dashboard P age

4 I. Data Logger Upload Feature In effort to capture the Continuous Data Logger data that volunteer water quality monitoring programs are collecting, we have developed an upload feature specifically adjusted to the parameters and sampling time typical of spreadsheets containing logger data. This process has three main steps: (1) sending latitude/longitude coordinates to 3RQ staff to enter the site into the database, (2) creating a data logger profile for each site, and (3) uploading data from the site to the database Step 1: Setting up a Site Before a data logger profile can be created, the site location must be entered into the database by 3RQ staff. If the site has not been previously entered into the database, send the latitude/longitude coordinates and the name for the site location (in decimal degrees) to a 3RQ contact and the site will be entered. For more details, contact Reva Dickson via at rfdickson@mix.wvu.edu. Step 2: Setting up a Data Logger Profile Before a CSV or excel file containing data from a data logger at a particular site can be uploaded, the respective site must have a data logger profile. This must be done initially, but once the logger profile is made, this step will not be necessary for future uploads. The same profile used initially will be used each time an updated spread sheet is uploaded for a particular site. Getting started Setting up a data logger profile entails the following: Entering in basic information identifying the data logger profile Matching the columns and parameters to your spreadsheet Hit submit! You ll need the following information - (per logger) Profile Name: Group: Site: Data Logger Serial #: a name for the logger profile usually the same as the site the name of the volunteer watershed group with which the site is affiliated the name of the site where the data logger is located, as entered in the database the serial number on the logger used at the site 4 P age

5 Step 1: Select Create a Data Logger Profile Step 2: The following table will appear, fill this table in with your Profile Name, Group, Site, and Data Logger Serial Number. Helpful Hint: to search for your site click your cursor in the box next to site and hit enter- this will bring up a list of possible sites and you can click the one you are looking for. Step 3: The next step is matching the data profile with the corresponding excel spreadsheet. To do this open the excel spreadsheet that contains your data logger data. The first step in matching the excel spreadsheet with your data logger profile is finding the starting row for data. The starting row for data is the row on your excel sheet where the actual data begins. For example the starting row for the following excel sheet on the following page would be row P age

6 Step 4: Next we must complete the column name and column number portion of the data logger profile. Select a parameter from the drop down menu and insert the corresponding column number. In the example above, Column B contains the date each sample was taken. Column C contains the time, Colum E contains Level, Column F contain Water Temperature, and Column G contains electric conductivity. The database recognizes columns as numbers not the letters that excel assigns. Therefore if your data for date was in column B you would enter column number 2, and for column C it would be column number 3 and so on. An example of filling out the column name and column number is below. For data logger profiles, the following parameter options will appear: Level Temperature Conductivity Date Time Site Name 6 P age

7 Step 3: Uploading Data Step 1: To upload data logger data contained in an excel spreadsheet, click on Data Logger Import, located under the Data Entry heading. Step 2: Next Enter the Profile Name and Profile ID (if available). If you are uncertain of the exact name, you can select the search icon to the right and a list of data logger profiles will appear. If the search icon is selected the following screen will appear. You can browse by Group, Active Status, or look through all existing profiles by hitting enter in the Profile Name search box. Select Return Value, located in the far left of the table. 7 P age

8 Step 3: Next, Select which file you would like to upload by hitting Choose File. The file will need to be saved somewhere on your computer in order for it to be selected. Note: spreadsheets must be saved as CSV files in excel to upload. To save an excel file as a CSV click save as and then my computer. A box will come up (see below) and where it says save as type: select CSV comma delimited from the drop down menu. Step 4: Once your excel file is saved as a CSV file and you have entered the profile name you can now upload your data logger data. Select browse next to water sample data files. You will be able to select the CSV file from a list of files on your computer. 8 P age

9 Step 5: Once the correct CSV file is selected click the scan button and the follow message should appear on your screen. Congratulations, you ve uploaded data logger data into the 3RQ database! This data will now be displayed in the 3RQ map, as well as being made available for data reporting and interpretation tools discussed below. Data stored in the database can also be exported at any time via the export function or by contacting 3RQ staff. II. Multi-Site Upload Feature In effort to make data upload easier for everyone we have developed a feature where you can upload an entire spreadsheet. This process has three main steps: (1) sending latitude/longitude coordinates to 3RQ staff to enter the site into the database, (2) creating a multi-site profile for each site, and (3) uploading a spreadsheet to the database. Step 1: Setting up a Site Before a multi-site profile can be created, the site location must be entered into the database by 3RQ staff. If the site has not been previously entered into the database, send the latitude/longitude coordinates and the name for the site location (in decimal degrees) to a 3RQ contact and the site will be entered. For more details, contact Reva Dickson via or at rfdickson@mix.wvu.edu. 9 P age

10 Step 2: Setting up a Multi-Site Profile Before a CSV or excel file containing data from a particular site can be uploaded, the spreadsheet must have a multi-site profile. This must be done initially, but once the logger profile is made, this step will not be necessary for future uploads. The same profile used initially will be used each time an updated spread sheet is uploaded for a particular site. Getting started Setting up a multi-site profile entails the following: Entering in basic information identifying the multi-site profile Matching the columns and parameters to your spreadsheet Hit submit! You ll need the following information - (per spreadsheet) Profile Name: a name for the multi-site profile- usually the name of the watershed group Group: the name of the volunteer watershed group with which the site is affiliated Step 1: Select Create a Multi- Site Profile Step 2: The following table will appear. Fill in the Profile Name, and select your Group from the drop down menu. 10 P age

11 Step 3: The next step is matching the multi-site profile with the corresponding excel spreadsheet. To do this open the excel spreadsheet that contains your data. The first step in matching the excel spreadsheet with your multi-site profile is finding the starting row for data. The starting row for data is the row on your excel sheet where the actual data begins. For example the starting row for the following excel sheet on the following page would be row 23. Step 4: Next we must complete the column name and column number portion of the multi-site profile. Select a parameter from the drop down menu and insert the corresponding column number. In the example above, Column B contains the date each sample was taken. Column C contains the time, Colum E contains Level, Column F contain Water Temperature, and Column G contains electric conductivity. The database recognizes columns as numbers not the letters that excel assigns. Therefore if your data for date was in column B you would enter column number 2, and for column C it would be column number 3 and so on. An example of filling out the column name and column number is below. 11 P age

12 For multi-site profiles, the following parameter options will appear: Legacy Site ID Date Time Field Notes ph Specific Conductivity Total Dissolved Solids Alkalinity Chloride Water Temperature Air Temperature Turbidity Surrogate Flow Dissolved Aluminum Dissolved Barium Dissolved Calcium Dissolved Iron Dissolved Magnesium Dissolved Manganese Dissolved Selenium Dissolved Sodium Dissolved Strontium Dissolved Sulfur Bromide Chloride Sulfate Lab ph Lab Specific Conductivity Lab Total Dissolved Solids Lab Acidity 12 P age

13 Lab Alkalinity Lab Hardness Stream Discharge Step 3: Uploading Data Step 1: To upload multi-site data contained in an excel spreadsheet, click on Multi-Site Import, located under the Data Entry heading. Step 2: Enter the Profile Name and Profile ID (if available). If you are uncertain of the exact name, you can select the search icon to the right and a list of data logger profiles will appear. 13 P age

14 If the search icon is selected the following screen will appear. You can browse by Group, Active Status, or look through all existing profiles by hitting enter in the Profile Name search box. Select Return Value, located in the far left of the table. Step 3: Next, Select which file you would like to upload by hitting Choose File. The file will need to be saved somewhere on your computer in order for it to be selected. Note: spreadsheets must be saved as CSV files in excel to upload. To save an excel file as a CSV click save as and then my computer. A box will come up (see below) and where it says save as type: select CSV comma delimited from the drop down menu. 14 P age

15 Step 4: Once your excel file is saved as a CSV file and you have entered the profile name you can now upload your multi-site data. Select choose file next to Multi-Site Water Data File. You will be able to select the CSV file from a list of files on your computer. Step 5: Once the correct CSV file is selected click the scan button. Congratulations, you ve uploaded a multi-site spreadsheet into the 3RQ database! This data will now be displayed in the 3RQ map, as well as being made available for data reporting and interpretation tools discussed below. Data stored in the database can also be exported at any time via the export function or by contacting 3RQ staff. III. Manual Data Entry Feature In effort to make the data within our database consistent across all of our different volunteer groups we have developed an upload feature where groups can enter their data. In addition, we have created universal terms for describing field observations which can simply be selected from a drop down menu. This process has two main steps: (1) sending latitude/longitude coordinates to 3RQ staff to enter the site into the database, (2) entering field data. 15 P age

16 Step 1: Setting up a Site Before you can begin entering field data, the site location must be entered into the database by 3RQ staff. If the site has not been previously entered into the database, send the latitude/longitude coordinates and the name for the site location (in decimal degrees) to a 3RQ contact and the site will be entered. For more details, contact Reva Dickson via at rfdickson@mix.wvu.edu. Step 2: Entering Field Data Step 1: Selecting Site Data First select your group from the drop down menu, once your group is selected you will be able to choose the site from the drop down menu. Once you select the site you need, enter the date and approximate time that this sample was taken. You can also enter any field notes that you have at this point. Step 2: Field Observations Many groups use different terms for field observations. In an effort to increase consistency throughout the site we created drop down menus that give you options for each field observation. The field observations section includes options for weather conditions, precipitation type, water level, and visual turbidity. Shown below is the drop down menu for weather conditions. If data is not collected for a certain parameter you can leave it blank. 16 P age

17 Step 3: Field Collected Data This section allows you to fill in data for measurements that are taken in the field. These measurements include ph, specific conductivity (us/cm), total dissolved solids (mg/l), alkalinity (mg/l), and chloride (mg/l), water temperature (Celsius), air temperature (Celsius), turbidity (ntu), and surrogate flow (ft2). It is important to make sure that all of the data being entered is recorded in the same units used on the website. Step 4: Lab Data This section allows you to fill in data from samples sent to a lab. There are places to record a variety of dissolved metals as well as total metals and anions. The lab bench is a place to record additional lab data such as lab ph, hardness, and alkalinity. Lastly, there is a place to record stream discharge. If you don t have data for each section you may leave the section blank. 17 P age

18 Step 5: Visual Assessment This section is placed to answer additional questions regarding the areas around the stream. This section is shown below. Step 6: Select save. Congratulations, you ve uploaded field data into the 3RQ database! This data will now be displayed in the 3RQ map, as well as being made available for data reporting and interpretation tools discussed below. Data stored in the database can also be exported at any time via the export function or by contacting 3RQ staff. IV. Reporting Features The reporting tools within the 3RQ database are divided into three categories lab analysis, field monitoring, and data logger data. These tools allow for users to limit results by site, date(s), and parameters. Within several clicks, helpful graphs and summary charts allow users to see their data visually, as well as a list of averages, min, max, etc., without spending valuable time manipulating spreadsheets. Moreover, these reports can be shared easily in a variety of formats (pdfs, excel spreadsheets, word documents, etc.). As mentioned above, the 3RQ reporting tools are divided by lab analysis, field monitoring, and data logger data. The main screen shows the following options below: 18 P age

19 Export Data by Site: This tools allows any data to be exported from the database into an excel spreadsheet. Field Summary by Site: This tool is used both for field sampling and data logger data. The parameters here are limited to ph, EC, level, and temperature. Lab Summary by Site: This tool is used to interpret samples with a variety of parameters tested in lab analysis. This is usually done sporadically (i.e. bi-weekly or monthly), and/or during limited sampling. including individual graphs of ph, EC, level, and temperature, and customizable threshold options. Exporting Data by Site Step 1: Click on export data by site the following screen will appear 19 P age

20 Step 2: Next, use the calendar button (circled above) next to each box to select the date range you want data from. The difference between sample date and create date is the sample date is the date the sample was actually taken and the create date is the date that the information was entered into the database. Depending on what you are looking for you could use either or. Step 3: Next select your group from the drop down menu, once you do this you will be able to select the site you want from the drop down menu. Once your group and site are selected hit apply. Step 4: Once you hit apply your screen should look like the following image. Next, select the export button (circled in the image below) on the upper right hand corner of your screen, when you scroll over this button a drop down list will appear like the one in the below image. Select the type of document you would like to export this data as. Step 5: Once you have selected the document type you want, it will either open in another window automatically or a pop up will come up with the option to open the document. Once you have the document open you can save it to your computer. 20 P age

21 Data Interpretation and Management Data Logger Analysis The graph below shows an example of what a complicated data logger spreadsheet can easily be converted into by using our database. As shown below, the database automatically generates minimums, maximums, and averages for the site, as well as visually showing changes over time in EC, Temperature, ph, and Water Level. 21 P age

22 Step 1: Select field summary by site from the main menu. Step 2: Once you select field summary by site the following screen will appear. Here you select the start date and end date of the data you want to view by using the calendar button to the right of the boxes. You will then select your group name from the drop down menu, once you select your group name you will be able to select the site you would like data for. Once you hit apply a graph similar to the one above will appear in the box below the apply button. Step 3: Once you have your graph you can now export it in many different formats. If you run your mouse over the export feature (circled below) in the upper right hand corner you can choose to export your graph as a variety of different documents. 22 P age

23 Field Data Analysis The graph below shows an example of what a spreadsheet of field data can easily be converted into by using our database. As shown below, the database automatically generates minimums, maximums, and averages for the site, as well as visually showing changes over time in conductivity, temperature, ph, and water Level. Step 1: Select field summary by site from the main menu. Step 2: Once you select field summary by site the following screen will appear. Here you select the start date and end date of the data you want to view by using the calendar button to the right of the boxes. You will then select your group name from the drop down menu. Once you 23 P age

24 select your group name you will be able to select the site you would like data for. By clicking apply, a graph similar to the one above will appear in the box below the apply button. Step 3: Once you have your graph you can now export it in many different formats. If you run your mouse over the export feature (circled below) in the upper right hand corner you can choose to export your graph as a variety of different documents. 24 P age

25 Lab Data Analysis The report on the following pages was created from a sampling site from which several years of bi-weekly analytical data was taken. The lab analysis tool has both charts and visualization tools for easy analysis. This feature produces easy to follow graphs that show us the changes in different parameters through time. It also shows us the minimum, maximum, and average for each parameter in a chart. An example of the chart and graphs are shown below. 25 P age

26 Current Date: Record Count: 43 02/10/2015 Allegheny River-Parker Lab Sample Summary From: 02/01/2013 To: 02/10/2015 Minimum Average Maximum Acidity 0 (mg/l) 4 (mg/l) 5 (mg/l) Alkalinity 20 (mg/l) 42 (mg/l) 68 (mg/l) Barium Bromide (mg/l) (mg/l) (mg/l) Cholride 5 (mg/l) 18 (mg/l) 27 (mg/l) Conductivity 110 (ms/cm) 210 (ms/cm) 269 (ms/cm) Dissolved Aluminum (mg/l) (mg/l) (mg/l) Dissolved Calcium 10 (mg/l) 19 (mg/l) 25 (mg/l) Dissolved Iron (mg/l) (mg/l) (mg/l) Dissolved Magnesium 6 (mg/l) 4 (mg/l) 6 (mg/l) Dissolved Manganese (mg/l) (mg/l) (mg/l) Dissolved Sodium 7 (mg/l) 12 (mg/l) 21 (mg/l) Dissolved Sulfur 3 (mg/l) 5 (mg/l) 7 (mg/l) ph (su) (su) (su) Stream Discharge 2,900 (mg/l) 14,504 (mg/l) 35,500 (mg/l) Strontium (mg/l) (mg/l) (mg/l) Sulfate 9 (mg/l) 14 (mg/l) 34 (mg/l) Total Dissolved Solids 10 (mg/l) 105 (mg/l) 162 (mg/l) Total Suspended Solids 0 (mg/l) 10 (mg/l) 61 (mg/l) 26 P age

27 27 P age

28 28 P age

29 Step 1: Select lab summary by site from the right site of the main screen. Step 2: Once you select lab summary by site the following screen will appear. Now select the start date and end date of the data you want to view by using the calendar button to the right of the boxes. You will then select your group name from the drop down menu, once you select your group name you will be able to select the site you would like data for. Once you hit apply a graph will appear in the box below the apply button. Step 3: Once you have your graph you can now export it in many different formats. If you run your mouse over the export feature (circled below) in the upper right hand corner you can choose to export your graph as a variety of different documents. 29 P age

30 Data Logger Dashboard The data logger dashboard gives us the opportunity to view all of our sites in comparison to each other. We can view the temperature, water level, conductivity, and ph levels for each site in one graph. The data logger dashboard also gives us minimums, maximums, and averages for ph, temperature, conductivity, and water level. Step 1: Select volunteer/data logger dashboard from the right site of the main menu. Step 2: After you select the data logger dashboard the following screen will appear. When you get to this screen use the calendar buttons to the right of the time frame boxes to select the date range you want data displayed from. Next select your group. Once your group is selected a list of sites will appear to the right. You can either select a specific site by clicking on the name, or you can click the select all button at the bottom of the list to compare all the sites at once. 30 P age

31 You are also given the option to select thresholds for the different parameters. If you choose to do this a line will appear on the graph for the parameter that you selected a threshold for. This is a good way to see which sites have levels higher or lower than you would like for each parameter. An example of what the graph looks like when a threshold is added for conductivity is shown below. Step 3: Hit the refresh button. Once the refresh button is selected you will see a screen that looks similar to the following 31 P age

32 As you can see each sampling station is assigned a different color which can be seen at the key beneath the graph. By clicking the button to the right of the graph title (shown below) the graph will be displayed larger in a separate window. 32 P age

33 We are very excited about the new features available through our database and hope you all will take advantage of using them. If you have and questions or concerns regarding the database feel free to contact Reva Dickson by at 33 P age