Water Quality Data Analysis & R Programming Internship Central Coast Water Quality Preservation, Inc. March 2013 February 2014

Transcription

1 Water Quality Data Analysis & R Programming Internship Central Coast Water Quality Preservation, Inc. March 2013 February 2014 Megan Gehrke, Graduate Student, CSU Monterey Bay Advisor: Sarah Lopez, Central Coast Water Quality Preservation, Inc. Submitted: May 2014

2 Table of Contents Acknowledgements... 3 Executive Summary... 4 Introduction... 5 Project Objectives... 5 Project Approach... 6 Project Outcomes... 6 Sample Completeness... 7 Literature Review... 7 R Code and Outputs... 9 Conclusion References Appendix Samples of Figures and Graphs Produced

3 Acknowledgements This project was supported by Agriculture and Food Research Initiative Competitive Grant no from the USDA National Institute of Food and Agriculture. 3

4 Executive Summary Central Coast Water Quality Preservation, Inc. (CCWQP) is charged with implementing a Cooperative Monitoring Program (CMP), in which monthly water quality samples from 50 sites in agricultural watersheds of the Central Coast are collected. The major goal of the CMP is to show changes in water quality over time, hopefully related to advances in agricultural practices that protect water quality. This internship was focused on the development of code for use within R statistical software to increase automation in the production of figures, tables, and statistical results for the CMP annual report. My specific objectives for this internship were to enhance my knowledge and skills pertaining to agricultural water quality issues and water quality data analysis, as well as to gain further professional experience in this field. I was able to meet my objectives through experience working with, and learning how to visually and quantitatively interpret, a large agricultural water quality dataset with guidance from my advisor at CCWQP. Specifically, these objectives were met through internship tasks, which included verifying dataset completeness, researching water quality statistical trend analyses, and creating R code for the purposes of trend analysis, data characterizations, and data summaries. This experience has helped prepare me for a future career in water quality and environmental analysis, which could include future work with the USDA under the Forest Service or the NRCS. 4

5 Introduction Central Coast Water Quality Preservation, Inc. (CCWQP) is charged with implementing a Cooperative Monitoring Program (CMP), in which monthly water quality samples from 50 sites in agricultural watersheds of the Central Coast are collected. The samples are analyzed for ammonia, chlorophyll a, conductivity, total dissolved solids, nitrate, dissolved oxygen and oxygen saturation, ph, salinity, and turbidity, in addition to in-field measurements of air temperature, water temperature, and flow. Samples are also analyzed for toxicity to invertebrates, fish, and algae 4 times annually. The results are reported to the Regional Water Quality Control Board (RWQCB) on behalf of farmers enrolled in the Conditional Waiver for Irrigated Lands. The major goal of the CMP is to show changes in water quality over time, hopefully related to advances in agricultural practices that protect water quality. Past CMP reports were used by farmers, regulators, conservation agencies, researchers, and environmental organizations. This internship was focused on the development of code for use within R statistical software package (R Core Team 2012) to increase automation in the production of figures, tables, and statistical trend analysis results for the CMP annual report. Project Objectives The main objective for this internship was to develop and implement code for use within R to provide figures and results for the 2013 CMP report, and to provide a means of automation for the production of these figures and results for future annual reports. A personal goal for this internship was to gain professional experience relevant to watershed science and policy. Specifically, through this internship I hoped to enhance my knowledge of the field of agricultural water quality, as well as my data and statistical analysis and reporting skills. Having these skills and experience will greatly improve my ability to pursue my desired career path, especially at agencies such as the USDA. 5

6 Project Approach The internship was structured according to the following tasks: 1. Data preparation and sample completeness check R was used to review 7 annual data files for completeness, as well as to format and collate these files as necessary to support further data analysis. 2. Literature review A review of water quality trend analysis literature was performed. 3. Trend analyses Code was developed for use within R to test for trends within the 7-year dataset. Trend tests used include Seasonal Mann Kendall and Bayesian Point Change tests. Additionally, routines were created to provide summary figures of the Seasonal Mann Kendall trend analysis results. 4. Time series and data characterization Code was developed for use within R to perform data summaries and to create figures for time series plots, box plots, stacked proportional bar plots, and pie charts of regulatory exceedances. These tasks were successfully completed through instruction and guidance from my supervisor at CCWQP. Additionally, a great deal of independent work and research was required to improve my skills in R and my knowledge of environmental statistical analysis techniques. Project Outcomes Through this project I gained extensive experience with analyzing a large water quality dataset using numerous data and statistical analysis techniques. Additionally, I was able to broaden my knowledge of agricultural water quality issues as well as my technical skills in Excel and R. 6

7 Sample Completeness The dataset was tested for completeness using R. The results showed the locations of missing data in the dataset. Sampling event summaries were then reviewed for each missing data value to determine whether or not the value was truly missing from the dataset. Literature Review A literature review of work on water quality trend analysis was performed, with focus on agricultural water quality and the Seasonal Mann Kendall trend test (Table 1). Table 1. Results of a literature review of work in water quality trend analysis, with focus on agricultural water quality and the Seasonal Mann Kendall trend test. Citation Antonopoulos VZ, Papamichail DM, Mitsiou KA Statistical and trend analysis of water quality and quantity data for the Strymon River in Greece. Hydrology and Earth System Sciences, 5(4): Bekele A, McFarland A Regression- based flow adjustment procedures for trend analysis of water quality data. Transactions of the ASAE, 47(4): Summary Monthly water quality and discharge data were analyzed for trends and evaluated for best- fit models. The relationships between concentrations and loads with discharge were also examined, using simple regression. Relation between concentration and discharge was weak, while relation between load and discharge was very strong. Trends were detected using the non- parametric Spearman's criterion. Used non- parametric Kendall's tau trend test (which is suitable for situations in which data are non- normal, missing values, or is censored; non- parametric tests are preferred for data sets of moderate length) to test for monotonic trends of data adjusted for flow in 3 different ways. Kendall's tau is based on the rank order statistic (compares rank rather than actual values). Objective was to determine best flow adjustment method (OLS or LOWESS). The LOWESS method was found to be more appropriate than the OLS method because it was better able to define relationships between constituent concentration and flow. The default f- value of 0.5 was found to be adequate for reducing variability in constituent concentrations due to flow. 7

8 Berryman D, Bobee B, Cluis D, Haemmerli J Non- parametric tests for trend detection in water quality time series. Water Resources Bulletin, 24(3): Bouraoui F, Turpin N, Boerlen P Trend analysis of nutrient concentrations and loads in surface water in an intensively fertilized watershed. J. Environmental Quality, 28(6): Crain AS, Martin GR Trends in surface water quality at selected ambient- monitoring network stations in Kentucky, Scientific Investigations Report , USDOI and USGS. Hirsch RM, Slack JR, Smith RA Techniques of trend analysis for monthly water quality data. Water Resources Research, 18(1): McLeod AI, Hipel KW, Bodo BA Trend analysis methodology for water quality time series. Environmetrics, 2(2): Methods for choosing trend detection tests based on the identification of sources of serial dependence in the time series are discussed. The Spearman and Kendall tests for monotonic trends and the Mann- Whitney test for the detection of steps are identified as powerful non- parametric tests. Analyzed nutrient concentrations and loads at the surface water outlet of a heavily fertilized watershed. A non- parametric statistical analysis (seasonal Mann- Kendall) was performed on mean monthly and mean annual data, which detected no trend. Next, data were compared from the same month of each year and both decreasing and increasing trends were detected for certain constituents. The Mann- Kendall test was chosen based upon reviews of trend analysis of water quality by Walker (1994), Hirsch et al. (1991), and Berryman et al. (1988). Used the S- Plus statistical software program (designed to detect monotonic trends) to perform trend analyses on water quality data. Tests used were the Seasonal Kendall non- parametric test and the Tobit- regression parametric test. One of these tests was selected for each constituent. Flow- adjustment methods provided with the S- Plus software were used to eliminate effects of flow on WQ variability. Presents techniques for analysis of monotonic water quality trends that account for non- normal distributions, seasonality, flow- relatedness, censored values, and serial correlation. The Seasonal Kendall test, the Kendall slope estimator (an estimator of trend magnitude for skewed data), and flow- adjusted constituent concentrations coupled with the Seasonal Kendall test were explored. Concluded that the methods explored were useful for long time series, as it is useful to have a set of objective procedures that are powerful over a wide range of situations for identifying trends. Developed a general trend analysis methodology for water quality time series. Designed for use with non- normal, positively skewed data, with seasonal variation and interdependence of water quality variables and flow. The approach is divided into two categories: graphical studies and trend tests. Trend tests included Mann- Kendall, Kruskal- Wallis, and Spearman's partial- rank correlation. It was found that (1) the Spearman test has high power for WQ trend testing with seasonality, (2) flow- adjusted WQ data can eliminate sampling bias, and (3) it is important to test for seasonality before applying a test such as the seasonal Mann- Kendall test (which is less powerful when seasonality is not present). 8

9 Renwick WH, Vanni MJ, Zhang Q, Patton J Water quality trends and changing agricultural practices in a Midwest U.S. watershed, J. Environmental Quality, 37: Yu Y, Zou S, Whittemore D Non- parametric trend analysis of water quality data of rivers in Kansas. J. Hydrology, 150: Analyzed changes in farm management practices that were likely to have an effect on water quality. Used an auto- regressive moving average model to include effects of discharge and season on constituent concentrations. Also used LOWESS plots and analyses of changes in relation between discharge and concentration. Four different non- parametric trend detection methods (Mann- Kendall, Seasonal Kendall, Sen's T test, Van Belle and Hughes Chi- square test) were used for a 9- year water quality dataset. The different methods were compared and were found to have practically equal power for datasets of at least 9 years in length. Lays out the steps for preliminary analyses (dist, dependence, seasonality, flow relatedness tests). R Code and Outputs R code was developed to automate the creation of boxplots, time-series plots, precipitation and flow plots, turbidity and flow plots, stacked barplots for toxicity results, pie charts of regulatory exceedances, and summary statistics tables (see Appendix for samples of these products). Additionally, code was developed to analyze water quality trends using the Seasonal Mann Kendall test from the Kendall package (A.I. McLeod 2011). For sampling sites with sufficient data, each analyte was tested for long-term monotonic trends using these routines. The data were tested for trends across both the full dataset as well as the dataset divided into wet and dry months. The dataset was divided by season due to potential differences in trends detected as a result of the effects of seasonality on agricultural water quality data. Meaning that irrigation during the dry season and precipitation during the wet season may have differing effects on water quality. Another R routine was developed to create summary figures of the Seasonal Mann Kendall test results for each hydrologic unit. 9

10 The analyses, figures, and tables discussed above were produced for inclusion in the 2013 CMP report. The R routines themselves will be used for reproduction of these items in future annual reports. Conclusion Through this internship I was able to gain and develop valuable knowledge, skills, and experience in my field of interest. I learned a great deal about water quality within agricultural watersheds on the Central Coast and gained the skills necessary to evaluate a large water quality dataset in terms of general data characterization, statistical trend analysis, and summary statistics. Through all of this, I gained further expertise in the use of R and am more confident in my technical skills and knowledge. These skills are extremely valuable to my professional development and to the enhancement of future career opportunities. 10

11 References A.I. McLeod (2011). Kendall: Kendall rank correlation and Mann-Kendall trend test. R package version R Core Team (2012). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN , URL 11

12 Appendix Samples of Figures and Graphs Produced A. Precipitation and flow time series plot. B. Toxicity stacked proportional bar plot. 12

13 C. Turbidity and flow plots. 13

14 D. Pie charts showing proportions of regulatory exceedances. 14

15 E. Trend analysis summary 15

16 F. Analyte boxplots. 16

17 G. Analyte time series plots. 17