AUDIT PROTOCOL FOR THE VICTORIAN WATER QUALITY MONITORING NETWORK

Transcription

1 AUDIT PROTOCOL FOR THE VICTORIAN WATER QUALITY MONITORING NETWORK ENVIRONMENT PROTECTION AUTHORITY June 1999

2 AUDIT PROTOCOL FOR THE VICTORIAN WATER QUALITY MONITORING NETWORK, June 1999 David Robinson Freshwater Sciences Environment Protection Authority 40 City Road Southbank Victoria 3006 Australia Printed on recycled paper State of Victoria, December 1999 ISBN: X

3 TABLE OF CONTENTS 1. RATIONALE 1 2. OBJECTIVES Primary Objectives Specific Objectives 1 3. OBLIGATIONS OF PARTIES Contractors Environment Protection Authority (EPA) Department of Natural resources and Environment (DNRE) Environmental Water Quality Monitoring Committee (EWQMC) 2 4. GENERAL REQUIREMENTS OF CONTRACTORS Laboratory Contractors Field Contractors Criteria for selecting contractors Audit management 3 5. FIELD SAMPLING Procedures Training Assessment Use of Blanks, Controls and Replicates Field Blanks Transport Blanks Control samples Field Replicates Field instruments 5 6. LABORATORY ANALYSIS Methodology Replication Data management Performance assessment Independent analysis of replicates Blind Trials 7 7. REPORTING 8 i

4 AUDIT PROTOCOL FOR THE VICTORIAN WATER QUALITY MONITORING NETWORK 1. RATIONALE The management and conduct of the Victorian Water Quality Monitoring Network (VWQMN) is undergoing change. With the move towards competitive tendering and regionalisation of the state into five discrete regions, the water quantity and water quality contracts could be let as combined or separate contracts. These changes have the potential to affect the continuity of the data set. To minimise possible effects due to changing contractors it will be necessary to standardise procedures as much as possible and to have an on-going program of audit in place that will rigorously assess the quality of work of the VWQMN s contractors. This document contains an outline of the principles and requirements of the audit protocol for the VWQMN. 2. OBJECTIVES 2.1 Primary Objectives Through the development of an audit protocol for the network the Environmental Water Quality Monitoring Committee (EWQMC) seeks to ensure: Continuing and improving high standard of data collection for the network. That quality control procedures are consistent with current world best practice for such networks. Continuity of data and data quality, irrespective of contract arrangements for sample collection and analysis. Network clients are assured of the quality of the data, through regular reporting of the results of audit activities. 2.2 Specific Objectives The program for quality assurance/ quality control (QA/QC) and auditing developed for the network has the following specific objectives: Ensure that contractors are routinely carrying out the correct sampling procedures and analytical techniques. Detect any systematic bias or errors in the analytical and field determinations. Provide diagnostic information on the sources of contamination and the effects of interferences. Quantify the errors involved in measuring each indicator. 1

5 3. OBLIGATIONS OF PARTIES 3.1 Contractors Contractors are required to carry out all of their contractual obligations with respect to external and internal QA/QC programs, and to cooperate with and assist audit teams in the conduct and reporting of audits. Contractors are expected to participate in training activities and workshops as deemed necessary. 3.2 Environment Protection Authority (EPA) EPA s obligations are to manage all auditing activities for the VWQMN, and to report to the EWQMC and DNRE on auditing outcomes by December of any given year. EPA will meet the costs of the blind trials and the field auditing activities. 3.3 Department of Natural Resources and Environment (DNRE) DNRE s obligations are to establish contractual arrangements in accordance with the audit protocol. DNRE will manage and monitor all aspects of execution of the contracts. If a contractor s performance is found to be unsatisfactory, DNRE will take action to rectify the poor performance including termination of a contract and engagement of new contractors if deemed necessary. 3.4 Environmental Water Quality Monitoring Committee (EWQMC) The Committee s obligations are to oversee the conduct of the network and maintain its scientific integrity; review proposals for change to the network; and make recommendations on changes to the network. 4. GENERAL REQUIREMENTS OF CONTRACTORS 4.1 Laboratory Contractors Laboratories must be NATA (National Association of Testing Authorities, Australia) accredited for all analyses required for network monitoring. All contracted laboratories will be required to participate in nationally recognised trials to assess data quality and consistency, for example the National Low Level Nutrient Collaborative Trials (NLLNCT) or other future equivalents of relevance. The trials that contractors are required to participate in will be specified in contracts and this requirement will be modified as the need arises. 4.2 Field Contractors Contractors for fieldwork should maintain, as a minimum, ISO 9000 certification or its equivalent. 2

6 4.3 Criteria for selecting contractors Contractors need to be selected with great care. While cost is an important consideration, it should not be the exclusive criterion used to select a contractor. When selecting a contractor the emphasis should be on obtaining the best value for the State Network. To ensure best value, the EWQMC recommend the following criteria be used when selecting a contractor: Qualifications of the company Experience in environmental water quality monitoring. Past performance (contractor to provide evidence). Facilities and resources. Quality control and assurance systems. Accreditation, laboratories must be NATA accredited for relevant analyses. Ability to measure in the ranges required and meet the required detection limits and levels of accuracy. Flexibility to accommodate new arrangements and requirements. Participation in inter-laboratory trials and performance audits. Data management facilities and ability to transfer data electronically in the desired format and time frames. Cost structures, total cost, indication of hourly rates and charges for additional analyses. 4.4 Audit Management EPA will carry out auditing of the network and will report the results of their auditing activities, on an annual basis, to the EWQMC. Changes to auditing procedures will be subject to the approval of the EWQMC. Should EPA, in the conduct of their auditing, detect any serious shortcomings in the performance of contractors, EPA shall inform the DNRE contract managers immediately. 5. FIELD SAMPLING 5.1 Procedures All sampling and analysis is to be carried out in accordance with the most recent revision of EPA s Guidelines to Sampling and Analysis of Waters and Wastewaters: Publication No Contractors for field sampling are to produce the following procedures manuals: 1) Concise field manual containing only the essential information for field sampling and measurement, and 2) Office manual containing detail about methodologies and equipment used. The purpose of the manuals is to ensure that field data is collected in a manner that maintains high quality, reliability and consistency by: Providing field staff with a concise and readily useable guide to field procedures. 3

7 Establishing that contractors have correctly interpreted the general guidance given in the EPA guidelines and the requirements of their contract. 5.2 Training New contractors will be required to participate in a once-off training/induction session, conducted by EPA or a suitably qualified contractor, to establish the correct sampling procedures at the start. On-going training and accreditation is the responsibility of the contractor who must maintain appropriate records and documentation of training and accreditation of staff. This documentation is to be made available to auditors upon request. All staff used for sampling must have appropriate training and accreditation. Contractors must provide EPA with a report on training activities for each calender year, by December of that year. 5.3 Assessment EPA will audit sampling teams in the field. This type of assessment is needed to demonstrate that contractors are putting into practice the requirements of the manuals and protocols for the network. The EPA assessment will aim to evaluate each sampling team at least once during a given year. For a field audit the assessment team will: Assess sampling technique and procedures, Check that the right equipment and instrumentation is being used, Check the calibration and performance of the instruments and the proficiency of the samplers in using the instruments, by using standard solutions and taking parallel measurements. Sampling teams may be audited without notice. Six monthly sampling schedules will need to be supplied to EPA at least one month in advance to enable EPA to plan auditing activities. EPA must be notified of any changes to the sampling schedule. 5.4 Use of Blanks, Controls and Replicates Blanks, controls and replicates are basic quality assurance tools and should be incorporated into standard network procedures Field Blanks This blank is used to estimate possible contamination of a sample during the collection procedure. At least one field blank should be taken per sampling team per trip per collection apparatus. Once knowledge of the homogeneity of sampling 4

8 conditions is established it may be possible to reduce the frequency of collection of field blanks Transport Blanks This blank is used to estimate the amount of contamination introduced during the transport and storage of samples up to the time of analysis. At least one set of sampling bottles should be allowed per sampling team per trip Control samples Samples with known amounts of indicators are used to check for loss or changes to an indicator during transport and storage. At least one control should be taken per sampling team per trip across the range of indicators. Once knowledge of the stability of sampling conditions is established it may be possible to reduce the number of controls used Field Replicates Replicates are needed to quantify the random error involved in sampling. Approximately 1% of samples should be replicated. (5 replicates). 5.5 Field instruments On-going maintenance and calibration of all field instruments is the responsibility of the field contractors who should carry out maintenance and calibration in accordance with the manufacturers specifications and general guidance in EPA publication No. 441 (A guide to the Sampling and Analysis of Water and Waste Water). Maintenance schedules for each type of field meter are to be included in the procedures manuals produced by contractors (see 4.1). Contractors will be required to maintain documentation on the maintenance and calibration of each instrument or measurement device. This documentation is to be made available to auditors upon request. Independent auditors will check all instruments on a regular basis to assess the operational condition of each instrument. Any instruments found to be nonoperational will be required to be taken out of service by the contractor. 6 LABORATORY ANALYSIS 6.1 Methodology Contractors are expected to carry out all analyses in accordance with NATA requirements for quality control and assurance. 5

9 All methods used for measuring network indicators are to be documented including procedures for dealing with possible interferences and variable sample matrices. All laboratory quality control procedures are to be documented, and all such documentation is to be made available to the auditors upon request. 6.2 Replication Laboratory analyses are to be replicated to estimate the error involved in applying analytical techniques. Laboratories should aim for about 1% of samples to be replicated (3-5 replicates). Replicates are to be spread over a range of sample matrix types to obtain estimates of error under various conditions. These replicates will be additional to normal QA/QC samples used during laboratory analysis and additional to replication of samples collected in the field. 6.3 Data management In addition to the normal checking for keystroke errors and unusual values, all samples are to be checked for internal consistency (eg. ion balance, multivariate outlier check). This last type of check should be amenable to automation and will need to be done quickly such that there will be sufficient time for reanalysis of samples if deemed necessary. After data has been checked and verified, contractors are to submit the data to the Data Warehouse in the required format. Contractors will be required to maintain all field and analytical data in their own systems for a minimum of six years to enable verification of samples if necessary. 6.4 Performance assessment Laboratory performance will be independently assessed by two means: Analysis of replicates by an independent laboratory, and Blind trials conducted by EPA. These performance assessments will be directed to detecting systematic bias and excessive levels of variability. They will also provide useful diagnostic information to the contractors on the sources of error that will help them improve their work practices, and assist those using and interpreting the data Independent analysis of replicates Replicates taken in the field will be sent to an independent laboratory for analysis. The taking of replicates for independent analysis will be the responsibility of the field contractor. The regular contracting laboratory is not to know which samples have 6

10 been replicated for independent assessment. The replicated samples used for independent assessment could be a subset of those used for measuring the errors involved in sampling (see 4.4.4). Results from these independent analyses will be returned to the collection contractor who will forward them on to EPA Blind trials EPA will conduct blind trials for proficiency testing. All laboratories used for the network will be tested in this way and consistent procedures will be applied to all laboratories. The trials will employ an ANOVA design and a reference laboratory to detect systematic bias and excessive variability. Samples used for the trials will be prepared from homogeneous batches of test solutions and will be submitted for analysis by EPA or a third party acting on behalf of EPA. Maintenance of chain of custody up to the delivery of samples to each laboratory is essential. Batches of test solutions will be divided up into equal portions and random selections from these portions will be used to make up replicated samples to be sent out to each laboratory being tested. The blind trials will incorporate: Natural samples from a range of waters (high and low concentration). Standard reference samples covering a range of concentrations including levels close to levels of detection. Blanks and control samples. The trials will need to incorporate sufficient replication to enable accurate calculation of confidence limits and statistical assessment of results. Errors arising from both sampling and application of analytical techniques within the laboratory will be quantified (see and 6.3). Statistical assessment will employ analysis of variance techniques to identify unsatisfactory systematic bias or variability. The statistical techniques used by NATA for proficiency testing and in the NLLNCT are not suitable because of the small number of laboratories that will be participating in the trials. EPA will determine how much deviation is allowed (the effect size); including criteria to be met before inadequate performance is claimed.). In addition to the blind inter-laboratory trials, replicate samples will be routinely taken in the field by sampling contractors to measure the random error inherent in sampling. Some of these replicates will also be sent to a laboratory other than the normal contracting laboratory for an independent comparison. The comparison laboratory will be of a high standard and will need to know that they are being used as a reference. The regular contracting laboratory will be unaware which samples are being assessed by the reference laboratory. Individual trials will focus on groups of indicators (not all in each trial) and will aim for a frequency of testing of 2 times per year. The focus indicator group may change each year. 7

11 Laboratories will know in advance the general form of the trials but will not know when the trials are being conducted or which groups of indicators are the subject of the trials in any given year. 7. REPORTING EPA will prepare a report on the auditing activities for a given calender year for the VWQMN annual report. All of the information required from contractors for audit reporting is to be submitted by December of any given year. The following timetable for the various components of the audit reporting will apply. Blanks, controls and replicates submitted to EPA monthly. Laboratory procedures submitted to EPA yearly by December of any given year. Field Training timetable and data Submitted to EPA by December of any given year. National Low Level Nutrient Collaborative Trials annually as they become available. Laboratory Proficiency testing to be completed by December of any given year. EPA will advise contractors of the appropriate reporting formats for all components of the QA/QC program. All correspondence concerning the audit protocol is to be addressed to one of the following: EPA Audit Project Manager P.O. Box 4395QQ MELBOURNE VIC DNRE Project Manager: VWQMN Project Manager: VWQMN Department of Natural Resources and Environment 1/240 Victoria Parade EAST MELBOURNE VIC 3002 EWQMC C/- Project Manager: VWQMN Department of Natural Resources and Environment 1/240 Victoria Parade EAST MELBOURNE VIC