Evaluating Laboratory Data. Tom Frick Environmental Assessment Section Bureau of Laboratories

Transcription

1 Evaluating Laboratory Data Tom Frick Environmental Assessment Section Bureau of Laboratories

2 Overview of Presentation Lab operations review QA requirements MDLs/ PQLs

3 Why do we sample? Protect physical, chemical, and biological integrity (Clean Water Act) Water quality standards, FAC Permit compliance Interpret biological community results

4 How Do You Measure an Collect sample Prepare sample Analyte? Calibrate equipment Measure analyte Compare analyte measurement to known standard Run Quality Control samples

5 Proper location Correct equipment or bottles Prevent contamination Proper preservation

6 Sample log-in & tracking (1000 s per day) Preparation steps Holding times Pairing sample with result and QC elements

7 Calibration & maintenance Blank contamination Spike recovery Detection limits Statistically, QC failures occur routinely

8 Bottom Line on Lab Measurements Literally 1000 s of different methods to measure various types of analytes. Each method has specific quality control requirements to assure valid measurements.

9 Required Laboratory Quality Control Activities

10 QA Requirements for Laboratories, Rule NELAC Certification Performance Test Samples Mandatory Quality Control Ethics and training Reporting Location, methods, QA elements (extensive list) Audits Lab, field, document review, corrective action Data validation Completeness, data integrity & usability

11 Performance Test Samples NELAC required Blind sample, lab does not know concentration of analyte. Lab runs PT sample using routine procedures. Must achieve acceptable result in 2 of 3 previous PTs. Unknown Result must be within range

12 Instrument Response Calibration Establishing a relationship between instrument response and known concentrations. Regression line R 2 should be or better Concentration

13 Continuing Calibration Verification Instrument Response Calibration must be rechecked every 20 samples. Real value = 1.0 ug/l Usually required to be + or 5 to 15%. Concentration 0.99 ug/l

14 Lab Blank Analyte-free water processed as routine sample. Result evaluated to determine sources of internal lab contamination. Analyte-free water Result evaluated, should be U

15 Accuracy and Precision Accuracy: The ability to measure the true value. True value is NIST standard Precision: Consistency of Measurements.

16 Lab Control Sample (LCS) A known concentration of analyte added Sample processed routinely. Determine if amount detected matches amount added (+ or acceptable %). Add known Amount (spike) Determine % recovery Acceptance criteria is usually 80% to 120%

17 Lab Control Sample Duplicate Add known amount Split Compare results, determine precision

18 Why Quality Control Samples are Important Good precision, Good accuracy Good precision, Poor accuracy Poor precision, Good accuracy Poor precision, Poor accuracy

19 Accuracy The true or standard value is based on the central tendency of measurements involving multiple high quality testing laboratories, using exacting NIST procedures. Other laboratories purchase standards for calibration and QC. 1.0 ug/l

20 Precision The relative agreement in values from repeated measures of the same sample (relative standard deviation) during routine testing runs (measurement repeatability). Tight, acceptable precision Noisy, unacceptable precision

21 To Ensure the Correct Decision: DEP must verify that data are useable. Consistent with Data Quality Objectives and program requirements. We have Statutory Authority to reject data!!!

22 Data Quality & DEP If real result is higher than reported Environment is not protected If real result is lower than reported Costly, unneeded treatment requirements In either case: Legal challenges Adverse publicity Ultimately Results in More Staff Time and $$

23 Detection and Quantitation Limit Concepts

24 Definitions Minimum Detection Limit: The minimum concentration that can be measured with 99% confidence that the analyte concentration is greater than zero. Practical Quantitation Limit: The lowest level of measurement that can be reliably achieved during routine laboratory operating conditions within specified limits of precision and accuracy.

25 Radio Tower MDL/PQL Radio Reception Analogy Whole sentences, complete comprehension > PQL A few words, but no meaning = MDL Static Only: < MDL I, M U,T

26 Simplified MDL Study Take analyte free water or low level spikes and place in 7 containers Process as normal samples All prep steps, reagents, instrument calibration, etc. are routine Generate results Calculate standard deviation of results MDL is 3 times this SD This is the 99% confidence limit that you can distinguish signal from noise

27 Theoretical Probability Distributions Analytical Noise 0.4 Noise Distribution MDL set to exclude 99% of the Noise Distribution 0.3 Frequency 0.2 MDL Standard Deviations

28 MDL for More Sensitive Method MDL for Less Sensitive Method

29 Method 1631 MDL by 40 CFR Method Mercury ( n g /L ) January - March

30 MDL/PQL Relationship Concentration Present Absent 0 M I U T Region of known precision and accuracy PQL (4 x MDL) = 4 x 0.3 = 1.2 ug/l Region of high uncertainty for quantitation (greater certainty as PQL is approached) MDL (3 x SD) = 3 x.1 = 0.3 ug/l Use MDL as an Estimate of a Lab s Ability to Detect (not quantitate) at the MDL concentration

31 POSSIBLE DECISIONS Detected TRUE SITUATION Analyte Present CORRECT DECISION Analyte Not Present INCORRECT DECISION FALSE POSITIVE (Type I error) Not Detected INCORRECT DECISION FALSE NEGATIVE (Type II error) CORRECT DECISION

32 Detection Limit Values U Values > Permit Limits? Method may not be appropriate; Use more sensitive method (if available) Matrix interference or dilution may contribute to an elevated detection limit I or M Values Near Permit Limits Compliance Issue? May Require Additional Testing to Resolve

33 FAC PQL, MDL and Compliance Refers to checklist of DEP approved MDLs Results > PQL are reported as measured Results < PQL and > MDL shall be deemed as = to the MDL Results < MDL shall be reported as non-detects ( U )

34 Co Co ncentrati ncentrati on on Interpreting FAC: Examples Result Permit Limit PQL Permit Limit Result PQL MDL MDL Non- Compliance Compliance

35 Concentr Concentr ation ation Interpreting FAC: Examples PQL Result Permit Limit MDL Compliance (Results < PQL are deemed = MDL) PQL Permit Limit Result MDL Compliance

36 Concentr Concentr ation ation Interpreting FAC: Examples PQL Result MDL Permit Limit PQL Result is Non-detect MDL Permit Limit Non-Compliance Compliance

37 Tom Frick (850) Quality of Science enewsletter