Evaluating Laboratory Data Tom Frick Environmental Assessment Section Bureau of Laboratories
Overview of Presentation Lab operations review QA requirements MDLs/ PQLs
Why do we sample? Protect physical, chemical, and biological integrity (Clean Water Act) Water quality standards, 62-302 FAC Permit compliance Interpret biological community results
How Do You Measure an Collect sample Prepare sample Analyte? Calibrate equipment Measure analyte Compare analyte measurement to known standard Run Quality Control samples
Proper location Correct equipment or bottles Prevent contamination Proper preservation
Sample log-in & tracking (1000 s per day) Preparation steps Holding times Pairing sample with result and QC elements
Calibration & maintenance Blank contamination Spike recovery Detection limits Statistically, QC failures occur routinely
Bottom Line on Lab Measurements Literally 1000 s of different methods to measure various types of analytes. Each method has specific quality control requirements to assure valid measurements.
Required Laboratory Quality Control Activities
QA Requirements for Laboratories, Rule 62-160 NELAC Certification Performance Test Samples Mandatory Quality Control Ethics and training Reporting Location, methods, QA elements (extensive list) Audits Lab, field, document review, corrective action Data validation Completeness, data integrity & usability
Performance Test Samples NELAC required Blind sample, lab does not know concentration of analyte. Lab runs PT sample using routine procedures. Must achieve acceptable result in 2 of 3 previous PTs. Unknown Result must be within range
Instrument Response Calibration Establishing a relationship between instrument response and known concentrations. Regression line R 2 should be 0.995 or better Concentration
Continuing Calibration Verification Instrument Response Calibration must be rechecked every 20 samples. Real value = 1.0 ug/l Usually required to be + or 5 to 15%. Concentration 0.99 ug/l
Lab Blank Analyte-free water processed as routine sample. Result evaluated to determine sources of internal lab contamination. Analyte-free water Result evaluated, should be U
Accuracy and Precision Accuracy: The ability to measure the true value. True value is NIST standard Precision: Consistency of Measurements.
Lab Control Sample (LCS) A known concentration of analyte added Sample processed routinely. Determine if amount detected matches amount added (+ or acceptable %). Add known Amount (spike) Determine % recovery Acceptance criteria is usually 80% to 120%
Lab Control Sample Duplicate Add known amount Split Compare results, determine precision
Why Quality Control Samples are Important Good precision, Good accuracy Good precision, Poor accuracy Poor precision, Good accuracy Poor precision, Poor accuracy
Accuracy The true or standard value is based on the central tendency of measurements involving multiple high quality testing laboratories, using exacting NIST procedures. Other laboratories purchase standards for calibration and QC. 1.0 ug/l
Precision The relative agreement in values from repeated measures of the same sample (relative standard deviation) during routine testing runs (measurement repeatability). Tight, acceptable precision Noisy, unacceptable precision
To Ensure the Correct Decision: DEP must verify that data are useable. Consistent with Data Quality Objectives and program requirements. We have Statutory Authority to reject data!!!
Data Quality & DEP If real result is higher than reported Environment is not protected If real result is lower than reported Costly, unneeded treatment requirements In either case: Legal challenges Adverse publicity Ultimately Results in More Staff Time and $$
Detection and Quantitation Limit Concepts
Definitions Minimum Detection Limit: The minimum concentration that can be measured with 99% confidence that the analyte concentration is greater than zero. Practical Quantitation Limit: The lowest level of measurement that can be reliably achieved during routine laboratory operating conditions within specified limits of precision and accuracy.
Radio Tower MDL/PQL Radio Reception Analogy Whole sentences, complete comprehension > PQL A few words, but no meaning = MDL Static Only: < MDL I, M U,T
Simplified MDL Study Take analyte free water or low level spikes and place in 7 containers Process as normal samples All prep steps, reagents, instrument calibration, etc. are routine Generate results Calculate standard deviation of results MDL is 3 times this SD This is the 99% confidence limit that you can distinguish signal from noise
Theoretical Probability Distributions Analytical Noise 0.4 Noise Distribution MDL set to exclude 99% of the Noise Distribution 0.3 Frequency 0.2 MDL 0.1 0.0-2 0 2 4 6 8 Standard Deviations
MDL for More Sensitive Method MDL for Less Sensitive Method
Method 1631 MDL by 40 CFR Method 0.04 0.03 0.02 Mercury ( n g /L ) 0.01 0-0.01-0.02-0.03-0.04 January - March
MDL/PQL Relationship Concentration Present Absent 0 M I U T Region of known precision and accuracy PQL (4 x MDL) = 4 x 0.3 = 1.2 ug/l Region of high uncertainty for quantitation (greater certainty as PQL is approached) MDL (3 x SD) = 3 x.1 = 0.3 ug/l Use MDL as an Estimate of a Lab s Ability to Detect (not quantitate) at the MDL concentration
POSSIBLE DECISIONS Detected TRUE SITUATION Analyte Present CORRECT DECISION Analyte Not Present INCORRECT DECISION FALSE POSITIVE (Type I error) Not Detected INCORRECT DECISION FALSE NEGATIVE (Type II error) CORRECT DECISION
Detection Limit Values U Values > Permit Limits? Method may not be appropriate; Use more sensitive method (if available) Matrix interference or dilution may contribute to an elevated detection limit I or M Values Near Permit Limits Compliance Issue? May Require Additional Testing to Resolve
62-4.246 FAC PQL, MDL and Compliance Refers to checklist of DEP approved MDLs Results > PQL are reported as measured Results < PQL and > MDL shall be deemed as = to the MDL Results < MDL shall be reported as non-detects ( U )
Co Co ncentrati ncentrati on on Interpreting 62-4.246 FAC: Examples Result Permit Limit PQL Permit Limit Result PQL MDL MDL Non- Compliance Compliance
Concentr Concentr ation ation Interpreting 62-4.246 FAC: Examples PQL Result Permit Limit MDL Compliance (Results < PQL are deemed = MDL) PQL Permit Limit Result MDL Compliance
Concentr Concentr ation ation Interpreting 62-4.246 FAC: Examples PQL Result MDL Permit Limit PQL Result is Non-detect MDL Permit Limit Non-Compliance Compliance
Tom Frick (850) 245-8065 thomas.frick@dep.state.fl.us Quality of Science enewsletter http://www.dep.state.fl.us/labs/training/listserve.htm