Method validation and quality control procedures

Transcription

1 Method validation and quality control procedures Department of Food Chemistry and Analysis, ICT Prague Vladimír Kocourek Prague, 2012

2 WHAT IS VALIDATION? Validation is a process, within which the method is demonstrated to be suitable for its purpose. It documents methods performance! During validation process, methods Performance characteristics are estimated. Validation documents, that the methods performance characteristics are capable of producing results in line with the needs of the analytical problem. Is it possible to detect pesticide residues at regulation levels using the method? Is it possible to correctly quantify the amount of residues in apple/orange/? Validation procedure (protocol) is related to a particular analyte and matrix Applicability

3 Analytical method for decision making Defining the task Selection of method Sample handling Analytical procedure measuring Calibration a Calculation Data processing Interpretation, Presentation

4 VALIDATION VALIDATION PARAMETERS PRECISION TRUENESS ACCURACY RANGE & LINEARITY LIMIT OF DETECTION & LIMIT OF QUANTIFICATION SPECIFITY & SELECTIVITY RUGGEDNESS

5 TRUENESS AND PRECISION = ACCURACY ERRORS OF MEASUREMENT What is included in result value (X)? X = μ + ε + Σδ RELATED TO TRUENESS RELATED TO PRECISION X mean result of repeated measurement µ true value real content of analyte in sample ε systematic error always the same value and sign (+/-) Σδ sum of random errors variable value and sign (+/-) Error should not be confused with a mistake!!!

6 TRUENESS AND PRECISION = ACCURACY ERRORS OF MEASUREMENT What is included in result value (X)? X = μ + ε + Σ δ True value is an idealized concept and true value cannot be known exactly! Hence the REFERENCE VALUE represents a true value in routine practice Reference value usually provided with reference to: Certified reference material Reference measurement procedure Known amount of analyte added into the sample (spike)

7 TRUENESS TRUENESS is closeness of agreement between the mean of an N number of replicate measured values and a REFERENCE (TRUE) value. Trueness is inversely related to systematic error: The lower the systematic error, the higher the trueness Estimate of a systematic error: bias In analytical chemistry: RECOVERY Correction of the result can be carried out using the recovery => compensation for an estimated systematic effect

8 Traceability to reference material: trueness Assessment of trueness using CRM SAMPLE CRM (matrix) RESULT Compare with certified value t- test : t exp < t crit (P, n-1)

9 Certified Reference Material (CRM, SRM)

10 TRUENESS HOW TO ESTIMATE TRUENESS (RECOVERY) (Certified) Reference materials are available CRM Y ± y Sample 1 Sample 2 Sample 3 Sample 4 Sample 5 n 6 ANALYTICAL PROCESS Value 1 Value 2 Value 3 Value 4 Value 5 COMPARISON STATISTICAL TESTING MEAN VALUE X ± x (C)RMs at concentration levels close to expected analyte levels should be used

11 TRUENESS HOW TO ESTIMATE TRUENESS (RECOVERY) (Certified) Reference materials are not available THE MOST COMMON CASE A) BLANK SAMPLE IS AVAILABLE SAMPLE BLANK Sample 1 Sample 2 Sample 3 Sample 4 Sample 5 n 6 INCUBATION ANALYTICAL PROCESS Value 1 Value 2 Value 3 Value 4 Value 5 Addition of known amount of an analyte to each replicate (SPIKE) MEAN VALUE X ± x Blank samples should be checked, multiple spiking levels should be used After spike addition sample should be incubated (analyte incorporation into the matrix)

12 Certified Reference Material (CRM, SRM)

13 TRUENESS HOW TO ESTIMATE TRUENESS (RECOVERY) (Certified) Reference materials are not available A) BLANK SAMPLE IS AVAILABLE RECOVERY (%) MEAN VALUE =. 100 ADDED AMOUNT Recovery values can be both below or above 100% Recoveries between 80 and 120 are usually acceptable.

14 TRUENESS HOW TO ESTIMATE TRUENESS (RECOVERY) (Certified) Reference materials are not available B) BLANK SAMPLE IS NOT AVAILABLE Sample 1 Sample 2 ANALYTICAL PROCESS Value 1 Value 2 Sample 3 Value 3 SAMPLE Sample 4 Sample 5 Value 4 Value 5 n 6 MEAN VALUE Sample 1 Value 1 SAMPLE Sample 2 Sample 3 Sample 4 Sample 5 n 6 INCUBATION Value 2 Value 3 Value 4 Value 5 MEAN VALUE SPIKE

15 TRUENESS Codex Alimentarius: numerical values for the criteria

16 PRECISION PRECISION is related to RANDOM ERRORS Component of measurement error that - in replicate measurements - varies in an unpredictable manner Random error = error - systematic error Correction of random error can not be done Some sources of random errors: Methods (procedure, calibration,...) Laboratory (facility, environment) Equipment and materials / reagents / calibrants Personnel Time

17 PRECISION PRECISION represents random errors of a set of replicate measurements PRECISION is calculated as a (relative) standard deviation of replicate measurements σ x Less precision is reflected by a larger standard deviation Precision depends critically on the conditions! REPEATABILITY and REPRODUCIBILITY conditions are particular sets of extreme conditions....nothing to do with true or reference value!

18 PRECISION REPEATABILITY AND REPRODUCIBILITY Repeatability: a set of conditions that includes the same measurement, procedure, operators, same measuring system, operating conditions and location, and replicate measurements on the same or similar objects over a short period of time Reproducibility: a set of conditions that includes different locations, operators, measuring systems, or even methods on the same or similar objects. Intermediate precision (intra-laboratory reproducibility): the same laboratory, method, procedure but within an extended period of time - may include new calibrations, calibrants, operators, measuring systems, etc.

19 PRECISION INCREASING NUMBER OF CONSIDERED RANDOM ERROR SOURCES REPEATABILITY INTRA-LABORATORY REPEATABILITY REPRODUCIBILITY SAMPLE: SAME OPERATOR: SAME INSTRUMENT: SAME TIME PERIOD: SHORT CALIBRATION: SAME LAB: SAME SAMPLE: SAME OPERATOR: DIFFERENT INSTRUMENT: SAME / DIFF. TIME PERIOD: LONG CALIBRATION: DIFFERENT LAB: SAME SAMPLE: SAME OPERATOR: DIFFERENT INSTRUMENT: DIFFERENT TIME PERIOD: LONG CALIBRATION: DIFFERENT LAB: DIFFERENT Precision value is related to a certain analyte and concentration level

20 PRECISION VARIABILITY OF RESULTS: 2 DIFFERENT OPERATORS LU Standard FLD1 A, Ex=248, Em=374, TT (C:\DATAHP~1\FLD154-4\DATA02\MJKOLONY MePyr B[a]A Integration 1 Integration 2 FLD1 A, Ex=248, Em=374, TT (C:\DATAHP~1\FLD154-4\DATA02\MT021016\ D) LU MePyr BaA Area: Area: Area: LU FLD1 A, Ex=248, Em=374, TT (C:\DATAHP~1\FLD154-4\DATA02\MT021016\ D) MePyr Area: Area: m m m Difference between results 42 % (concentration level ppb)

21 PRECISION VARIABILITY OF RESULTS: 2 DIFFERENT MEASUREMENT SYSTEMS % Same sample, conditions, operator, Two different chromatographs of the same type

22 PRECISION INCREASING NUMBER OF CONSIDERED RANDOM ERROR SOURCES REPEATABILITY INTRA-LABORATORY REPEATABILITY REPRODUCIBILITY Repeated analyses of a sample containing analyte(s) at: level close to expected concentration in analyzed matrix level close to regulatory limit low level close to limit of quantification of the method Appropriate number of repeats: 8 15 (at least 5) Calculated as standard deviation or relative standard deviation (RSD)

23 PRECISION INCREASING NUMBER OF CONSIDERED RANDOM ERROR SOURCES REPEATABILITY INTRA-LABORATORY REPEATABILITY REPRODUCIBILITY STANDARD DEVIATION σ = 1 (x i x) 2 n x = 1 n MEAN x i RELATIVE STANDARD DEVIATION RSD (%) = σ 100 x CAN BE EASILY CALCULATED IN MS EXCEL

24 PRECISION INCREASING NUMBER OF CONSIDERED RANDOM ERROR SOURCES REPEATABILITY INTRA-LABORATORY REPEATABILITY REPRODUCIBILITY RESULT VALUES RSD (%) = σ 100 x

25 PRECISION INCREASING NUMBER OF CONSIDERED RANDOM ERROR SOURCES REPEATABILITY INTRA-LABORATORY REPEATABILITY REPRODUCIBILITY Can be estimated within an inter-laboratory study Two components: σ 2 r intra-laboratory variance (typical) σ 2 L between laboratories variance σ 2 R = σ 2 r + σ 2 L

26 PRECISION INCREASING NUMBER OF CONSIDERED RANDOM ERROR SOURCES REPEATABILITY INTRA-LABORATORY REPEATABILITY REPRODUCIBILITY Can be estimated within an inter-laboratory study however inter-laboratory study is time-demanding and costly it is problematic to find sufficient number of competent laboratories in multi residue analysis it is almost impossible to perform this kind of study for all analytes / matrices / concentration levels

27 PRECISION REPRODUCIBILITY - HORWITZ Reproducibility can be alternatively estimated from an empirical model developed based on numerous interlaboratory studies (more than 150) Horwitz (Thompson) empirical model of precision the RSD can be expressed as a function of the concentration William Horwitz

28 PRECISION REPRODUCIBILITY - HORWITZ Relative standard deviation variation coefficient: lower concentration of analyte increasing RSD nature of analyte, matrix, analytical method etc.: less important even can be ignored! RSD = 2 (1-0.5*logX) X is an analyte concentration expressed as a mass ratio

29 PRECISION REPRODUCIBILITY - HORWITZ 50 RELATIVE STANDARD DEVIATION (%) CV (%) Horwitzova kř ivka ANALYTE CONCENTRATION (ppb) ppb HORWITZ CURVE

30 PRECISION REPRODUCIBILITY - HORWITZ Official requirements on precision trace analysis: Concentration (ppb) CV r (%) Codex Alimentarius RSD R (%) Horwitz < 1 35 > > < 22

31 TRUENESS AND PRECISION = ACCURACY RELATIONSHIPS BETWEEN TYPE OF ERROR, RELATED CHARACTERISTICS AND THEIR QUANTITATIVE EXPRESSION

32 LINEARITY AND CALIBRATION LINEARITY can be tested by linear regression of the responses on the concentrations in an appropriate calibration set R 2 = 0,9977 Peak area DON concentration (µg kg -1 )

33 LINEARITY, CALIBRATION AND RANGE GENERAL RECOMENDATIONS FOR LINEAR CALIBRATION There should be six or more calibration points (standards) Even spacing over the concentration range of interest The calibration range should encompass 0 150% or % of the concentration likely to be encountered in samples Calibration standards should be run at least in duplicate in random order VALIDATION RANGE is the interval of analyte concentration within which the method can be regarded as validated Typically narrower than linear range In practice, most methods will be validated at only one or two levels of concentration. The validated range may be taken as a reasonable extrapolation from these points at concentration scale

34 LINEARITY, CALIBRATION AND RANGE VALIDATED RANGE VALIDATED CONCENTRATION LEVEL Signal VALIDATED CONCENTRATION LEVEL Linear range Concentration

35 LIMIT OF DETECTION / QUANTIFICATION

36 LIMIT OF DETECTION / QUANTIFICATION Limit of Detection (LOD): the smallest concentration of analyte in the test sample which can be reliably distinguished from zero. LOD is concentration of analyte which induce signal (S) that is 3 times higher than the background noise level (N). S/N=3 Limit of Quantification (LOQ): the smallest concentration of analyte in the test sample which can be reliably quantified. LOQ is concentration of analyte which induce signal (S) that is 10 times higher than the background noise level (N). S/N=10, LOQ usually corresponds to lowest calibration point. S/N can be usually calculated in processing software

37 LIMIT OF DETECTION / QUANTIFICATION S/N > 2 S/N > 6 signal noise

38 DETECTABILITY how to lower LOD? Increase of S/N ratio in GC/MS-NCI analysis of nitronaphtalenes (parsley sample) due to increasing electromultiplyier voltage in detector (1-nitroNAP = 0.1 ppb, 2-nitroNap = 0.08 ppb) A - EMV = 1300 V B - EMV = 1800 V 1-nNap 2-nNap A nNap 2-nNap B Time--> Time--> Vladimir.Kocourek@vscht.

39 LIMIT OF DETECTION / QUANTIFICATION HOW TO ESTIMATE LOD & LOQ SPIKING OF BLANK MATRIX WITH DECREASING AMOUNT OF ANALYTES SAMPLE PREPARATION & MEASUREMENTS CALCULATION OF S/N VALUES STANDARD SOLUTION WITH DECREASING AMOUNT OF ANALYTES MEASUREMENTS CALCULATION OF S/N VALUES LOD LOQ

40 Signal at LOD vs. background noise 1,64*s blank 1,64*s blank = 5 % = 5 % s blank y blank Response to blank Y LOD Analyte signal

41 Codex Alimentarius: numerical values for the criteria Applicability: The method has to be applicable for the specified provision, specified commodity and the specified level: maximum and/or minimum - ML. The minimum applicable range of the method depends on the specified level (ML) to be assessed, and can either be expressed in terms of the reproducibility standard deviation (s R ) or in terms of LOD and LOQ. CL 2008/7-MAS, March 2008 Minimum applicable range: ML 0.1 mg/kg [ML - 3 s R, ML + 3 s R ] ML < 0.1 mg/kg [ML - 2 s R, ML + 2 s R ] Limit of Detection (LOD): ML 0.1 mg/kg LOD ML 1/10 ML < 0.1 mg/kg LOD ML 1/5 Limit of Quantification (LOQ): ML 0.1 mg/kg LOQ ML 1/5 ML < 0.1 mg/kg LOQ ML 2/5

42 SELECTIVITY AND SPECIFITY Selectivity: the degree to which a method can quantify the analyte accurately in the presence of interferents. Selective method the results are influenced by the sample matrix (interferents, cross-reactivity, matrix effects, ) Specific methods the results are not influenced by the presence of sample matrix The presence and influence of any sample matrix interference on method results should be tested and described

43 SELECTIVITY AND SPECIFITY MATRIX EFFECTS: DECREASE (INCERASE) AND PEAK SHAPE DETERIORATION High matrix content Low matrix content

44 RUGGEDNESS Ruggedness of an analytical method is the resistance to change in results when minor deviations are made from the experimental conditions described in the SOP. The aspects of the method that are likely to affect results should be identified and described in SOP Examples of factors relevant to ruggedness: ph of a solution stability of the instrumental system extraction time concentration of (derivatization) reagents temperature/time of (derivatization) reaction time allowed for completation of whole analytical process

45 VALIDATION PROCEDURE Development and optimization of the method Spiked samples Training Reference material Repeated measurement Independent method Inter-laboratory study REPEATABILITY TRUENESS REPRODUCIBILITY LOD/LOQ Range Performance characteristics Fit-for-purpose Selectivity Ruggedness

46 USEFUL DOCUMENTS

47 USEFUL DOCUMENTS Criteria for validation of methods used in official control of contaminants and residues in food and feed:

48 All documents and methods issued to personnel in the laboratory shall be reviewed and approved for use by authorized personnel prior to issue. Documents are periodically reviewed and, where necessary, revised to ensure continuing suitability and compliance with applicable requirements; Invalid or obsolete documents are promptly removed from all points of issue or use, or otherwise assured against unintended use; Procedures shall be established to describe how changes in documents maintained in computerized systems are made and controlled. All records and data shall be stored and retained in such a way that they are readily retrievable in facilities that provide a suitable environment to prevent damage or deterioration and to prevent loss. The laboratory shall have procedures to protect and back-up records stored electronically and to prevent unauthorized access to or amendment of these records.

49 The laboratory shall retain records of primary data, calculations, calibration records, staff records and a copy of each test report issued. The records for each test shall contain information to facilitate identification of factors affecting the uncertainty and to enable the test to be repeated under conditions as close as possible to the original. The records shall include the identity of personnel responsible for the performance of each test and checking of results. When mistakes occur in records, each mistake shall be crossed out, not erased, made illegible or deleted, and the correct value entered alongside. All such alterations to records shall be signed by the person making the correction. In the case of records stored electronically, equivalent measures shall be taken to avoid loss or change of original data.

50 The laboratory shall use test methods, including methods for sampling, which meet the needs of the customer and which are appropriate for the tests it undertakes. Methods published in international, regional or national standards shall preferably be used. Appropriate methods have been also published by reputable technical organizations, or in relevant scientific texts or journals, or specified by the manufacturer of the equipment. Laboratory-developed methods or methods adopted by the laboratory may also be used if they are appropriate for the intended use and if they are validated. The laboratory shall validate nonstandard methods, laboratory developed methods, standard methods used outside their intended scope, and modifications of standard methods to confirm that the methods are fit for the intended use. Testing laboratories shall apply procedures for estimating uncertainty of measurement based on the method validation data.

51 USEFUL DOCUMENTS

52 Pesticides standards Pure standards should be of known purity and each must be uniquely identified and the date of receipt recorded. They should be stored at low temperature, preferably in a freezer, with light and moisture excluded, i.e. under conditions that minimise degradation. The identity of freshly acquired pure standards should be checked if the analytes are new to the laboratory. When preparing stock standards of pure standards of analytes and internal standards, the identity and mass of the pure standard and the identity and amount of the solvent must be recorded. The solvent(s) must be appropriate to the analyte (solubility, no reaction) and method of analysis. Moisture must be excluded during equilibration of the pure standard to room temperature before use and concentrations must be corrected for the purity of the pure standard. Not less than 10 mg of the pure standard should be weighed using a 5 decimal place balance. The ambient temperature should be that at which the glassware is calibrated, otherwise preparation of the standard should be based on solvent-mass measurement. Existing stock and working solutions may be tested against newly prepared solutions by comparing the detector responses obtained from appropriate dilutions of individual standards or mixtures of standards. The means from at least three replicate measurements for each of two solutions (old and new) should not normally differ by more than ±10 %. The mean from the new solution is taken to be 100%. Differences in apparent concentration between old and new standards must be investigated.

53 Calibration Responses used to quantify residues must be within the dynamic range of the detector. Extracts containing high-level analytes may be diluted to bring them within the calibrated range. Validation of analytical methods should include determination of recovery at the proposed reporting limit. Calibration by interpolation between two levels is acceptable providing the difference between the 2 levels is not greater than a factor of 4, and where the mean response factors, derived from replicate determinations at each level, indicate acceptable linearity of response with the higher being not more than 110 % of the lower response factor. Single-level calibration may provide more accurate results than multi-level calibration if the detector response is variable with time. When single-level calibration is employed, the sample response should be within ± 20 % of the calibration standard response if the MRL is exceeded. The potential for matrix effects to occur should be assessed at method validation. They are variable in occurrence and intensity but some techniques are particularly prone to them. If the techniques used are not inherently free from such effects, calibration should be matrix-matched routinely. Where a calibration standard is a mixture of isomers, etc., of the analyte, detector response generally may be assumed to be similar, on a molar basis, for each component.

54 Contamination & interference Samples must be separated from each other, and from other sources of potential contamination, during transit to, and storage at, the laboratory. Pest control in, or near, the laboratory must be restricted to pesticides that will not be sought as residues. Volumetric equipment, such as flasks, pipettes and syringes must be cleaned scrupulously, especially for re-use. As far as practicable, separate glassware, etc., should be allocated to standards and sample extracts, in order to avoid cross-contamination. Avoid using excessively scratched or etched glassware. Equipment, containers, solvents (including water), reagents, etc., should be checked as sources of possible interference. Rubber and plastic items (e.g. seals, protective gloves, wash bottles) and lubricants are frequent sources. Vial seals should be PTFE-lined. Extracts should be kept out of contact with seals by keeping vials upright. Vial seals may have to be replaced quickly after piercing, if re-analysis of the extracts is necessary. Analysis of reagent blanks should identify sources of interference in the equipment or materials used.

55 Validation parameters and criteria Parameter What/How Criterion Linearity Through calibration curve Residuals < ±20 % Matrix effect LOQ Specificity Accuracy Precision (RSD r ) Comparison of response from solvent standards and matrix-matched standards Definition: lowest level for which it has been demonstrated that criteria for accuracy and precision have been met Response in reagent blank and control samples Determine average recovery for both spike levels Determine repeatability RSDr, determine for both spike levels MRL < 30 % of LOQ % 20 %

56 Method validation Within-laboratory method validation should be performed to provide evidence that a method is fit for the purpose for which it is to be used. Method validation is a requirement of accreditation bodies, and must be supported and extended by method performance verification during routine analysis (analytical quality control). The method must be tested for sensitivity, mean recovery (as a measure of trueness or bias), precision, and limit of quantification (LOQ). This means that spiked recovery experiments to check the accuracy of the method should be undertaken. A minimum of 5 replicates is required at both the reporting limit, and at least another higher level, perhaps an action level, for example the MRL. The (method) LOQ is defined as the lowest validated spike level meeting the method performance acceptability criteria: mean recovery for each pesticide in the range %, with a RSD r 20%. Exceptionally, where recovery is low but consistent (demonstrating good precision) and the basis for this is well established (e.g. due to pesticide distribution in partition), a mean recovery below 70% may be acceptable.

57 Acceptability of analytical performance Acceptable limits for a single recovery result should normally be within the range of the mean recovery +/- 2x RSD). The laboratory must participate regularly in relevant proficiency tests. When a low number of compounds are analysed with respect to the pesticides present in the test sample, false positive(s) or negative(s) are reported or the accuracy achieved in any of the tests is questionable or unacceptable, the problem(s) should be investigated. Unacceptable performance, has to be rectified before proceeding with further determinations of the analyte involved. Negative results for represented analytes are supported only indirectly by the recovery and LCL data for representative analytes and must be interpreted with caution. Positive results (residues at or above the RL) usually require additional confirmation. Suspected MRL exceedances or unusual residues must be identified. The use of a highly specific detection system, such as mass spectrometry, is recommended. Selective detectors employed with GC or LC such as ECD, FPD, NPD, DAD and fluorescence, offer only limited specificity. Their use, even in combination with different polarity columns, does not provide unambiguous identification.

58 Identification - confirmation Mass spectrometry in conjunction with chromatographic separation is a very powerful combination for identification of an analyte. It simultaneously provides: i. retention time ii. ion mass/charge ratio; and iii. abundance data. The retention time (or relative retention time) of the analyte in the sample extract must match that of the calibration standard within a specified window after taking into consideration the resolving power of the chromatographic system. The relative retention time of the analyte, should correspond to that of the calibration solution with a tolerance of ± 0.5 % for GC and ± 2.5 % for LC. Reference spectra for the analyte should be generated using the instruments and techniques employed for analysis of the samples. If major differences are evident between a published spectrum and that generated within the laboratory, the latter must be shown to be valid. To avoid distortion of ion ratios, the response of the analyte ions must not overload the detector.

59 Identification - confirmation Different types and modes of mass spectrometric detectors provide different degrees of selectivity, which relates to the confidence in identification:

60 Identification - confirmation Different types and modes of mass spectrometric detectors provide different degrees of selectivity, which relates to the confidence in identification: For a higher degree of confidence in identification, further evidence may be required. This can be achieved through additional mass spectrometric information, for example evaluation of full scan spectra, additional accurate mass (fragment) ions, additional product ions (in MS/MS), or accurate mass product ions.

61 Identification points Commission decision 2002/657/EC:

62 Method validation pesticides: conditions

63 Thank you for your attention