STATISTICAL THEORY AND ANALYSIS OF GMO ENFORCEMENT (STAGE) DEFRA PROJECT CB0209 FINAL PROJECT REPORT

Transcription

1 STATISTICAL THEORY AND ANALYSIS OF GMO ENFORCEMENT (STAGE) DEFRA PROJECT CB0209 FINAL PROJECT REPORT JANUARY 2005 CENTRAL SCIENCE LABORATORY, SAND HUTTON, YORK, YO41 1LZ

2 CONTENTS Contents...1 EXECUTIVE SUMMARY INTRODUCTION REVIEW OF LITERATURE AND CURRENT PRACTICE Outline of current sampling methods Certified seed lot size Sampling size and frequency International Seed Testing Association (ISTA) Rules [2] ISO 13690: 1999 Cereals, pulses and milled products sampling of static batches [1] ISO 6644 Flowing cereals and milled cereal products - automatic sampling by mechanical means [8] EN ISO 542: 1995 (BS 4146: 1991) Oilseeds sampling [7] Commission Recommendation on technical guidance for sampling and detection of genetically modified organisms and material produced from as or in products in the context of Regulation (EC) No. 1830/2003 [3] American Association of Cereal Chemists (AACC) method 64-70A and method [22] USDA Grain Inspection, Packers and Stockyards Administration (GIPSA) European Enforcement Project (EEP) Kernel Sampling Technique Evaluation (KeSTE) [10] / Kernel Lot Distribution Assessment ( KeLDA) GM testing methods Commercial GM testing methods and procedures Quality control of GM testing methods

3 2.4 EU recommended methodology and detection limits Background EU legislation Theoretical detection limits of polymerase chain reaction (PCR) EU recommended methods for PCR tests Review of seed audits DATA COLLATION AND ANALYSIS Database description Statistical analysis of database Data description Results of statistical analysis Is there a difference in performance between before and after extraction methods were changed? Do some primer/ positive control sets perform better than others? Is there any change in performance over time? What is the estimated false negative detection rate for unknown samples? SIMULATION OF SAMPLING AND ANALYSIS OF GM SEEDS Model Design Stratified-binomial model for heterogeneity Beta-binomial model for heterogeneity Sampling from a working sample Features of the stratified-binomial and beta-binomial models Comparing beta-binomial and stratified binomial models to a physical model for diffusion of GM seeds into a lot Simulating the detection of GM seed in an analytical sample Analytical replication in the model Outputs from the model

4 4.3 Input values for model parameters Commodity parameters Lot parameters Sampling parameters Analytical parameters Seed lot / as grown scenario Results from the model simulations Seed lot / as grown scenario Grain as commodity scenario Effect of heterogeneity on the detection of 0.1% GM seeds in a lot Model sensitivity analysis CONCLUSIONS Future work...68 References

5 EXECUTIVE SUMMARY 1. This project was commissioned to research the theory and practice of sampling and detection for genetically modified organisms (GMOs) and to develop an enhanced understanding of our statistical confidence in results obtained. The project has draw together practical experience, data and statistical theory to address the scenarios commonly encountered in the work of the GM Inspectorate (GMI). This has enabled us to state clearly and with supporting evidence the confidence we have in the whole testing process and to propose how to deal rationally with situations such as re-tests and false positives. 2. A novel statistical simulation model has been developed that combines sampling and testing for GMOs in one framework, incorporating the uncertainties in each. This model offers policy makers and regulators an evidence-based tool that can be used to inform decision taking. The model has been used to critically review the performance of current practice in sampling and testing to detect reliably the presence of genetic modification (GM) in oilseed rape at permitted thresholds under a range of commonly encountered situations. 3. Model outputs and application. The model produces operating characteristic curves that show the probability of detection of a GMO as a function of the true mean proportion in the lot. Other summary statistics relating to the repeatability of various stages in the process are also produced. The model directly incorporates all parameters with significant impact on uncertainty and limits of detection for GM sampling and testing; it can therefore be used to comprehensively define the minimum parameter bounds within which good practice (i.e. fitness-for-purpose reliability) is achievable. Thus, the model can inform guidance that should be issued to underpin the enforcement of European Union (EU) and national legislation. The model can also be used to optimise the allocation of resources to greatest effect to meet cost or logistic constraints. The model could be used in the future to assist the GM Inspectorate in its audits of the procedures of seed importers and producers by providing objective standards against which to judge the supplied information and, through model sensitivity analysis, an understanding of what data are critical to know in order to be confident in the fitness for purpose of the submitted procedure. False positive 4

6 and negative rates are sometimes provided in supporting data these can be incorporated in the model to assess their impact on the claimed results. The model could also be used to inform decisions and provide an objective basis upon which to propose further sampling and/or testing in any investigation where a potential breach of consent or legislation is suspected. 4. Model construction. The model structure explicitly includes all stages of the sampling and testing process. Its mathematical and statistical calculations are derived from the best available and generally accepted theory. The input parameters of the model relate directly to information commonly available in the course of collecting samples and conducting laboratory tests. The model is implemented in Microsoft Excel VBA and runs readily and quickly on a moderately powerful personal computer. 5. Modelling heterogeneity. Heterogeneous distribution of GM seeds in bulks is an issue that has been given much consideration. Heterogeneity is important because, as the degree of heterogeneity increases, the choice of appropriate sampling plan to ensure samples are representative of the material being analysed becomes increasingly important. Heterogeneity can be modelled by considering the bulk/lot as composed of a mixture of sub lots each characterised by a mean proportion of GM seed, and specifying how the lot is partitioned into these sub lots. Two methods of taking account of heterogeneity in the sampled bulk were considered in the development of the model used in this study: first, a model that partitions the bulk into two components with respectively zero or constant proportion of GM seeds (called stratified-binomial ) and, second, a more general model where each sub lot conforms to the binomial distribution and the mixture of these (i.e. whole lot) is modelled by the beta distribution this is the beta-binomial model. Heterogeneity is input to these models as the proportion of the bulk / lot expected to contain 100% or 95%, respectively, of any GM seeds present. This is an intuitive parameter that could be estimated by expert opinion where measurements are not available. Results from simulations based on these two methods for heterogeneity showed that the stratified-binomial approach generally gives results with sharp discontinuities in the probability distribution 5

7 functions for number of seeds in each increment. The results from the beta-binomial model give plausible continuous probability distributions for GM throughout the whole bulk / lot. The stratified-binomial model assumes no mixing between the GM and non-gm sub lots ; whereas, the beta-binomial model gives results that are consistent with a simple physical model for diffusion and seems more realistic given that there is inevitably some mixing as seeds are handled and transported. 6. Model inputs. Model input data were derived from a review of international protocols that define sampling and testing processes and relevant literature together with an analysis of data held at the Central Science Laboratory (CSL) collected in the course of quality control for GM testing. The model has been applied to a range of situations that relate to the work of the GM Inspectorate which include sampling and testing scenarios such as certified seed lots, seeds as traded commodity, and as-grown seeds prior to certification. 7. Sensitivity and specificity. False negatives (presence of GM seeds in the bulk / lot not detected) can arise even where the testing process is fully under control and are typically expected to be close to the limits of detection. Scenario-specific false negative rates (i.e. sensitivity) can be estimated from the model or can be an input parameter where sufficiently detailed information on the testing process is not available. False positives (i.e. positive results for samples that should be negative) can only occur where the testing process is out of control and are caused by extraneous factors such as cross-contamination of analytical aliquots. Hence, estimation of false positive rates (i.e. specificity) is not included in the model. Laboratory protocols reject all test results from a batch where negative controls are found to be positive and this provides effective protection against spurious results. In the dataset used in this study, the rate of failure of negative controls (false positive rate) was 0.7% and was approximately two orders of magnitude less than the false negative rate. False positive rates can be included in the model where desired, for example when modelling decision rules for replication to reduce the risk of reporting false positive results. 6

8 8. Results from simulations. The application of the model has confirmed the interdependence of sampling and testing in determining the reliability of the combined process. Results from the range of practical scenarios for oilseed rape modelled show that current best practice is generally reliable in assuring that existing thresholds down to 0.1% can be met by the analysis of samples (of 3000 seeds) from a small number of increments taken from a lot, even in the presence of heterogeneity, provided that suitable analytical replication is employed (e.g. duplicate analytical samples with duplicate DNA extractions). However, if no analytical replication is employed then the presence of 0.1% GM will not be reliably detected by the analysis of samples (of 3000 seeds) based on a large number of increments, even if taken from a homogenous lot; any heterogeneity in the lot will make this less reliable. The model has also been used to show that the level of replication (analytical sample, DNA extraction, polymerase chain reaction (PCR)) is at least as important as the number of replicate analyses. The effects of bulk / lot heterogeneity can be ameliorated to a large degree by employing replicate analytical samples, but carrying out replicate PCR determinations of a single analytical extract generates little improvement. 9. Potential extensions. The model structure and flexible output provision will allow it to be readily adapted to new situations (e.g. quantification by Most Probable Number based on splitting samples into pools for testing). In addition, a further quantitative module could be developed to estimate the risks (for producers and consumers) attached to labelling or rejection decisions based on the entire sampling and analytical process whether the analysis is semi-quantitative (e.g. number of positive seed pools) or fully quantitative (e.g. real-time PCR). 10. Conclusions. It is essential to combine both sampling and testing uncertainty in any model of the reliability of GM seed testing. Heterogeneity is a crucial factor in such a model and should be incorporated in a physically-plausible manner. Results from simulations under a range of commonly-encountered scenarios for GM inspection and enforcement of oilseed rape seed show that good practice can ensure the reliability of the sampling and testing process in meeting the requirements of legislation on thresholds and confidence levels. This study has 7

9 objectively demonstrated the information required to ascertain the reliability of claimed results. 8

10 1. INTRODUCTION The regulation of adventitious presence of GMOs in seeds, crops and foodstuffs requires sampling and testing processes that can provide a predictable minimum level of confidence in their results. This is essential if test results are to be used to inform decisions on labelling and/or rejection of commodities due to their GM content, and to maintain public confidence in the food supply chain. The issue under investigation in this study is the confidence that can be placed on the combined performance of sampling and testing processes to detect (not quantify) GM at low level. It is particularly relevant, therefore, to 'unauthorised'gmo adventitious presence for which acceptable thresholds are low ( 0.1%). The overall aim of this project was to examine the whole process of GM testing, and its component stages, in order to estimate the total confidence that can be placed on final test results. By considering various different elements from sampling through to laboratory testing, the study has also identified the processes and parameters most critical to the overall test confidence levels and the sensitivity of the process to each of its component elements. As an example, we have investigated the sampling and testing of oilseed rape (OSR). There is no single internationally accepted protocol for sampling and GM testing as an integrated process. Accepted methods are commodity specific and can vary regionally. International standards do exist for sampling methods, which also can be specific to commodity type e.g. International Standards Organisation (ISO) Standards [1] for grain and International Seed Testing Association (ISTA) Rules [2] for seed; some are specific to GM testing e.g. EU legislation guidance [3]. In many situations, none of these sampling methods need necessarily have been employed. It is important to be able to estimate the effect that any sampling scheme could have on the confidence in a test result. Testing methods for GM presence in OSR are currently restricted to PCR-based methods. Lists of recommended and/or validated tests are available [4] but they do not stipulate the entire procedure to be used (including sampling) or precisely describe the parameters specifying the laboratory methods. As a result of these uncertainties, it is difficult for regulators, producers and consumers to have a known confidence in test results or known level of risk attached to decisions based on those results. 9

11 Hence, a tool for modelling the entire measurement process of sampling and PCR-based GM detection by computer simulation has been produced. The model has been used to examine the detection of GM seeds in oilseed rape certified seed lots and oilseed rape grain lots using parameters that are the most realistic and up-to-date at this time. The framework of the model, however, allows the modelling of many different scenarios and inputs to provide several key outputs that can be used to assess the robustness of any test result, given sufficient information on how that result was obtained. The outputs can be used to estimate several commonly requested parameters and information such as the expected false-negative rate and effectiveness of further analytical replication. 10

12 2. REVIEW OF LITERATURE AND CURRENT PRACTICE 2.1 Outline of current sampling methods The purpose of sampling is either to obtain a sample corresponding in characteristics and composition to the lot from which it was taken, or to detect unwanted hidden contamination. This review summarises and discusses the various sampling protocols for seed and grain that have been published (or are pending) in the EU and elsewhere. Particular emphasis is given to oilseed rape, but other commodities are also covered. At present there are very few published protocols designed specifically for sampling for the detection of adventitious GMOs. Exceptions are the United States Food and Drug Administration s (USFDA) Cry9C protocol, which provides the agricultural sector with guidelines for sampling maize for Cry9C protein residues from Starlink GM maize; also a draft protocol produced on behalf of the European Enforcement Project (Reed, 2002) [5]. Kay and Paoletti (2001) [6] suggest employing ISO (cereals) [1] or ISO 542 (rapeseed) [7] when sampling in order to detect GM presence in static bulks. For flowing grain, they suggest using ISO 6644 [8]. In their crop guidance documents the CSL GM Inspectorate recommend using International Seed Testing Association (ISTA) rules when sampling for adventitious GM presence, although it must be borne in mind that these rules have been developed in order to fulfil quality criteria that fall short of the 0% level for unauthorized GM presence. For example, in the case of oilseed rape the varietal purity standard is 99.7% for certified seed (and only 99% for fodder rape). The New Zealand Ministry of Agriculture and Forestry also stipulate that samples are collected using ISTA (or Association of Official Seed Analysts) methodology. However, very few of the current sampling standards have been devised for the purpose of GM detection (including ISTA), and little, if any, validation has been conducted on them to determine whether they are fit for this purpose. Guidelines from various EU and US sources for the detection of GMOs are summarised in a draft document by Paoletti et al., (2003) [9]. The authors conclude that most sampling protocols are based on the false assumption of homogeneity or random distribution, and propose a validation protocol to identify appropriate sampling strategies by simulation studies based on data from EU member states (Kernal Lot Distribution Assessment, KeLDA) [10]. 11

13 2.1.1 Certified seed lot size The fundamental basis of seed certification is to enable certain parameters, e.g. purity and germination, to be stated for a particular lot with a high degree of confidence. Statements on seed quality are based on tests carried out on samples from each lot. Confidence in these statements requires that the samples are representative, which in turn is based on the lot being homogeneous and the samples being drawn randomly. The issue of samples being representative is particularly important when considering GMOs as the tolerable levels are very low, adventitious GM presence is unlikely to be inherently uniformly distributed and it may be necessary to distinguish between two lots derived from the same as grown crop. There are two issues of concern in the literature for this project. First, none of the studies cited specifically deals with oilseed rape. Second, as the studies tend to be precipitated by pressure from the industry to increase maximum lot sizes, there are no hard data on long-established lot sizes although it is expected under current maxima that 1% of lots will be expected to be heterogeneous [11]. 2.2 Sampling size and frequency International Seed Testing Association (ISTA) Rules [2] The first issue of the ISTA rules in 1931 included maximum seed lot sizes and ISTA continue to set maximum seed lot sizes as a precautionary measure to limit heterogeneity in seed lots. ISTA tests [2] for heterogeneity are based on a variance ratio method [12], which compares measured variance with expected variance from a binomial distribution assuming equal means and sets acceptable limits on heterogeneity. A modification of this method [13] gives the statistic D which, although related to heterogeneity, is independent of the number of samples and can be used to combine data from disparate studies. This has led to a number of studies ([13], [14], [15]) and comprehensive reviews ([16], [17]) demonstrating that as lot size increases both the number of heterogeneous lots and the average heterogeneity (H value) increase in a linear fashion. ISTA stipulates sampling frequencies for seed lots up to 40t (±5%) in mass (this is the maximum lot size and applies to maize). For oilseed rape, the maximum lot size is 12

14 10t ±5%. There is no specified minimum mass for a seed lot. Legislation in England [18] currently sets the maximum lot size for oilseed rape at 10 tonnes; these regulations stem from EC Directive 2002/57 [19], which, in turn, was informed by the ISTA rules [20]. Actual sampling frequencies range from 3 primary samples per container (Table 1) for small containers, to 1 sample per 700kg for large containers or seed in bulk (Table 2). Table 1: Sampling requirements (following ISO standards) for seed in sacks or similar sized containers, containing at least 15kg but not more than 100kg of seed. Number of containers in the lot Minimum number of primary samples to be taken primary samples from each container primary samples from each container primary samples from each container primary samples from the seed lot primary samples from the seed lot 60 or more 30 primary samples from the seed lot Table 2: Sampling requirements (following ISO standards) for large containers or seed in bulk. Lot mass Minimum number of primary samples to be taken Up to 500kg At least 5 primary samples 501 3,000kg One primary samples for each 300kg but not less than 5 3,001 20,000kg One primary samples for each 500kg but not less than 10 20,001kg and above One primary samples for each 700kg but not less than 40 For sampling containers holding more than 100kg of seed, and for seed in bulk, primary samples must be taken from different horizontal and vertical positions systematically selected or at random. The ISTA rules are periodically updated to take account of increases in current knowledge. The latest edition is Edition 2004 (valid from 01/01/04) [2]. Containers may be sampled systematically or at random and primary samples drawn from top, middle or bottom. The position from which the seed is taken is to be varied from container to container. 13

15 Where seed lots are held in containers holding less than 15kg of seed, a 100kg mass of seed is taken as the basic unit and the small containers are combined to form sampling units not exceeding this mass (e.g. 10 packages of 10kg, 8 packages of 12kg). For sampling purposes, each unit is regarded as one notional container and the sampling procedures prescribed in Table 1 are used. For example, a seed lot consists of 3.84t divided into 320 packages of 12kg each: 8 12kg = 96kg, 320/8 = 40 containers, therefore the number of primary samples to be taken from the lot is 20. The sampling instrument must be capable of sampling all parts of the seed lot. Samples are generally taken using spears, and these range from small sack spears to large multi-chambered instruments. Samples may also be drawn from a seed stream during processing, using an approved automatic sampling device. In this case portions of seed are taken at regular intervals throughout the processing of the lot using the same sampling intensity as for seed in bulk (Table 2). At the time of sampling the lot must have been subjected to appropriate mixing, blending, and processing techniques so that it is as uniform as practicable. If there is evidence of heterogeneity then the seed lot should not be sampled. In cases of doubt, the ISTA rules state that heterogeneity can be determined using an equation that takes account of the expected and observed variance of the particular attribute being tested. However, there are likely to be a number of quality characteristics where it is not possible to visually differentiate between a homogeneous and a heterogeneous lot and therefore there will be no cue to conduct heterogeneity testing GMO presence falls into this category. Primary samples should be of approximately equal size and, whilst no indication is given of the desired mass, this factor will be influenced in practice by the actual seed mass and the various test requirements. Once all the primary samples have been taken and appear to be uniform, they are combined to form the composite sample. The sample submitted for testing is obtained by mixing and reducing the composite sample (if necessary) to an appropriate size using a riffle or centrifugal divider. For OSR, the minimum mass of a submitted sample is 200g. In the laboratory the composite sample is once again thoroughly mixed and further reduced to obtain the working sample for each specific test. 14

16 ISTA sampling methodologies have been adopted by the CSL GM Inspectorate when sampling to detect adventitious GM presence and are recommended in the guidance issued by the Inspectorate to UK seed producers and importers ISO 13690: 1999 Cereals, pulses and milled products sampling of static batches [1] ISO concerns the sampling of static bulks of cereals, pulses and milled products. This protocol covers manual or mechanical sampling up to a depth of 3m and mechanical sampling up to 12m. For bulks exceeding 12m depths, ISO 6644 must be used [8]. Consignments are considered in lots of 500t maximum and there are recommended sampling rates for lot sizes from 500 to 10,000 tons. Samples must consist of at least 3000 grains each and samples must be tested individually in order to fall within expected detection limits and to reveal potential heterogeneity in the lot. ISO does not apply to seed grain or when sampling to test for hidden infestation. a) Sampling from bags: increments are taken from different parts of the bag (e.g. top, middle, bottom) using a sack/bag spear. The sampling rate is shown in Table 3, however there is no indication of the maximum or minimum size of the bag. Table 3: Sampling requirements for seed in bags according to ISTA rules. Number of bags in consignment Number of bags to be sampled Up to 10 Each bag 10 to , taken at random More than 100 Square root (approx.) of total number, taken according to a suitable sampling scheme a a For example, divide into (n-1) groups containing n (or n-1) bags; the remaining bags constitute a group. For a consignment comprising 200 bags, 200 = , therefore n=14. Make up 14 groups of 14 bags (total 196 bags); draw up a list of 1 to 14, and cross out one number (e.g. 7); sample the 7 th bag from each group; sample 1 bag at random from the remaining group. A total of 15 bags have therefore been selected. b) Sampling from rail or road wagons, lorries, barges or ships: each laden wagon is sampled and a suggested pattern is provided for increments depending on the lot size (Figure 1) 15

17 Figure 1: Sampling from rail or road wagons, lorries, barges or ships [7]. Up to 15t: 5 sampling points 15 to 30t: 8 sampling points t: minimum of 11 sampling points c) Sampling from silos, bins or warehouses: grain is sampled using a grid system (e.g. one similar to b)). The minimum number of increments is determined as follows: square root of tonnage divided by 2 and rounded to the next whole number (e.g. 500t = 12 samples; 10,000t = 50 samples). The composite sample is formed by combining the increments and mixing them thoroughly. The laboratory sample is obtained by coning and quartering or by using a riffle or centrifugal divider. The size of laboratory sample(s) is determined by the type and requirements of the tests, but a minimum of 1kg is generally recommended ISO 6644 Flowing cereals and milled cereal products - automatic sampling by mechanical means [8] This method is applicable for all depths of bulk cereals/milled cereal products. It describes an approach whereby a mechanical sampling device is used to take an increment or series of increments from a lot, either continuously or intermittently, and repeatedly. The sampling device must be capable of taking increments from the entire cross section, or as much of it as possible. Consignments are considered in lots of 500t maximum, and a maximum increment size of 1kg is recommended for 16

18 intermittent sampling. A maximum composite sample size of 100kg is recommended giving a sampling rate of 1 sample per 5t EN ISO 542: 1995 (BS 4146: 1991) Oilseeds sampling [7] ISO 542 specifies general conditions relating to the sampling of oilseeds for the assessment of quality. Consignments are considered in lots not exceeding 500t and the material may be in bulk or in bags. Increments are taken either from the flowing product (preferred method) or, in the case of lorries and wagons by sampling at least 5 different positions according to the size of the consignment. Recommended sampling apparatus includes spears, scoops and steam samplers. It is necessary to take a sufficient number of increments to provide a representative bulk sample. Products in bags increments must be taken from 2% of the bags forming the lot, with a minimum of 5 bags sampled. Products in bulk increments taken from flowing products must be taken across the whole section of the flow at time intervals depending on the rate of flow. In the case of wagons or lorries increments shall be taken at 3 levels at least, at the points specified in Figure 1 (ISO (see 2.2.2, above), although ISO 542 specifies only 30 to 50t for this last sampling plan. Bulk samples are mixed and divided to obtain the required number of laboratory samples (e.g. using quartering iron, conical or slot divider). Recommended sample sizes are shown in Table 4. Table 4: Recommended mass of samples of oilseeds [7] Product Increment mass (kg) Bulk sample mass (kg) Laboratory sample mass (kg) Medium- and to 5 large-sized seeds Small seeds to 2 17

19 It is apparent that for small seeds such as oilseed rape 250 increments of 0.2kg should be taken per lot to achieve a bulk sample of 50kg. This sample is then reduced to obtain a laboratory sample of 1 to 2 kg Commission Recommendation on technical guidance for sampling and detection of genetically modified organisms and material produced from as or in products in the context of Regulation (EC) No. 1830/2003 [3] Regulation 1830/2003 [21] describing Member States obligations in respect of the traceability and labelling of GMOs requires that technical guidance on the sampling and detection of GMOs is established. This draft guidance document was accepted in October It should be noted that this guidance document has been developed to facilitate compliance with labelling requirements and is not concerned with unauthorised GM events although the sampling issues for both are analogous. This recommendation states that sampling of seeds must be carried out in accordance with internationally accepted methods and that these will normally be the latest ISTA methods. The general principles and methods of sampling should be in accordance with the ISTA rules and the associated ISTA Handbook on Seed Sampling. The recommendation also states that systematic sampling be used as at this time there is no substantial information on the distribution of GM seed in conventional seed lots. In this context, the associated risks and corresponding quality levels are defined in relation to the thresholds for genetically modified seeds and relate to the percentage of GM seeds in the total seed. The associated risks are defined as follows: the risk to the producer (α risk) shall be no greater than 5% that their seed lot will be rejected if it has a true GM content (%) below the Acceptable Quality Level (AQL) (this gives a 95% confidence level to the producer that a lot below the AQL is accepted); the risk to the consumer (β risk) shall be no greater than 5% that they receive a seed lot that has a true GM content (%) above the Lower Quality Level (LQL) (this gives a 95% confidence level to the consumer that a lot above the LQL is rejected). For thresholds greater than 0%, the Acceptable Quality Level (AQL) is half the threshold and the Lower Quality Level (LQL) is twice the threshold, according to normal analytical practice, but not exceeding 0.9%. For a threshold of 0%, the Lower Quality Level 18

20 (LQL) is 0.1%. The minimum number of seeds to be examined is set at 3000 and is determined on the basis of the LQL of the 0% threshold for unauthorised events being 0.1% and a β risk = 5%. Sampling of bulk commodities (grains) should be carried out in accordance with the relevant ISO standards (e.g. ISO 6644 and ISO for cereals and pulses) where available. Where such standards do not exist, guidance may be sought from the European Network of GMO Laboratories (ENGL) but all sampling strategies should ensure that (a) any sample-sized unit within a bulk should have the same probability of being selected as any other, and (b) grain size is taken into account. It cannot be assumed that grain lots are homogeneous with respect to GM content, thus the sampling plan must aim to provide a number of samples the test results of which approach as closely as is practicable the true mean and variance of the entire lot. Samples must consist of at least 3000 grains each and samples must be tested individually in order to meet expected detection limits and to reveal potential heterogeneity in the lot. The recommendation states that a sequential testing plan may be adopted and therefore further sampling and testing may be necessary (e.g. if the test result is sufficiently close to the threshold to preclude a labelling decision with acceptable risk) American Association of Cereal Chemists (AACC) method 64-70A and method [22] Method 64-70A concerns the manual sampling of grains and suggests sampling plans but does not recommend sampling rates other than for moving streams where it is suggested to take 1 sample (size unspecified) for every 500 bushels (approximately 14.6t wheat). Method recommends diverter sampler activation rates of once every 5.5t USDA Grain Inspection, Packers and Stockyards Administration (GIPSA) GIPSA s Grain Inspection handbook Book 1 Sampling [23] discusses the mechanics of sampling grain and provides various sampling plans for barges, trucks, and hopper cars, although the principles would presumably apply equally to static bins of similar dimensions. Grain in sacks is also covered, albeit briefly, with a recommended sampling intensity of 72 samples for lots over 10,000 sacks. No indication of 19

21 sampling intensity is given for lots with less than this number of sacks, and the size of the sacks and samples is not stipulated. Whether the sampling regime is designed for regulatory or market requirements is not specified, and although the book emphasizes the need for representative sampling it seems to rely on number and position of sampling points per container rather than number of samples per mass of material GIPSA s Practical application of Sampling for the Detection of Biotech Grains [24] This document [24] briefly discusses sampling theory with reference to the detection of GM maize, although the principles it refers to can be applied to other commodities. Although the sampling plan assumes that a single kernel can be detected in a sample regardless of the size of the sample, it does acknowledge that in practice analytical methods have limits and it suggests that large samples should be subdivided and each part tested separately. The following formula and Table 5 give sample sizes for various lot concentrations for different probabilities: N = log(1-(g/100))/log(1-(p/100)) Where: n is the sample size (number of kernels) G is the probability of rejecting a lot concentration, and P is percent concentration in the lot Table 5: Requisite sample sizes and associated probability levels for maize according to different lot concentrations [24] Lot concentration (%) Number of kernels required 99 % rejection 95 % rejection 90 % rejection 0.05 % % % GIPSA s Sampling and testing recommendations for the detection of Cry 9C protein in hybrid seed corn [25] This document [25] prescribes the use of the sampling protocols set out in the GIPSA Grain Inspection handbook to obtain a representative sample when testing for Starlink corn. The method involves systematically dividing the sample until 2400 kernels remain. These are then tested using lateral flow test kits (e.g. the 20

22 Cry9C QuickStix test). GIPSA has validated several of these test kits and claim reliable detection limits down to one StarLink kernel in 800 (0.125%) (see The original document also gives the probabilities of accepting a lot for varying concentration levels based on a detection limit of 0.2% (Table 6). Table 6: Probability of accepting maize at various GM concentrations (% of kernels containing Cry9C protein) [25] Lot concentration (%) Probability of acceptance (%) European Enforcement Project (EEP) Protocol for seed sampling for testing for GM presence This standard operating procedure has been produced on behalf of the EEP and states that sampling of seed should be in accordance with Article 7 of Council Directive 66/400/EEC for beet seed, or the equivalent article in the applicable directives for other crops, and that the general principles and methods of the ISTA rules (1999) [26] (latest edition [27]) and associated ISTA Handbook on Statistics in Seed Testing (revised version 2002) [28] should be followed Protocol for bulk grain sampling for testing for GM presence [5] The EEP bulk sampling Standard Operating Procedure (SOP) is based on ISO 950, with samples taken from 5-11 points for lorries of 15-50t (sample size 1kg for rapeseed or 2kg for soya or maize). Samples from ships holds, docksides, stores or silos should be from lots up to 500t, and the suggested rate for flowing grain is 1kg/100t for cereals (wheat and barley) and double this for maize and soya. The 21

23 ISO 950 approach for static bulks is 10 samples (about 5kg) per 100t from different depth and positions. ISO 950 has since been superseded by ISO Kernel Sampling Technique Evaluation (KeSTE) [10] / Kernel Lot Distribution Assessment ( KeLDA) The aim of the KeLDA project is to estimate the distribution of GM material in soybean kernel lots (i.e. whole soya) imported within EU Member States, evaluate currently adopted sampling strategies for the detection of GM material in lots of bulk raw materials and provide recommendations for implementing sampling strategies. The project has been initiated on the basis that in the past there has been a general oversimplification of the distribution of GM material (or other contaminants) in bulk lots. According to Lischer (2001) [29, 30] the degree of heterogeneity of a lot can be described as a scalar function, the homogeneity of which is zero. The current status of the project is that the sampling has been completed and the majority of analyses have been carried out. A trial sampling model has been developed and is available to participants on CD (Excel format). 2.3 GM testing methods Commercial GM testing methods and procedures There is little information available on the testing procedures of commercial organisations involved in the analysis of GM plants. These organisations are in competition with each other and closely guard their methods and procedures. The methods and procedures used by the Central Science Laboratory GM testing service ( provided much of the information and data required to mathematically model the testing process in this study Quality control of GM testing methods The literature contains many methods for the detection and more recently the quantification of GM plants. Methods applicable to the detection and quantification of GM OSR are referred to e.g. by Block et al (2003) [31], Demeke et al (2002) [32], Zeitler et al (2002) [33] and James et al (2003) [34]. These recommended methods 22

24 are all based upon polymerase chain reaction (PCR) analysis of GM DNA. PCR is a sensitive and robust technology that is particularly applicable to the detection of GM plants due to its versatility and rapid application Specificity The methods in the literature must, however, be used with caution. Recently PCR-based methods for the detection and quantification of OSR have been published [33], [35]. These methods, as reported, include OSR-specific PCR systems, however, when they were tested at CSL, it was found that they amplified numerous Brassica species, and were not therefore OSR specific (unpublished observations). This has particular implications for the detection and quantification of GM OSR where there is the possibility of mixed Brassica seed lots. Many Brassica family seeds look identical and cannot be distinguished visually from each other. Testing methods would not identify the proportion of OSR seeds within a mixed Brassica sample and could conceivably underestimate the quantity of GM OSR present Calibration standards PCR-based analyses must be controlled using standards against which results can be measured. To ensure quality and reproducibility, standards must be maintained across all analyses. Currently there are certified reference standards only for RoundUp Ready soya and BT11, Bt176 and MON810 maize produced and certified by the Institute for Reference Materials and Measurements (IRRM) (European Commission Joint Research Centre, Geel, Belgium). There are currently no certified reference standards for OSR, which means that it is not possible to compare methods used by different laboratories and monitor and maintain quality of the analysis. Additionally most laboratories will be unable to calculate a true limit of detection and will, therefore, be unable to determine the sensitivity and robustness of their methods. A certified reference material for OSR will be difficult to make due to the stability problems inherent in the oily meal produced when OSR seeds are milled. An alternative may, however, be produced in the form of DNA, either genomic DNA 23

25 isolated from OSR or, more likely, plasmid DNA. Plasmids are small circular pieces of DNA found in bacteria and can be produced on a large scale, stored for long periods of time without loss of quality and can be spiked into different DNA backgrounds. They can therefore be used as gold standards during GM analyses. A range of plasmids could be produced, covering the whole range of GM elements included in GM plants and the repertoire of plasmids could easily be extended to include new genetic elements. CSL is currently developing plasmid calibration standards under the remit of the Defra-funded project Plasmid Standards for Real Time PCR and UKAS Accreditation of GM Enforcement Testing. 2.4 EU recommended methodology and detection limits The following sections summarise and discuss publications that are part of, or have arisen from, European Commission regulations on GMO release and monitoring. In particular, how issues arising from these publications impact on the statistics of GMO detection methodologies for OSR. To date, only methods that use PCR to amplify and detect GM DNA are available for OSR. This review does not aim to examine the quantification of GM DNA as performed by real-time PCR or sequential seed bulk tests although the properties of these methods in GM detection are discussed Background EU legislation Two articles of EU legislation are of particular importance to statistical issues in the detection of OSR: Directive 2001/18/EC [36] details the procedures required in Member States for the authorisation of release and marketing of GMOs and their appropriate labelling. In conjunction with this directive, the Scientific Committee on Plants (SCP) published an opinion [37] which addresses several questions in the remit of the 2001/18/EC Committee. Most pertinent of these is the opinion on 'zero tolerance'in which the SCP states that for routine analysis and reasons of the limit of analytical sensitivity, the detectable threshold for GMOs should be 0.1%. Following this opinion, and sampling constraints as shown below, 0.1% has been adopted widely as the limit of detection (LOD) for analytical methods. 24

26 Regulation 1830/2003 [21] describes Member States obligations for traceability and labelling of GMOs. The regulation requires the development of EU guidance on sampling and detection (see section 2.2.5) [3]. The guidance gives sampling and analytical protocols for food products and grain commodities with particular reference to methods recommended by the European Commission s Joint Research Centre (JRC). Included are specific recommendations for sample sizes and procedures but it does not give minimum requirements for LOD or analytical method performance other than the requirement that the JRC should approve such methods. Minimum performance requirements for sampling and detection are therefore not set by this legislation. Specific EU legislation on GMOs in seeds does not yet exist. However, following the SCP opinion on GMO thresholds in seeds [37], a committee (Working Group on Seed Legislation, Sampling and Detection) has written draft recommendations for sampling and detection of GMOs in seeds. Currently these recommendations follow ISTA rules for sampling [2] and statistics [28]. ISTA rules do not currently specifically address adventitious presence of GM seed, however the rules and statistics applying to 'other seeds content'have been used as these satisfy the requirements for GM seed detection. Under the draft seed recommendations a minimum sample size of 3000 seeds is required. Following the binomial distribution, this ensures at least 95% probability that the sample will contain GM if the 'real'level in the lot is at least 0.1%. However, it should be noted that this does not mean that there is a 95% probability that the level in the 3000 seed sample is 0.1%, only that there is at least 1 GM seed in the sample. To ensure 95% confidence of detection in the sample, the analytical method should therefore display a 100% detection rate for the presence of 0.033% GM DNA (1 seed in 3000) or better. Different testing plans are under discussion for inclusion in the recommendations for sampling and detection of GMOs in seeds. As is shown in the following section, the choice of testing plan can have profound effects on the statistics of GM detection Theoretical detection limits of polymerase chain reaction (PCR) In theory PCR has the ability to detect one copy of a target GM DNA molecule and amplify it exponentially to give a strong positive signal. However, the occurrence of a single target in a PCR test is governed by binomial (or Poisson) probability [28]. In 25

27 order for a test PCR solution to contain at least one target copy of the GM DNA, the mean number of copies must be three for 95% confidence and five for 99% confidence. A common known variable in the PCR test is the total amount of DNA that is added to the reaction. Given the constant haploid genome size for OSR (one genome = pg [38], the total number of haploid genomes in the PCR can be estimated. Table 7 shows the theoretical detection limit for a GM test PCR containing different amounts of test DNA. Table 7: Limits of Detection (LODs) for differing amounts of test DNA (with 95% confidence). The LODs are shown for a homozygous and heterozygous GM event. Theoretical and empirical LODs [39] are shown Total DNA amount in PCR 10 ng 25 ng 50 ng 100 ng Theoretical LOD, homozygous GM Theoretical LOD, heterozygous event Empirical LOD, homozygous GM Empirical LOD, heterozygous event % % % % % 0.03 % % % 0.22 % % % % 0.44 % % % % Hughes and Totten [39] used a parametric regression of PCR test data to estimate PCR sensitivity. In their study, 95 % sensitivity (i.e. 95 % probability of a positive result when >1 copy of target is present) required a mean target copy number of approximately 18. This empirical result greatly increases the theoretical LOD for OSR as shown in Table 7. However, these LODs, while providing a useful guide, are unlikely to be applicable generally to all OSR GM tests. It is advisable that each LOD should be determined individually using thorough methods similar to those of Hughes and Totten [39]. This is because DNA extracts from different plant tissues will contain variable amounts of compounds that inhibit PCR. Different plant tissues will also contain variable amounts of organelle DNA (extranuclear DNA that does not contain GM material but can significantly contribute to the total amount of DNA extracted). 26

28 Using the LODs in Table 7 as a guide, it can be seen that PCR tests may often fail to detect the minimum content in a 3000 seed bulk (0.033 % if homozygous or % if heterozygous). The 3000 seed bulk size is recommended in EC food/feed and seeds testing guidance [3] for test plans that use real-time PCR techniques. The use of smaller bulk sizes and increased numbers of PCR tests, such as the recommended sequential seed tests would therefore be advisable to increase the likelihood of detection of GM at low levels EU recommended methods for PCR tests The EU guidance for food and feed GM testing recommends using the JRC methods database [4]. For OSR, 13 methods are listed [31], [32], [33], [34] (Table 8). To date none of these methods have been validated to international criteria or evaluated in ring-trials. The methods detect a variety of GM elements, either individually or in multiplex, but none are event-specific, i.e. identify a unique junction of plant and GM DNA. Table 8: JRC listed methods for GM detection in OSR. For definitions of GM elements, see BATS report [40]. Method, Reference Matrix type GM element LOD Mass of DNA used in PCR (ng) 1. Block et al [31] seed P35S-BAR copies - 2. Block et al [31] seed P35S-BAR Demeke et al [32] seed CTP2 0.1 % Demeke et al [32] seed EPSPS 0.1 % Demeke et al [32] seed P35S 0.1 % Demeke et al [32] seed Pnos 0.1 % Demeke et al [32] seed BXN 0.1 % Demeke et al [32] seed multiplex 0.5 % Zeitler et al [33] seed EPSPS 0.05 % Zeitler et al [33] seed PAT 0.05 % Zeitler et al [33] seed P35S 0.05 % James et al [34] leaf multiplex James et al [34] leaf NPT - 75 Variable LODs are reported for the methods in Table 8. Block et al. [31] give only a theoretical LOD in terms of estimated target copy numbers in the PCR. Other methods report the lowest GM level tested which gave positive results as the LOD. It can be seen that multiplex methods i.e. those where more than one GM element is 27