Setting Specifications Statistical considerations Enda Moran Senior Director, Development, Pfizer Melvyn Perry Manager, Statistics, Pfizer
Basic Statisticsti ti Population distribution 1.5 (usually unknown). Normal distribution 1.0 described by μ and σ. 0.5 True batch assay Distribution of possible values 0.0 98 99 100 We infer the population from samples by calculating and s. x 1.5 1.5 1.5 1.0 0.5 Sample 1 Average 98.6 True batch assay y Distribution of possible values 1.0 0.5 True batch assay y Sample 2 Average 98.7 1.0 Distribution of possible values 0.5 Sample 3 Average 99.0 True batch assay Distribution of possible values 0.0 98 99 100 0.0 98 99 100 0.0 98 99 100 2
Intervals 30 28 26 24 22 Population average 20 18 16 0 10 20 30 40 50 60 70 80 90 100 100 samples of size 5 taken from a population with an average of 23.0 and a standard deviation of 2.0. The highlighted intervals do not include the population average (there are 6 of them). For a 95% confidence level expect 5 in 100 intervals to NOT include the population average. Usually we calculate just one interval and then act as if the population mean falls within this interval. 3
Intervals Point Estimation The best estimate; eg MEAN Interval Estimation A range which contains the true population parameter or a future observation to a certain degree of confidence. Confidence Interval - The interval to estimate the true population parameter (e.g. the population mean). Prediction Interval - The interval containing the next single response. Tolerance Interval - The interval which contains at least a given proportion of the population. 4
Formulae for Intervals Intervals are defined as: x ± ks Assuming a normal distribution ib ti Confidence (1-α) interval 0.5 1 CI = x ± t α n 1,n 1 2 s Prediction (1-α) interval for m future observations PI 0.5 1 = x ± 1 + t α n 1 n m, 2m 1 s Tolerance interval for confidence (1-α) that proportion (p) is covered TI = x ± 1 2 1 + (1 p) n 2 s 2 χ α 5, n 1 ( n ) 1 z
Process Capability Process capability is a measure of the risk of failing specification. The spread of the data are compared with the width of the specifications. 3s 3s The distance from the mean to the nearest specification relative to half the process width (3s). The index measures actual performance. Which may or may not be on target i.e., centred. P USL x x LSL = min, pk 3s 3 s LSL x LSL x USL x USL 6
Process Capability Ppk and Cpk Ppk should be used as this is the actual risk of failing specification. Cpk is the potential capability for the process when free of shifts and drifts. Random data of mean 10 and SD 1, thus natural span 7 to 13. Added shifts to simulate trends around common average. With specs at 7 and 13 process capability should be unity. dividual Value Ind 15 14 13 12 11 10 9 8 7 6 1 4 7 10 Process Capability of Shifted Pp 0.51 PPL 0.53 PPU 0.50 When data is with trend Ppk less than Cpk due to method of I Chart of Shifted 1 LSL USL calculation of std dev. Process Data Within LS L 7 Overall UCL=13.589 Target * USL 13 Potential (Within) C apability Sample Mean 10.0917 Cp 0.86 Sample N 30 CPL 0.88 StDev (Within) 1.16561 CPU 0.83 StDev (O v erall) 1.95585 Cpk 0.83 O verall C apability _ X=10.092 13 16 19 Observation 22 25 28 LCL=6.595 Observed Performance PPM < LSL 0.00 PPM > USL 100000.00 PPM Total 100000.00 6 8 Exp. Within Performance PPM < LSL 3995.30 PPM > USL 6296.59 PPM Total 10291.88 10 12 Exp. O v erall Performance PPM < LSL 56966.34 PPM > USL 68512.70 PPM Total 125479.03 14 Ppk 0.50 Cpm * Ppk uses sample SD. Ppk less than 1 at 0.5. Cpk uses average moving range SD (same as for control chart limits). Cpk is close to 1 at 0.83. Individual Value I I Chart of Raw 13 UCL=13.038 12 11 10 9 8 7 1 4 7 10 13 16 19 Observation 22 25 28 _ X=10.092 LCL=7.145 Process Capability of Raw LSL Process Data 7 LS L Target * USL 13 Sample Mean 10.0917 Sample N 30 StDev (Within) 0.982187 StDev (O v erall) 1.14225 7 8 9 10 11 12 O bserv ed Performance Exp. Within Performance Exp. O v erall Performance PPM < LSL 3397.73 PPM < LSL 0.00 PPM < LSL 822.51 PPM > USL 5446.84 PPM > USL 0.00 PPM > USL 1533.13 PPM Total 8844.57 PPM Total 0.00 PPM Total 2355.65 USL 13 Within Overall Potential (Within) C apability Cp 1.02 CPL 1.05 CPU 0.99 Cpk 0.99 O verall C apability Pp 0.88 PPL 0.90 PPU 0.85 Ppk 0.85 C pm * When data is without trend Ppk is same as Cpk. Only small differences are seen. Cpk effectively 1 at 0.99. Ppk close to 1 at 0.85. 7
Measurement Uncertainty t Bad parts almost always rejected LSL Good parts almost always passed σ total USL σ measurement Bad parts almost always rejected ±3σ measurement ±3σ measurement The grey areas highlighted represent those parts of the curve with the potential for wrong decisions, or mis-classification. 8
Misclassification with less variable process σ total LSL USL σmeasurement ±3σ measurement ±3σ measurement 9
Measurement Uncertainty t Precision to Tolerance Ratio: How much of the tolerance is taken up by measurement error. This estimate may be appropriate for evaluating how well the measurement system can perform with respect to specifications. Gage R&R (or GRR%) What percent of the total variation is taken up by measurement error (as SD and thus not additive). MS P / T = 6 σ Tolerance Tolerance = USL LSL USL = Upper Spec Limit LSL = Lower Spec Limit σ MS = Std. Dev. of Measurement Sys. σ MS % R & R = 100 σ Total Use Measurement Systems Analysis to assess if the assay method is fit for purpose. It is unwise to have a method where the specification interval is consumed by the measurement variation alone. 10
Specification example Tl Tolerance It Interval Data from three sites used to set specifications. Tolerance interval found from pooled data of 253 batches. 16.5 16.0 15.5 15.0 Tolerance interval chosen as 95% probability that mean ± 3 standard deviations are contained. 14.5 14.0 13.5 Sample size: n=253 Mean ± 3s Tolerance interval 13.0 Code 1 2 3 Mean = 14.77 (95% / N k multiplier 99.7%) 5 6.60 s=0.58 10 4.44 Range 13.03-16.51 12.89-16.65 65 15 389 3.89 30 3.35 2.58 If sample size was smaller, difference between these calculations increases. Table of values for 95% probability of interval containing 99% of population values. Note at value is 2.58 which is the z value for 99% coverage of a normal population As the sample size approaches infinity the TI approaches mean ± 3s. 11
Specification Example - Stability Batch history Total SD (process and measurement) Stability batch with SD (measurement) Shelf life set from 95% CI on slope from three clinical batches. Need to find release criteria for high probability of production batches meeting shelf life based on individual results being less than specification. Review of batch history will lead to a process capability statement t t against release limit. 12
Specification Example - Stability Release limit is calculated from the rate of change and includes the uncertainty in the slope estimate (rate of change of parameter) and the measurement variation). 8 7 6 RL = Spec - Tm + t T 2 s 2 m 2 s a + n 5 4 Specification = 8 3 Shelf life (T) = 24 Parameter slope (m) = 0.04726 2 Variation in slope (s m ) = 0.00722 Variation around line (s a ) = 0.86812 1 t on 126 df approx. = 2 N= 1 for single determination 0 Release limit = 5.1 0 6 12 18 24 30 36 MONTHS A similar approach can be taken in more complex situations; e.g,, after reconstitution of lyophilised product. Product might slowly change during cold storage and then rapidly change on reconstitution. Allen et al., (1991) Pharmaceutical Research, 8, 9, p1210-1213 13
Coping with limited it data Problems with n<30. Difficult to calculate a reliable estimate of the SD. Difficult to assess distribution. Difficult to assess process stability. With very limited data the specifications will be wide due to the large multiplier. This reflects the uncertainty in the estimates of the mean and SD. Is there a small scale model that matches full production scale? How variable is the measurement system? Alternatives are: Min / max values based on scientific rationalisation Mean ± 3s Mean ± 4s or minimum Ppk = 1.33. Tolerance interval approach Example with 4 values N(20,4) 19.9, 9 16.9, 22.1, 21.3 Mean ± 3s 13.2 27.0 Mean ± 4s 10.9 29.3 TI (95/99.7) -1.5 41.7 14
Updating Specifications When, why, and a scenario for consideration Specifications for CQAs might require revision during the product lifecycle. For example, new clinical data and intelligence on continued use of a product might drive a change to a CQA specification Scenario: What if a process producing a new product approaching regulatory approval is incapable of meeting the clinicallytested CQA in the long-term? For example... CQA Proposed Spec. Process capability estimate Clinical experience If spec. required to be set at the clinicallytested level... CQA Proposed Spec. Process capability estimate Clinical experience / New spec. Clinical / Validation Lots Clinical / Validation Lots An incapable process How could we manage this situation? Complex. Many factors need to be considered if setting specifications at clinically tested level (will be discussed through this workshop) IF it must be done for risk reduction purposes, and if there is sufficient uncertainty that the Proposed Spec. for the CQA could cause harm, a possible approach may be to move towards the clinically-tested level in stages. - Accrue approx. n=25-30 batches/lots to consider estimates of future process capability acceptable. Adjust spec. to a revised process capability estimate. - Accrue further lots of the stabilised process to approx. n=85. Make FINAL spec. adjustment to a revised process capability estimate. 15
Summary Process capability & measurement uncertainty Setting spec.s - large number of datapoints Setting spec.s small number of datapoints 16