How to Conduct the Method Validation with a New IPT (In-Process Testing) Method Weifeng Frank Zhang QA Engineer BMS Ernest Lee Senior Manager, Facilities & Engineering Medarex, a fully owned subsidiary of BMS IVT Method Validation, San Francisco, July 30, 2010 Forward Looking Statements Except for historical information, the matters contained in this slide presentation may constitute forward-looking statements that involve risks and uncertainties, including uncertainties related to product development and clinical trials, unforeseen safety issues resulting from the administration of antibody products in patients, uncertainties related to the need for regulatory and other government approvals, dependence on patents and proprietary technology, the need for additional capital, uncertainty of market acceptance of Medarex s product candidates, the receipt of future payments, the continuation of business partnerships and other risks detailed from time to time in Medarex s filings with the Securities and Exchange Commission (SEC). All forward-looking statements included in this slide presentation are based on information available to us as of July 30, 2010. We do not assume any obligation to update any information contained in these materials. Our actual results may differ materially from the results discussed in the forward-looking statements. 1
Disclaimer: All data in this presentation is not from actual tests. Virtual data is used in this presentation for confidentiality reasons. Weifeng Frank Zhang B.Sc. (Materials Eng.), M.Eng. (Chem. Eng.), 12 years in pharmaceutical, biotech Ernest Lee B.Eng. (Mechanical Eng.), M.Sc. (Eng. Management), 20+ years in pharmaceutical, biotech 2
AGENDA Overview of method validation and IPT New automated method Validation approach Q&A, Open Discussion OVERVIEW Method validation elements Accuracy Specificity Precision Suitability Linearity Repeatability Limit of Detection Limit of Quantitation 3
Things to Consider in Method Validation Parameters Applicable to Different Analytical Procedures: From: Method Validation Guidelines, Sep 15, 2005 By: Alex D. Kanarek, Ph.D. BioPharm International IPT in This Study Cedex Automated Method IPT (In-Process Test) The most important parameters Total cell counts Viability 4
Cedex Description Photos from: http://www.innovatis.com IPT in This Study Cedex Description Counts cells and determines viability based on the Trypan Blue Exclusion method Takes 20 digital images of the fluid as it passes through a flow cell Evaluate light level differences between the background and the discovered cells Counts Cells are included or excluded from the count based on relative size Viability Shading differences between the cells are utilized to assess viability 5
IPT in This Study Dynamic Data In bio-production, every day, every hour, every minute counts! Accounting for Human Factors Variance User-dependent Not 100% reliable 6
New Automated Method Manual vs. Automated (human factors in here?) Manual method uses a microscope and a hemacytometer after staining the cells with Trypan blue After study and practice, develop SOPs to minimum the human variance in the manual measurement Challenges Different testing instruments between two methods No Gold Standard Time-intensive 7
Validation Acceptance Criteria IOQ and PQ for instruments have been successfully performed Identify parameters No significant difference between new automated method and existing manual method Validation Approach Design validation experiments We will have to consider all factors! 8
Accounting for All Variance Formula: 2 σ Where, σ is total variance, = σ + σ 2 1 2 2 +... In manual method, σ1 is sample variance, σ2 is human factor variance; In automated method, σ1 is sample variance, σ2 is instrument variance; We need to make sure sample variance is minimized between methods! Sampling Matrix Design Day # Sample Taken Bioreactor Manual Cedex 0 Two samples taken at the same time. Time 1 Two samples taken at the same time. Time 2 Two samples taken at the same time. Time 3 Two samples taken at the same time. Time #1 Operator 1 #1 Operator 2 #1 Operator 3 #1 Operator 1 Cedex log 1 Cedex log 2 Cedex log 3 Cedex log 4 #1 n Two samples taken at the same time. Time #1 Operator 3 Cedex log n 9
Raw Data Cedex\Method Validation\Cedex Method Validation Analysis -- Final.xls Data Overview Comparison Between Two Methods Viale Cell Counts (x10 6 cell/ml) 18.00 16.00 14.00 12.00 10.00 8.00 6.00 4.00 2.00 0.00 0 10 20 30 40 50 60 Sample Runs Manual Method Auotmated Method 10
Statistical Since viable cell counts base changes on different days, suggesting the differences are not directly comparable. So the data was transformed to percent differences to exhibit constant variance over the entire study. Viable Cell Count Difference Between Manual and Auotmated Methods Viable Cell Count Percent Difference Between Manual and Automated Method 3.00 40.00% Cell Count Difference (x10 6 cell/ml) 2.50 2.00 1.50 1.00 0.50 0.00 0 1 2 3 4 5 6 7-0.50-1.00 Day 0 Day 4 Day 7 Day 11 Day 14 Day 18 Percent Difference 30.00% 20.00% 10.00% 0.00% 0 1 2 3 4 5 6 7-10.00% Day 0 Day 4 Day 7 Day 11 Day 14 Day 18-1.50-20.00% Days Days Statistical Since there is no Gold Standard in this study, we have to test whether the two methods are different null hypothesis: The measurements from two (2) methods have no significant difference From testing data of two methods, we can calculate the mean and standard of the percent differences. Then construct a 95% confidence interval on the mean Whether the 95% confidence interval includes zero will decide if the null hypothesis is rejected Remember, trade-offs associated with using higher confidence levels 11
Statistical Upper 95% confidence level Mean of data Zero Lower 95% confidence level null hypothesis can be accepted in this case. Zero Upper 95% confidence level Mean of data null hypothesis is rejected in this case Lower 95% confidence level Statistical Null Hypothesis Reject Actual Result Fail To Reject True Type I Error α: Probability of Type I Error (significance level) True False True Type II Error (1-β): Probability of Type II Error 12
Data Analysis Step by Step Step 1 Calculate mean for each day across all bioreactor runs Step 2 Calculate the percent difference in cell count Step 3 Calculate the mean and standard deviation for the percent difference in cell count Step 4 Calculate the upper and lower 95% confidence intervals Step 5 Verify that 0 falls within the upper and lower 95% confidence intervals Theory Behind Validation Approach ANOVA Test A collection of statistical models. It provides a statistical test of whether or not the means of several groups are all equal. 13
ANALYSIS OF VARIANCE ANOVA Test AGITATOR (P) GROWTH PLATEAU DEATH (ph, T, PO 2) M SS ERROR + SS TREATMENT (rpm) C avg (r, z, φ, t) BIOREACTOR SS TOTAL M = f ( t, ε(t)) M : Measurand (Cell Concentrations) (Viable and Non-Viable) H 0 : μ 1 = μ 2 H 1 : μ 1 μ 2 ANALYSIS OF VARIANCE ANOVA Test X MEAN = 4 SS = (X i X MEAN ) 2 AGITATOR (P) GROWTH PLATEAU DEATH (ph, T, PO 2) M SS ERROR + SS TREATMENT (rpm) C avg (r, z, φ, t) BIOREACTOR SS TOTAL M = f ( t, ε(t)) M : Measurand (Cell Concentrations) (Viable and Non-Viable) H 0 : μ 1 = μ 2 H 1 : μ 1 μ 2 14
ANALYSIS OF VARIANCE ANOVA Test REPLICATION HEMACYTOMETER / HUMAN SYSTEM MICROSCOPE LINEARITY AGITATOR RANDOMIZATION (TRYPAN BLUE) C 11 C 12 REPEATABILITY LIMIT OF DETECTION LIMIT OF QUANTITATION (P) C 13 GROWTH PLATEAU DEATH (ph, T, PO 2) M ε 0 MSERROR MS TREATMENT (rpm) C avg (r, z, φ, t) (TRYPAN BLUE) BIOREACTOR C 21 SENSOR M = f ( t, ε(t)) C 22 μp C 23 M : Measurand (Cell Concentrations) (Viable and Non-Viable) MS T = (c ij x) 2 /(Degrees of Freedom) SAMPLING / DILUTION / DIVISION H 0 : μ 1 = μ 2 H 1 : μ 1 μ 2 AUTOMATED CELL COUNTER SYSTEM F < F CRITICAL Summary Resolved the challenge of testing in the absence of a standard Designed experiments in validation protocol Used statistical method to perform data analysis Established acceptance criteria based on 95% confidence interval 15
Special Thanks to: David Lorah Supervisor, Validation Department Lance Marquardt Senior Manager, Biopharm Production Department John R. Mosack Senior Director, Clinical Manufacturing & Validation Interactive Exercise Application discussion 16
Q &A Open item 707 State Road Princeton, NJ 08540 T: +1-609-430-2880 F: +1-609-430-2850 www.medarex.com 17