Novel data analysis method for Continued Process Verification using change-point analysis

Transcription

1 Novel data analysis method for Continued Process Verification using change-point analysis Jochen Giese, Bernhard Pasenow-Grün, Walter Hoyer, Pablo Serrano Fernandez, Roman Pahl, Alexander Pysik Statistics & Mathematical Modelling (GSK Vaccines) nd Statistical and Data Management Approaches for Biotechnology Drug Development September 015

2 Comment All data and plots shown on the following slides are based on simulations and are not real manufacturing data. Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015

3 Overview Introduction FDA 1 Guidance of Process Validation Internal requirements for CPV CPV implementation at GSK (Marburg/Siena) Statistical methods used for CPV Control charts Change-point analysis Combined change-point / control chart / process capability analysis Summary References 1: US Food and Drug Administration : Continued Process Verification Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 3

4 FDA Guidance for Process Validation An ongoing program to collect and analyze product and process data that relate to product quality must be established. [...] Production data should be collected to evaluate process stability and capability Process validation includes three stages: Stage 1 Process Design The commercial manufacturing process is defined based on knowledge gained through development and scale-up activities. Stage Process Qualification The original design is evaluated to determine if the process is capable of reproducible commercial manufacturing. Stage 3 Continued Process Verification Ongoing assurance is gained during routine production that the process remains in a state of control Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 4

5 FDA Guidance for Process Validation An ongoing program to collect and analyze product and process data that relate to product quality must be established. [...] Production data should be collected to evaluate process stability and capability Process validation includes three stages: Stage 1 Process Design The commercial manufacturing process is defined based on knowledge gained through development and scale-up activities. Stage Process Qualification The original design is evaluated to determine if the process is capable of reproducible commercial manufacturing. Stage 3 Continued Process Verification Ongoing assurance is gained during routine production that the process remains in a state of control Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 5

6 Continued Process Verification CPV process Risk analysis to identify critical process parameters Interpretation of results Required actions Data collection Ensure data integrity Data analysis e.g. control chart, change-point analysis, process capability Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 6

7 Compiling Data Requirements by internal guidelines Starting a CPV program for legacy products: only output variables are assessed for stability and capability. Monitoring must include: Quality attributes Process yields Intra batch data (where possible) Complete CPV program must include output and input variables based on a risk assessment, especially including: Critical process parameters Critical IPC (in process control) data Critical raw material data Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 7

8 Data Evaluation Requirements by internal guidelines Process stability and capability have to be assessed (e.g. by control charts, Cpk, Ppk) Special causes (OOE/OOT/OOS/OOCL) have to be investigated Process performance levels (good / acceptable / poor) must be defined according to process capability / process stability in order to facilitate decisions Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 8

9 CPV Implementation status GSK (Marburg/Siena) Monthly CPV meetings for all commercial products in place (e.g. Menveo, Bexsero, RabAvert, Encepur) CPV analyses harmonized cross-sites by use of validated in-house software RAS (RedStar Analysis Software) automatized analysis for more than 30 products more than 1000 parameters included longterm data available ( ) longterm storage and reproducibility of analyses guaranteed full product update within less than 10 min Assurance of highest level of data integrity (e.g. manufacturing data base gredstar for non-release parameters) CPV phase started (e.g. raw materials, new data sources, multi-variate analyses) Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 9

10 Implementation of CPV Production Process collect process data Database External Data sources Optimize / control process Root cause analysis e.g. for changepoint/outliers Action items Continuous review of data analysis plots within the segment (e.g. weekly) monthly meetings for detailed data review and checking action items Regular Trending Report Shared Drive / Sharepoint Data Plots Change-Point Analysis Process Capability Control charts R Analytical Software EXCEL Report APR Track action items by SENTRY events 3-monthly Manufacturing Robustness Review Board (MRRB) Management IQP Leader, Process Expert Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

11 Implementation of CPV Data collection Production Process collect process data Database External Data sources Optimize / control process Root cause analysis e.g. for changepoint/outliers Action items Continuous review of data analysis plots within the segment (e.g. weekly) monthly meetings for detailed data review and checking action items Regular Trending Report Shared Drive / Sharepoint Data Plots Change-Point Analysis Process Capability Control charts R Analytical Software EXCEL Report APR Track action items by SENTRY events 3-monthly Manufacturing Robustness Review Board (MRRB) Management IQP Leader, Process Expert Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

12 Implementation of CPV Data analysis Production Process collect process data Database External Data sources Optimize / control process Root cause analysis e.g. for changepoint/outliers Action items Continuous review of data analysis plots within the segment (e.g. weekly) monthly meetings for detailed data review and checking action items Regular Trending Report Shared Drive / Sharepoint Data Plots Change-Point Analysis Process Capability Control charts R Analytical Software EXCEL Report APR Track action items by SENTRY events 3-monthly Manufacturing Robustness Review Board (MRRB) Management IQP Leader, Process Expert Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 1

13 Implementation of CPV Review / discussion Production Process collect process data Database External Data sources Optimize / control process Root cause analysis e.g. for changepoint/outliers Action items Continuous review of data analysis plots within the segment (e.g. weekly) monthly meetings for detailed data review and checking action items Regular Trending Report Shared Drive / Sharepoint Data Plots Change-Point Analysis Process Capability Control charts R Analytical Software EXCEL Report APR Track action items by SENTRY events 3-monthly Manufacturing Robustness Review Board (MRRB) Management IQP Leader, Process Expert Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

14 Implementation of CPV Take actions and decisions Production Process collect process data Database External Data sources Optimize / control process Root cause analysis e.g. for changepoint/outliers Action items Continuous review of data analysis plots within the segment (e.g. weekly) monthly meetings for detailed data review and checking action items Regular Trending Report Shared Drive / Sharepoint Data Plots Change-Point Analysis Process Capability Control charts R Analytical Software EXCEL Report APR Track action items by SENTRY events 3-monthly Manufacturing Robustness Review Board (MRRB) Management IQP Leader, Process Expert Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

15 Trending Meetings monthly meetings with all functions Collecting information from all functions involved in the process Biostatistics Validation Quality Assurance Quality Control MSAT Manufacturing Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

16 Trending Meetings monthly meetings with all functions Any root cause for the changepoint? Maybe a change in the raw material batch? I will check this now in our production list! Validation Biostatistics MSAT Manufacturing Quality Control Quality Assurance Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

17 Protein Content [mg/ml] Control Charts No tool for (automated) longterm analysis Control charts are suitable for short term analysis and/or data without change-point I Chart of Protein Content 8 7 UCL=7, _ X=6,001 no change-point within the data 5 LCL=4, Observation Minitab Graph 1: outlier : shift 3: trend Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

18 Protein Content [mg/ml] Control Charts No tool for (automated) longterm analysis Many type Shewhart rule violations due to change-points. I Chart of Protein Content UCL=7,114 _ X=5,951 LCL=4,787 change-point 1 + standard deviations change-point - standard deviations Observation Minitab Graph 1: outlier : shift 3: trend Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

19 Protein Content [mg/ml] Control Charts No tool for (automated) longterm analysis Effects within run violation data are hidden. Data could be split manually for further analysis. I Chart of Protein Content Is this really an outlier? No! UCL=7,114 _ X=5,951 LCL=4,787 change-point 1 + standard deviations change-point - standard deviations Observation Minitab Graph 1: outlier : shift 3: trend Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

20 Change-Point Control Charts Tool for detecting systematic mean shifts Change-point analysis: systematic shifts automatically detected RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 0

21 Change Point Analysis How does it work? (Weighted) difference in mean is compared to null distribution. The null distribution is estimated based on random permutations of given data. _ x _ x non optimal change point candidate Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 1

22 Change Point Analysis How does it work? (Weighted) difference in mean is compared to null distribution. The null distribution is estimated based on random permutations of given data. _ x _ x optimal changepoint candidate with maximum difference in mean Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015

23 Change Point Analysis How does it work? Let X 1,, X n independent random variables with mean µ i and standard deviation σ. Change-point analysis for a single change-point tests the hypothesis that the mean of the random variables is constant until the end of the observed time period against the alternative that there is a change-point where the mean changes. H 0 : µ 1 = µ = = µ n H A : 1 < k < n: µ 1 = = µ k µ k +1 = = µ n T n k = k(n k) n T n = max 1<k<n T n k 1 k k i=1 X i 1 n k n k+1 X i H 0 is rejected if T n > αth quantile of the null distribution, which is estimated based on T n calculated for a number of permutations of observed dats (e.g permutations). Location of the change-point k* such that T n = max 1<k<n T n k = T n k Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 3

24 Change Point Analysis How does it work? The algorithm applies the test recursively until no more changes are detected in the segments that are already found. Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 4

25 Change Point Analysis Properties Enables to detect automatically systematic shifts in time series data Applicable for not normally distributed data Good power for symetrically distributed data power decreases when data is highly skewed For highly skewed data a transformation is recommended Change-point analysis can be combined with control charts and process capability analysis. Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 5

26 Change-Point Control Charts Data without change-point RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 6

27 Change-Point Control Charts Data without change-point Specifications and control limits are shown. USL mean + 3 std. dev. mean mean - 3 std. dev. LSL RAS Graph Novel data analysis method for CPV using CPA 015 Statistical and Data Management Approaches for Biotechnology September 7

28 Change-Point Control Charts Data without change-point Process capability is evaluated. good Ppk 1.5 RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 8

29 Change-Point Control Charts Data without change-point Outlier values are detected. OOE RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 9

30 Change-Point Control Charts Data with change-point change-point 1 + standard deviations change-point - standard deviations RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

31 Change-Point Control Charts Data with change-point Control limits for every change-point segment separately. change-point 1 + standard deviations change-point - standard deviations RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

32 Change-Point Control Charts Data with change-point OOS value due to outlier value (single event). acceptable Ppk 1.0 change-point 1 + standard deviations change-point - standard deviations OOS RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 3

33 Change-Point Control Charts Data with change-point change-point standard deviations change-point - standard deviations RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

34 Change-Point Control Charts Data with change-point OOS value due to low Ppk (collective event). OOS poor Ppk < 1.0 change-point standard deviations change-point - standard deviations RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

35 Change-Point Control Charts Defining action items Approach for trending meetings: Following events should at least be commented and might trigger an action item: Significant change-points with a difference of more than three standard deviations between maximum and minimum level A Ppk < 1 (separate analysis for each change-point level) (e.g. analysis of source of variability: process/measurement system). A result outside ± 3 standard deviations from the respective change-point level (possible action item: review of BPR). The decision whether an action item is defined has to be taken within regular trending meetings. Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

36 Change-Point Control Charts Data with change-point (CPV phase ) Plots colored by raw material: identify root cause for CP CP not related to change in raw material R R1 R R3 No CP due to change in raw material R CP due to change in raw material R RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

37 Change-Point Control Charts Data with change-point (CPV phase ) Plots colored by raw materials: identify root cause for CP CP due to change in raw material S R1 & S1 No CP due to change in raw material R CP due to change in raw material R R & S1 R3 & S1 R3 & S RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

38 Box Plots Data with change-point (CPV phase ) Plots colored by raw materials: corresponding box plot RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

39 Scaled Variance Rising Sun Plots Combined with change-point analysis Scaled Drift from Target RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

40 Scaled Variance Change-Point Control Charts Combined with change-point analysis Scaled Drift from Target RAS Graph Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

41 CPV analysis overview Software implementation GSK in-house CPV software RAS: validated software based on statistical software R automatized long term analysis for all commercial products further improvements planned (e.g. correlation or multivariate analysis) Change-point analysis to detect long-term changes in process data process stability Control charts for detecting short-term changes in process data Data plots for process overview Descriptive statistics like mean, median, standard deviation Process capability analysis Process overview tables Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September

42 Summary CPV concept based on: Advanced statistical methods Regular meetings including all functions involved in the process. Integrated concept of data analysis including control charts, process capability analysis and change-point analysis. Long-term analysis based on change-point analysis offers deep insights into the process. Short term analysis based on Shewhart run charts Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September 015 4

43 References Novel data analysis method for CPV using CPA Statistical and Data Management Approaches for Biotechnology September