Monitoring is considered a quality control (QC) process (ongoing activities

Peer Reviewed Describing a drug development program initiated using a quality-by-design approach, with special attention paid to assuring proactive quality planning at all stages of development. Jules Mitchel, MBA, PhD Dean Gittleman, MS Judith M. Schloss Markowitz, MS Laura Suciu, MS Michael R. Hamrell, PhD, RAC, FRAPS, RQAP-GCP, CCRA A 21st Century Approach to QA Oversight of Clinical Trial Performance and Clinical Data Integrity Monitoring is considered a quality control (QC) process (ongoing activities to assure quality), whereas audits are considered quality assurance (QA). QA procedures are not typically an integral part of a drug development program, but encompass a process that occurs during or after the fact. Thus, it is not uncommon for corrective actions to occur too late to allow for meaningful changes, and for trial managers to spend an inordinate amount of time writing notes to file and performing crisis management. In order to address this issue, a drug development program was initiated using a quality-by-design (QbD) approach, recognizing that (a) change will be difficult to implement and (b) often quality departments are feared and have powerful roles in companies. Paying heed to such concerns, an independent quality expert was assigned to the development team with the mandate to assure proactive and upfront quality planning at all stages of development. As described in this article, since all of the program s data and documents were collected over the Internet and often in real-time, clinical site oversight, clinical monitoring, data management, safety alerts, and quality issues could be addressed rapidly and expeditiously. Finally, site and contract research organization (CRO) audits confirmed the feasibility of the QbD approach. Background In section 1.46 of the International Conference on Harmonization (ICH) guidelines, QA is defined as All those planned and systematic actions that are established to ensure that the trial is performed and the data are generated, documented (recorded), and reported in compliance with Good Clinical Practice (GCP) and the applicable regulatory requirement(s). Section 1.47 defines QC as The operational techniques and activities undertaken within the quality assurance system to verify that the requirements for quality of the trial-related activities have been fulfilled. 1 As already mentioned, QA procedures are not typically an integral and ongoing part of a clinical development program, but rather processes implemented based on general standard operating procedures (not on actual risks to a clinical trial). For example, audits are often performed only at relatively high-enrolling sites toward the end of a trial without any other rationale. Also, QC and QA procedures, such as frequent monitoring and extensive audits, can place undue burdens on compliance with processes rather than Risk-based monitoring 2013 ACRP Peer Reviewed x 41

compliance with the protocol-specified content of a clinical trial. As a result, critical-to-quality (CTQ) parameters are often managed with the same priority and intensity as parameters carrying lesser impact. It is not uncommon for corrective actions to occur too late to allow for meaningful changes. In addition, risk assessments and risk mitigation strategies are not commonplace. Rather than assuring that the protocol is being followed and the proper data are being collected, industry practices continue to focus on the quality of data transcription from source to electronic data capture (EDC), or on subordinate procedural forms, which may not shed light on overall site adherence to the protocol or on site competencies. Although maintaining regulatory compliance is important, it is most critical that plans are put in place to assure that the protocol is being followed and that site personnel are qualified, trained, and knowledgeable on the protocol. As described later in this article, such plans can help to move research teams away from a status quo where it is common for corrective actions to occur too late to allow for meaningful change. An Array of Approaches In 2011, Ball and Meeker-O Connell 2 presented the U.S. Food and Drug Administration s (FDA s) and other regulatory agencies approaches to QbD and quality risk management (QRM). Their article describes the forces for change in clinical research, highlighting the significant consequences of noncompliance, and outlining key attributes of QbD and QRM approaches to clinical drug development. According to Bhatt, 3 compliance to quality requirements is the cornerstone of a scientifically valid and ethically sound clinical trial, and data integrity and subject protection can be ensured by a systematic approach that demands continuous vigilance and continuous process improvement from the key stakeholders, including investigators and sponsors. Elsewhere, Sprenger et al. 4 presented the results of a pilot collaboration with Pfizer Inc. and the FDA to create an integrated quality management plan (IQMP). The IQMP approach to quality was founded on three main principles: 1. Quality is built in at the time of protocol development and systematically managed during study conduct through a process of continuous improvement. 2. Quality goals and relevant quality metrics are prospectively identified and measured throughout the duration of the study. 3. Risks to quality are prospectively identified, prioritized, and mitigated. Before any of the above approaches had been promoted, however, in 2010 a drug development program was initiated by a CRO on behalf of a sponsor using a QbD approach. The overall development program comprised six studies, and included a Phase III pivotal trial involving 18 study sites. The methodology included risk-based monitoring 5,6 and esource methodologies. 7 10 In addition, an initial quality meeting was held, at which potential study risks were identified prospectively and steps implemented in the monitoring and QA plan to minimize their potential impact on data integrity and data quality. At the beginning of the program, the CRO s study team tried to anticipate challenges associated with implementing change both within its own organization and that of the sponsor, as well as at the investigative sites. The team also anticipated resistance from the corporate QA function to instituting changes to its approach to QA, as many of the traditional procedures had been considered tried and true. Methods All data and documents from a single clinical trial were collected over the Internet, with the vast majority of data collected at the time of each subject s office visit. This real-time data capture enabled near real-time clinical trial oversight by the clinical research associates (CRAs) and the site staff. The CRO s own electronic case report form (ecrf) product was used for EDC, and was fully integrated with its electronic clinical trial record product to allow for an esource methodology. 11 In addition, another of the CRO s products was used for the electronic trial master file (etmf). As a result, monitoring, data management, quality, and compliance issues could be rapidly identified and addressed. At the beginning of the program, the CRO s study team tried to anticipate challenges associated with implementing change both within its own organization and that of the sponsor, as well as at the investigative sites. Each protocol included a detailed quality section on the study team s approach to quality management. In addition, the study teams, which included members from the clinical research, data management, regulatory affairs, and corporate QA units, agreed upon the clinical data monitoring plan. In addition to the standard onsite features, the monitoring plan contained sections on targeted source data verification (SDV) and central monitoring, as well as a section on risk management that included management of risks to the study, risks to subject safety, and risks to both. Each 42 x Monitor December 2013 2013 ACRP

risk was assigned a total risk score from 1 to 6 by multiplying the probability score (1-3) and the severity score (1-3), where 1 = low, 2 = medium, and 3 = high. In addition, for each risk, a risk mitigation strategy was developed (see Table 1). From the outset, the project team retained an independent quality expert to be part of the development team, with the mandate to assure proactive and up-front quality planning at all stages of development. The expert critically reviewed and signed off on each protocol and clinical data monitoring plan, and participated in weekly QbD meetings, which included team members from clinical research, data management, and the sponsor. Later in the development process, a member from the biostatistics team also joined the meetings. Minutes for each meeting were generated and maintained in the etmf, and quality site audits were performed by both the project QA expert as well as by corporate QA. Table 1 Example of a Risk Mitigation Strategy Category Risk Severity Probability Trial outcome/subject Site staff misunderstanding of the protocol Trial outcome Trial outcome Patient dosing compliance Blood storage and processing at trial site Total Risk Score Risk Mitigation high medium 6 Training, reinitiate the site, assess the site after three patients treated, investigator meeting high medium 6 Instruction for user, site training prior to study, weighing of drug supply, monitoring by CRO high medium 6 Onsite monitoring Trial outcome Missing blood draws high medium 6 Monitoring, training at interim visit immediately prior to first patient day 90, online reports Trial outcome Patient dropouts high medium 6 Training and evaluating and resolving reasons for dropout, phone alerts prompted by ecrf, online reports Trial outcome/subject Patient in another clinical trial medium medium 4 Use of Verifiedclinical.com software Trial outcome Bioanalytical errors from CRO high low 3 Auditing and review of data analysis turnaround, communication plan Trial outcome Loss of esource data high low 3 Multiple daily backup and disaster recovery Trial outcome/subject Improper enrollment of ineligible patients high low 3 Central review and sign-off patients by CRO when site confirms eligibility, disallow rescreening, review of medical record Trial outcome/subject Improper reporting of adverse events high low 3 Monitoring and review of patient records and medical record, online reports Trial outcome Fraud and misconduct medium low 2 Monitoring Trial outcome/subject Trial outcome/subject Increase in abnormal safety lab test results Improper storage of drug supply medium low 2 Monitoring by investigator and daily by CRO low medium 2 Onsite monitoring Trial outcome/subject CRO staff qualifications medium low 2 Sponsor oversight Subject Overdose low low 1 Instruction for user, site training prior to study, drug accountability, online reports Trial outcome/subject Skin reactions low low 1 Monitoring by investigator and daily by CRO, online reports Trial outcome/subject Use of prohibited medications low low 1 Monitoring review of medical record, online reports Trial outcome/subject Site out of business low low 1 Contingency plan and proper qualification visit 2013 ACRP Peer Reviewed x 43

Results QbD meetings occurred weekly for the first three months of the Phase III study, typically over the course of an hour. At each meeting, overall protocol and quality issues were discussed, as well as the status of data review and query resolution, the types of queries that were being generated and the responses to them, frequency and types of auto queries being generated, subject safety, compliant use of the study drug, enrollment status, and a review of all CTQ variables and issues. For the study, 90% of the clinical trial data were entered at the time of the subject s office visit; 75% of the data were reviewed remotely within seven hours; and 90% of the data reviewed within three days. A total of 3.8% of completed CRFs were queried, and 37% of those were modified, representing just 1.4% of all forms, with no effect on the study results. Over the course of the study, the query rate and changes to the database stabilized, in part because the ecrfs, edit checks, and studyspecific reports were modified to address observations in real time, and the sites and CRAs were retrained based on the findings. Over the course of the study, the query rate and changes to the database stabilized, in part because the ecrfs, edit checks, and study-specific reports were modified to address observations in real time, and the sites and CRAs were retrained based on the findings. The IQMP reports allowed for an assessment of findings across sites, and thus for the identification of outliers to any observation. For example, it became apparent that one site was not weighing the drug container properly just by observing the site s ecrf against all other sites. A total of 18 onsite monitoring visits were conducted over the ninemonth course of the study. Over the same period of time, the team performed ongoing centralized monitoring and generated 211 central monitoring reports, initially on a weekly basis. Approximately 25% of entered forms were subject to SDV, which resulted in less than 0.9% of the data being changed as a result of SDV findings (see Table 2). None of these changes affected the study results, data integrity, or quality. The vast majority of onsite and central monitoring reports were reviewed online by the project manager within one day of the report being posted, and the remaining reports were reviewed within five days of posting. This allowed the team to respond promptly and proactively to issues that required action. The independent auditor and corporate QA performed a combined total of nine onsite audit visits four by Table 2 Database Changes as a Result of SDV Form # of Forms # of Forms SDV d # of Forms Submitted (%) Modified (%) No Change Visit Date 2,940 547 (19%) 0 (0.0%) Drug Dispensing 2,579 101 (4%) 0 (0.0%) Questionnaires 1,676 32 (2%) 0 (0.0%) Lab Sample Collection 870 111 (13%) 0 (0.0%) Subject Contacts 881 75 (9%) 0 (0.0%) General Comments 633 4 (1%) 0 (0.0%) Prior Drug Administration 526 5 (1%) 0 (0.0%) Vital Signs 501 11 (2%) 0 (0.0%) Protocol Deviations 266 7 (3%) 0 (0.0%) End of Trial 180 19 (11%) 0 (0.0%) Sample Shipments 98 14 (14%) 0 (0.0%) >0%<1% Medical History 4,279 1,197 (28%) 10 (0.8%) Blood Sampling 3,019 746 (25%) 3 (0.4%) Outcome Results 2,572 1,326 (52%) 11 (0.8%) Demographics 656 238 (36%) 1 (0.4%) Eligibility 656 215 (33%) 2 (0.9%) >1%<5% Medications 2,614 530 (20%) 13 (2.5%) Informed Consent 656 328 (50%) 4 (1.2%) Body Measurement 648 26 (4%) 1 (3.8%) >5% Drug Administration 893 15 (2%) 1 (6.7%) Physical Examination 500 18 (4%) 1 (5.5%) Adverse Event 316 16 (5%) 1 (6.3%) TOTAL 27,959 5,581 (20%) 48 (0.9%) 44 x Monitor December 2013 2013 ACRP

the independent QA expert and five by corporate QA. Corporate QA listed a number of findings as major; however, in the opinion of the clinical team and independent auditor, most of these observations would have had no effect on data integrity or quality and study subject safety. Regardless of their formal classification, these observations did not raise concerns about GCP compliance, the acceptability of the data, the outcome of the study, or subject safety. As examples, the findings included: The investigational product was not stored properly in a separate secured cabinet in the locked drug storage room. This observation was addressed not as an FDA issue, but as one related to the use of a controlled substance. The first version of the informed consent form (ICF) stated that a second blood sample would be collected seven days after the first, when it should have been three days. Although the ICF was not updated immediately, a letter about the error was sent to the institutional review board (IRB) and the sites. An updated amendment was eventually sent to the IRB and a revised ICF generated. It could not be documented that the principal investigator (PI) or a delegated subinvestigator had reviewed and approved the subject eligibility prior to enrollment. Eligibility in this study was very easy to determine, as there were minimal inclusion/exclusion criteria and a subject could not have been deemed eligible to be treated due to restrictions in the EDC system. However, based on this finding, the sites were instructed to indicate in the comment section of the ecrf that the PI agreed that a subject was eligible to be treated. The printouts from the electronic subject record were not complete for one of the subjects; however, every subject s electronic record was available to monitors at each site, even if they were not all printed. The drug supply labels had no line for the sequential bottle number, as required in the protocol. This was a label design issue with no effect on the study quality or data integrity. The sites wrote the bottle number with a felt-tip pen directly on the bottle. Discussion and Conclusion The development program described in this article made use of a number of techniques intended to prospectively address quality-related issues. Among these were the retention of the services of an independent quality expert, prospective and explicit capture of assessment of risks within the monitoring plan, numerous EDC e-mail alerts, retraining of study monitors, and specific qualification questions and initiation training. The study team further leveraged these resources in carrying out near real-time monitoring of the data, as well as through regular periodic team reviews. Despite early and repeated efforts to refocus monitoring activities to exploit central or remote monitoring and maximize the value of dramatically reduced onsite monitoring visits, the team encountered numerous instances where monitors fell back into traditional behaviors, as evidenced, for example, by the number of queries issued initially that were later withdrawn. The observations made as a result of the QA audits were useful in terms of fine-tuning monitoring and GCP processes; however, they also reflected a more traditional approach to quality audits, and provided stark contrast with the findings of the independent QA expert. This scenario highlights one of the very real factors that contributes to the difficulty industry organizations experience when looking to adopt new approaches the temptation to stick to familiar routines and mindsets rather than embrace new processes. As previously mentioned, one novel technique employed here was the participation of an active, independent QA partner during the drug development process, who participated in the studies from cradle to grave as a proactive member of the clinical team. This expert both reviewed and signed off on Despite early and repeated efforts to refocus monitoring activities to exploit central or remote monitoring and maximize the value of dramatically reduced onsite monitoring visits, the team encountered numerous instances where monitors fell back into traditional behaviors. the protocol, participated in the QbD meetings, and audited four sites. By taking a proactive approach to quality oversight, the study team was able to address protocol compliance issues in near real-time, and to take multiple corrective actions. Actions included site and CRA retraining, rapid protocol amendments, EDC system changes to provide more effective system alerts and to fine-tune edit checks, and generation of new management and quality reports to help in data review. Safety issues were monitored as they occurred, so from each study s inception, the team was able to establish a comfort level that subject safety was properly being managed. The development program experienced numerous benefits from having put into place the quality approaches, tools, and processes described in this paper. For example: 2013 ACRP Peer Reviewed x 45

As a result of looking at data early and often, the team was able to apply lessons gained in earlier studies to those starting later, without having to wait for the earlier studies to finish. The sponsor realized significant direct cost savings as a result of minimizing the number of onsite monitoring visits. Sites experienced efficiency boosts and net time savings as a result of carrying out direct data entry, which minimized the quantity of study-related paper documentation that had to be managed. All parties realized improved working relationships as a result of focusing onsite monitor activities on topics that matter (e.g., subject eligibility, protocol adherence, drug supply management) and not, significantly, on the SDV activities that traditionally occupy so much time while producing little value. All of the benefits accrued from taking a fresh look at quality, and from a willingness on the part of all involved parties and especially the sponsoring pharmaceutical company to consider new approaches. All of the benefits accrued from taking a fresh look at quality, and from a willingness on the part of all involved parties and especially the sponsoring pharmaceutical company to consider new approaches. The benefits seen in the use of the quality measures used in these studies were predicated on: getting the data into the database early (i.e., in real-time through the use of direct data entry supported by esource), a willingness to respond to the data, and the willingness of all parties to take an evidence-based approach to quality-related processes. The 21st century has arrived, as have a number of recent regulatory gifts in the areas of esource and monitoring. It is now time to implement broadly and routinely the many process and technical innovations that are available. References 1. Guidance for Industry E6 Good Clinical Practice: Consolidated Guidance, April 1996. 2. Ball L, Meeker-O Connell A. 2011. Building quality into clinical trials. The Monitor 25(7): 11 6. 3. Bhatt A. 2011. Quality of clinical trials: a moving target. Perspectives in Clinical Research 2(4): 124 8. 4. Sprenger K, Nickerson D, Meeker-O Connell A, Morrison B. 2013. Quality by design in clinical trials: a collaborative pilot with FDA. DIA Journal 47(March): 161 6. 5. Food and Drug Administration. Guidance for Industry: Oversight of Clinical Investigations A Risk-Based Approach to Monitoring (Draft). Available at www.fda.gov/downloads /Drugs/GuidanceComplianceRegulatory Information/Guidances/UCM269919.pdf. 6. European Medicines Agency. 2011. Reflection Paper on Risk Based Quality Management in Clinical Trials (EMA/INS/GCP/394194/2011). Available at www.ema.europa.eu/docs/en_gb /document_library/scientific_guideline/2011 /08/WC500110059.pdf 7. U.S. Food and Drug Administration. December 2012. Guidance for Industry: Electronic Source Documentation in Clinical Investigations. Available at www.fda.gov/downloads /Drugs/GuidanceComplianceRegulatory Information/Guidances/UCM328691.pdf. 8. European Medicines Agency. 2010. Reflection Paper on Expectations for Electronic Source Data and Data Transcribed to Electronic Data Collection Tools in Clinical Trials (EMA/INS/ GCP/454280/2010). Available at www.ema.europa.eu/docs/en_gb/document_library /Regulatory_and_procedural_guideline/2010 /08/WC500095754.pdf. 9. Mitchel J, Schloss Markowitz JM, Yin H, et al. 2012. Lessons learned from a direct data entry Phase 2 clinical trial under a US IND. DIA Journal 46: 464 72. 10. Mitchel J, Schloss Markowitz JM. Risk based monitoring, time for change international clinical trials. International Journal of Clinical Trials February 2012. 11. Mitchel J, Kim YJ, Choi JH, et al. 2010. The final efrontier. Applied Clinical Trials Online, May 1, 2010. Jules Mitchel, MBA, PhD, is president of Target Health Inc., a full-service ecro dedicated to strategic planning; regulatory affairs; chemistry, manufacturing, and controls; clinical research; data management; biostatistics; and medical writing. He also leads the team developing software tools to enable paperless clinical trials. He has previously held positions with Wyeth and Pfizer, and is currently an adjunct professor at the School of Pharmacy at Rutgers, an adjunct clinical professor in the Department of Dermatology at SUNY Stony Brook School of Medicine, and a lecturer at the New York Medical College. He served as the study director for the project described in this article. He can be reached at jmitchel@targethealth.com. Dean Gittleman, MS, is senior director of operations at Target Health Inc. During his more than 20-year career, he has built a data management function from scratch, defended two medical device premarket approvals during advisory panel review, and built a successful functional service provider relationship with Accenture in Chennai, India, on behalf of Eisai. He also led Vertex s biometrics functions in that company s first New Drug Application filing (for hepatitis C). At Pfizer, he played a leadership role in the global deployment of Oracle Clinical, managing the QbD meetings and design and implementation of the EDC system. He can be reached at dgittleman@targethealth.com. Judith M. Schloss Markowitz, MS, received her bachelor s degree in 1974 and her master s degree in 1977, both from Queens College of CUNY. She has 35 years of experience in the pharmaceutical industry, and currently serves as a senior project manager at Target Health Inc., overseeing five clinical projects that are now running concurrently. Previous to this position, she worked for various divisions of Wyeth Research in its Clinical Department for 32 years. She was the lead project manager for the study described in this article, and can be reached at jmarkowitz@targethealth.com. Laura Suciu, MS, is director of quality assurance at Target Health Inc. Her areas of expertise include defining and implementing quality systems, policy and procedure development, training, and computer and software validation. She performed internal overall QA oversight of the project described in this article, and can be reached at lsuciu@targethealth.com. Michael R. Hamrell, PhD, RAC, FRAPS, RQAP-GCP, CCRA, is president of MORIAH Consultants, a regulatory affairs/clinical research consulting firm located near Los Angeles, Calif. He has worked in drug development, clinical research, and regulatory affairs for more than 30 years, has published numerous papers, and serves on the editorial boards of several leading journals in clinical research and regulatory affairs. For the project described in this article, he served as an independent quality expert to assure proactive quality planning at all stages of development; reviewed and signed off on each protocol and clinical data monitoring plan; participated in weekly QbD meetings; and conducted site audits. He can be reached at michael@moriahconsultants.com. 46 x Monitor December 2013 2013 ACRP