Your goal is to give people a better quality of life. We re proud to be a part of your mission.

Transcription

1 GLOBAL ONCOLOGY CRO Your goal is to give people a better quality of life. We re proud to be a part of your mission. This paper considers how sponsors can adapt conventional approaches to data management, cleaning and verification to make the best use of resources and technology that will ensure the trustworthiness of trial data. A Novella Clinical White Paper Moving Potential. Forward.

2 Oncology Clinical Global Trial Oncology Operations CRO Compelled by both regulatory and financial pressures, the biopharmaceutical industry is focused on hastening drug development times without comprising quality. The assurance of treatment safety and efficacy lie in the veracity of data generated during pre-clinical, clinical and post-marketing studies. With the advent of electronic data capture, remote technology and data visualization, trials have never before had ready access to such large volumes of high quality data. A complex aspect of data review is the challenge of data bloat, the increased use of less essential endpoints that can consume about 18 percent of a typical phase II or III clinical trial budget. 1 The challenge of managing the collection, maintenance and analyses of data, in light of costeffective metrics, limited resources and regulatory standards now embracing risked-based approaches, presents questions of priorities and processes for oncology trial sponsors. In this paper, the oncology specialty clinical research organization (CRO) Novella Clinical considers how sponsors can adapt their traditional thinking of data management, cleaning and verification to make the best use of resources and technology that will ensure the trustworthiness of their trial data and outcomes. 2

3 Source Data Verification: Is Less Than 100% Good Enough? Traditionally, clinical trial sponsors have insisted on examining every data point collected in a trial. To trust trial outcomes, verify all data. This 100 percent approach requires a significant resource commitment in staff expertise and time for data field design, review and validation. How big a commitment? Source data verification (SDV) can consume $90 million for just one industry phase III drug trial, about 25 percent of a trial budget. 2 Similarly, a model-based analysis has calculated data management and statistics activities in large trials comprise 11 to 14 percent of costs, while site management, monitoring and payments account for 25 to 30 percent. 3 Given that technology has advanced, evaluation methods have evolved and resources have remained limited, is 100 percent SDV prudent? While quality control is essential, 100 percent verification may no longer be essential. Regulatory agencies from the US Food and Drug Administration 4 to the European Medicines Agency 5 to the UK Medicines and Healthcare Products Regulatory Agency 6 have said so in their guidances on risk-based monitoring (RBM). A major industry analysis agreed. 7 So do we. A growing body of peer-reviewed research has explored different aspects of SDV as part of RBM strategies, lending support to move away from using 100 percent on-site SDV all the time. RBM requires changes in trial conduct, starting with initial study assumptions as well as data definition, prioritization and monitoring methods. The goal is to build risk mitigation into the study design and management, which consequently reduces the use of on-site verification, but includes the implementation of such centralized activities as remote data review via page-by-page EDC examination, listings review or patient profiles and analytics to identify the best focus of monitoring efforts. But, can a sponsor trust the correct application of SDV, when all clinical data is not assessed? A growing body of peer-reviewed research has explored different aspects of SDV as part of RBM strategies, lending support to move away from using 100 percent on-site SDV all the time. According to the first comparison of the impact of data monitoring methods on study outcomes and conclusions, a 2012 analysis of a controlled phase III clinical trial comparing investigational treatment in patients with advanced cancer, SVD identified random errors that made little impact on results and clinical conclusions of the trial. 8 Via their comparison of SDV with original trial data and centralized data monitoring, the investigators determined the most efficient approaches to examining data. For the primary endpoint of overall survival, central monitoring using an external data source was best, and for the more subjective secondary endpoint of objective response, they recommended an independent blinded review committee and tracking system to monitor missing scan data. 8 The authors included several different procedures that did not involve site visits within the central monitoring definition, such as reviewing accumulating data for consistency over time and across different data items, statistical analysis to identify unusual data patterns within and across trial sites, and external validation of data items, such as through birth and death registries. 8 3

4 Reducing monitoring by half will not compromise overall data quality, nor increase data queries, according the first efficacy evaluation of remote SDV of patient medical records, a small six-month assessment during a global phase III oncology trial. 9 By conducting two thirds of the site monitoring visits remotely, the quality of the onsite visits actually improved in the trial. The authors concluded the potential impact on [trial] productivity and cost Reducing monitoring by half savings associated with remote SDV to be considerable will not compromise overall when scaled up across a larger number of studies and data quality, nor increase trial sites, particularly in relation to reducing monitor data queries... travel time, reducing travel costs, and reducing demands on study site trial monitoring facilities. 9 Novella Clinical has confirmed 100 percent SDV and frequent on-site monitoring visits yield only a small percentage of data changes. When we examined the impact of data entry on overall monitoring and data cleaning activities using seven datasets that generated more than 77,000 queries, only 4.6 percent of the data fields changed because of manual queries generated by on-site monitors. Specifically, only 1 percent of the primary efficacy endpoint data changed, compared to about 9 percent of the safety analyses data, which included adverse events (AEs) and concomitant medications. An industry self-evaluation concurs. The international pharmaceutical industry consortium TransCelerate BioPharma Inc. (TransCelerate) analyzed nine studies from six member companies, and found that of all queries, those generated by SDV tallied an average of only 7.8 percent and the percentage decreased to just 2.4 percent when restricted to critical data queries, suggesting SDV has a negligible effect on data quality. 7 Given these low rates of SDV-generated data queries and changes, is there a more efficient and cost effective way to deliver a quality data set that is fit for purpose for submission to regulatory authorities? TransCelerate has proposed a methodology worth considering as a future RBM standard because it has the flexibility for application to different trial designs, phases and stages. 7 TransCelerate proposes shifting from 100 percent SDV to RBM, by using technology for off-site combined with central monitoring of pre-defined study parameters, while using an adaptive, targeted approach to onsite monitoring activities that supports sites processes, participant safety and data quality. 7 TransCelerate distinguishes between recommended use of SDV and Source Data Review (SDR). SDV compares data in patients case report forms (CRF) or other data collection conduits, such as laboratory reports, to the original source to confirm accuracy, while SDR is broader and remotely evaluates trial aspects not associated with a data field. SDR entails source review to check quality, protocol compliance and investigator involvement as well as adequate processes and compliance, such as to Good Clinical Practice (GCP). This distinction enables prioritization of tasks and response to errors. For example, high priority tasks include compliance checks, which might cause the exclusion of data from analysis, while low priority tasks include checking transcription errors, which TransCelerate observes are typically infrequent, insignificant, and do not lead to study data being unusable. 7 4

5 Of note, TransCelerate suggests, and we agree, that the rates of either SDV or SDR can change during a trial, if issues or risks occur. By working with a CRO, sponsors can identify trigger points that would alter the amount of data scrutiny to ensure accuracy. TransCelerate suggests sponsors should consider increasing SDV if a site is identified as an outlier based on lower-thanaverage reported AEs, with further focus on participants with no reported AEs. 7 For example, in a trial of a drug with no known cardiac issues, the occurrence of abnormal or changes from study start in the electrocardiograms may increase scrutiny either offsite or onsite to confirm if an emerging safety signal should be addressed and may trigger increased SDV. The use of data edit checks also helps determine monitoring diligence. When data cleaning does not examine all vital signs, an edit check set for a systolic blood pressure above 120, for example, will automatically generate data queries for larger values. This approach then helps preclude significant blood pressure data errors because outlier values are queried prior to interim or final analyses. The process also could reduce of SDV from 100 to only 25 percent for such fields. Data: What Matters Most? If 100 percent SDV is not necessary, how does a sponsor set the related quality objectives? Novella Clinical has found that small and mid-sized sponsors or those with limited pipelines, for which the risks in product development can be greater, are more conservative in their SDV approaches. Using a CRO to help ask and objectively answer hard questions about data during protocol development can help sponsors become more confident with RBM. Together, the sponsor and CRO decide what data are critical to the study. What other data must be collected? Does the CRF design capture the data appropriately and accurately? How will various datasets be cleaned, so as to lend accuracy and authenticity to the study outcomes and conclusions? The answers all contribute to the conduct of data review.... small and mid-sized sponsors or those with limited pipelines, for which the risks in product development can be greater, are more conservative in their SDV approaches. Using a CRO to help ask and objectively answer hard questions about data during protocol development can help sponsors become more confident with RBM. We have found that in creating an integrated data review plan, a best practice lets the end define the means. That is, take a step back, consider the trial s bigger picture and purpose and then use the endpoints to identify quality objectives. This process, rooted in protocol development, involves mapping data points to one or more study endpoints. Some endpoints are defined by one data point, but others involve not only a collection of key data points, but also secondary and tertiary data that can influence key data. Moreover, some factors influence information that might not be visible in the data outright, but rather in analyses, yet must be accounted for in data cleaning. A successful data map tells the story of each patient s experience during the study. 5

6 For example, in a trial with red blood cell (RBC)-transfusion independence as a primary endpoint, hematology laboratory data were critical and 100 percent verified, as were data related to study visits and drug compliance. When the data technically was clean, a review of hemoglobin and RBC values individually over time revealed large spikes or drops in values but nothing in the CRFs supported why these fluctuations occurred. The analyses raised questions. Why were these fluctuations occurring and were they significant influences on the endpoint? To determine this, a review of the patient charts confirmed data completeness and well documented transfusions. We strategically used a set of parameters to assess the endpoint data and, ultimately we determined why the RBC values changed during the observation period for each individual patient. Consequently, going forward in the trial, we revised the protocol training materials, improved clinical research associate training and facilitated more timely and efficient primary endpoint analysis through planned critical data reviews and a streamlined SDV and query processes. A sponsor s current study objectives are not necessarily the sole source of defining critical vs. non-critical data and the subsequent cleaning priorities, as future uses must be considered and ranked against available resources. Safety, efficacy and compliance endpoint data provide scientific and clinical value to the current study objectives, while future needs may include quality of life data to support marketing efforts and economic impact data to inform reimbursement plans. Mapping of the data fields and points to the endpoints will reveal overlaps, which are among the most significant data points. These points, with consideration of the protocol, the treatment, the patients risks and the anticipated AE profile, should garner more attention in cleaning. While secondary data contribute to the context of study outcomes, they also contribute to data bloat, a trend on the increase, and increased trial costs. In Phase II and III trials, 54.3 percent and 47.9 percent, respectively, of procedures support primary and key secondary endpoints. The typical clinical trial protocol averages 13 endpoints, of which half are tertiary or exploratory a doubling per protocol of the less essential endpoints in the last decade, reports the Tufts Center for the Study of Drug Development. 1 Tufts estimates the direct costs per protocol for these extra, non-core procedures tally an estimated $1.1 million, about 18 percent of a typical clinical trial budget. 1 Based on active FDA-regulated Phase II and III clinical trials, Tufts calculated that the pharmaceutical industry spends between $4 billion and $6 billion annually on procedures that generate extraneous clinical trial data. 1 A CRO can help a sponsor examine a trial design for endpoint designation, determine the related critical and noncritical data and then develop the hierarchy of data and their related cleaning processes and subsequent analyses. Mapping of the data fields and points to the endpoints will reveal overlaps, which are among the most significant data points. These points, with consideration of the protocol, the treatment, the patients risks and the anticipated AE profile, should garner more attention in cleaning. 6

7 Other factors in data field prioritization are the trial duration and stage. In a long-term oncology study in which overall survival is an endpoint, collection of patient data will increase and there will be more data queries generated during this follow-up period. An experienced CRO can anticipate the evolving data review needs and plan for the related allocation of resources. Data Review Now: Effective but Inefficient Data review must address two aspects: first, do the data match the source material, a one-to-one quality-based relationship authenticated through SDV, and then, do the data make sense, a comprehensive view of how the data come together from inception to output. The protocol design, staff training, CRFs, monitoring procedures and the resulting clinical and safety databases all play a role in setting up the data review for success. In considering these two objectives, sponsors should evaluate data review plans by asking about edit and logic checks on data consistency and completeness, safety and efficacy assessments, error and trend identification as well as process efficiencies and cost effectiveness. Sponsors should demand confidence that the data review will produce accurate and appropriate results needed to move a treatment into the next stage of the development and approval process. Currently review processes can involve many people, disciplines and functions. Often they work in parallel with limited interactions between the staff and they focus on the data from only their viewpoint, which is defined by their responsibilities and expertise. This process can cause an over-handling of data, some points being touched multiple times, and a silo mentality that can hinder ensuring the data make sense. Better data review structures will not only streamline the process for multiple parties but, more importantly, will influence data quality, trial efficiency and, ultimately, the delivery of improved treatments to patients. Revising Data Review: What the Future Holds At Novella Clinical, we believe the future of data review lies both in improved field training and in using tools to enhance execution, which will decrease data handling, reduce errors and bring efficiencies. We have instituted the use of an Integrated Data Review Program that uses a coordinated team to conduct and monitor the progress of data review throughout the study as well as to identify data issues. The five tenets of Novella Clinical s Quality Clinical Data Review Charter govern this cross-functional integrated program: Identify key data listings and reports define roles; establish communication and meeting frequency evaluate data relatedness (data mapping) create timelines.... we believe the future of data review lies both in improved field training and in using tools to enhance execution, which will decrease data handling, reduce errors and bring efficiencies. 7

8 The hallmark of Novella Clinical s integrated program is our in-house clinical scientist, who manages the team, both to establish and lead data review efforts. Our clinical scientist s role goes well beyond the trial team s 101 training on the medical aspects of the trial. Our clinical scientist is integral in developing and executing the Integrated Data Review Plan and engaging in regular communications with the sponsor, project manager and team members. The clinical scientist is the hub of our Integrated Program s team, which is comprised of members from each functional area involved in data review, and we have found that the best team structure may include members from our sponsor, depending on the role matrix. Our team also includes staff with roles as medical monitor, data manager, safety, biostatistics, site monitor and remote reviewers, which are part of our Clinical Data Review Center (CDRC). The trial s project manager is on the team as well, but has a small role because of greater responsibilities for the overall trial oversight. We believe sponsors greatly benefit when CROs have an in-house clinical scientist to manage data review. Historically, sponsors contracted with CROs to handle just routine aspects of trial conduct. The CRO delivered a data set that a sponsor would then assess to determine study outcomes and conclusions. But now the norm, we have found, is a more partnered-collaboration between sponsor and CRO, particularly those smaller and mid-sized biopharma companies that have less robust in-house experience or bandwidth to clinically analyze data. Such sponsors appreciate in-depth scientific and strategic conversations with a CRO s clinical scientist, which go beyond their more tactical exchanges with project managers regarding trial processes. These communications help balance the quest for quality with the related costs, 10 resolve issues in a timely matter and foster consensus so the clinical study report delivers what the sponsor needs. Within our Integrated Program team, the CRA and CDRC review the data with a clinical focus while other members focus on data management. The CRA s strong focus on SDV harmonizes well with CDRC reviews that combine both logistical and medical functions and the data management members focus on process. For example, our CRA will review a CRF entry regarding a patient s blood cell counts, matching it to the original laboratory reports, while our CDRC checks data for consistency and trends throughout the trial at both the site and patient levels. We have found that the dovetailed CRA and CDRC roles in clinical monitoring create great efficiencies. Although the CDRC does not have access to source documents, they are able for example to identify misunderstandings of the protocol or CRF completion guidelines and provide re-training; a screened patient that is seemingly ineligible and alert the site before randomization; and study trends at a given site or across the study. After site staff complete data entry, the CDRC s remote review will commence and allows for up-front data cleaning via checks for completeness as well as high-level logic. The triage effect of the CDRC review permits cutting away excess work, so subsequent reviews by a data manager or medical monitor can then focus on their primary responsibilities unencumbered with inaccuracies or data gaps. By using the CDRC as part of the cleaning process, the onsite monitoring is more productive because of ongoing data review between on-site visits, which can occur every four, six or eight weeks or longer. 8

9 In addition to the defining roles and coordinated responsibilities, our Integrated Program addresses the timing of data review how often and when and the methods used. A fundamental understanding of the protocol and the therapeutic nuances influences timing of reviews, which need to be planned long before data assessment begins and ideally shortly after the identification of necessary data fields and the related CRF design. We believe that early and ongoing reviews are a best practice because they can identify issues with protocol design or interpretation, CRF design or data collection, avert future mistakes and missteps with the potential to derail a trial as well as diminish the need and resources involved in data queries, data corrections and protocol amendments. For example, in a recent phase II oncology study, the primary study endpoint used tumor size changes as a measure of response to treatment. The correct comparison of a patient s CT or MRI scans was crucial. However, the data review team found via early review that a site team had compared the most recent scan results for some patients to the immediately previous scan, not the baseline scan, as per protocol. If these response data had not been reviewed early in the study, comparison inconsistencies might easily have been overlooked and patients erroneously allowed to stay on the trial too long or discontinued too soon. Because this particular data point was so critical to the final study analysis, incorrect response comparison methodology was a very serious problem. If determined early, corrective action, such as site personnel and monitor re-training, can make certain the appropriate collection of primary endpoint data. We believe that early and ongoing reviews are a best practice because they can identify issues with protocol design or interpretation, CRF design or data collection, avert future mistakes and missteps with the potential to derail a trial as well as diminish the need and resources involved in data queries, data corrections and protocol amendments. Working Smarter: Review Toolkits The process of data cleaning may involve creating patient profiles from the CRF dataset or external data sources or both. These profiles tell the story, in facts and figures, of what happened to the patient during the study. Profile development offers a means of data making sense reviews as well as of identifying any irregularities, which need to be investigated prior to the development of the final clinical study report. In addition to individual patients, data cleaning and review addresses aggregate data of a site s patients, of groups of participants or of the entire study. Standard tools for these analyses include tables, listings and figures of data points gleaned from CRFs. While the data management plan will dictate the required analyses, it is not unusual to also perform ad-hoc analyses if certain data trends or questions emerge during data review. A very important step in the data cleaning process is the reconciliation of aggregate data with the safety database. 9

10 Key fields in the clinical and safety databases, which are maintained separately, should match the serious adverse events (SAEs) reported during the study. Typically, these fields include the verbatim SAE term, start and stop dates, relatedness to study drug, action take with study drug and patient identifiers. The reconciliation at an individual patient level ensures the reported information is consistent across and in each database. Not only does this action ensure the recognition of any AEs of interest such as common toxicities or emergent safety signals, it can potentially impact future protocol development, informed consent forms, safety reporting to regulatory authorities and clinical study reports. The newest generation of data tools takes advantage of technology s ability to mine data and present graphic interpretations. Changes in data, which can easily slip by in listings or tables unless very pronounced, become readily evident when presented as an image or graph, especially data point trends. For example, if a protocol anticipates a therapy-related reduction in lymphoblast or white blood cell counts, minor changes are more easily seen in a single graph than in a 100 pages of counts. Graphics also have the ability to shed light on common oncology trial endpoints related to longevity, such as duration of effect, time to remission or overall survival. Novella Clinical currently uses software from Patient Profile, LLC in multiple trials to create individual patient reports. This tool quickly provides standard or customized graphic profiles of patients, individually or as a group, by integrating data from different sources on everything the patients experienced during the trial. The ability to graphical represent and then assess, for example, adverse events and concomitant medications over time, enables our reviewers to rapidly identify data values outside of normal ranges, whether for SAEs or other important data points. The software also facilitates retrospective data analysis, which we have found helpful for quarterly data reviews, for example. Patient Profile Sample Source: Patient Profiles, LLC. 10

11 For Novella Clinical, graphic representation of study data has made possible the quick identification of site concerns, such as the need for re-training related to data entry, data collection issues and casebook completion. Data visualization has helped reveal unusual data patterns within a single trial site s patient group as well as between sites. One caution, however, about graphic data representations is to be strategic about their use based on the trial timeline. Obviously, there is much less data early in the life of a study than later. A single graphic representation of just eight to 10 listings is not as powerful as four to six months of laboratory values to reveal data trends or questions. While graphics and trend software have their place in helping steer the focus of data reviewers, it is the expertise of those reviewers that helps identify key data points and sets. The why of data review goes back to a sponsor s basic reason for the trial what clinical question is this study trying to answer and then knowing via data mapping where and when to look. In Conclusion In considering the collection, management and verification of data for oncology trials, it is useful to reflect on current regulatory standards. While the FDA s RBM guidance focuses on protection of clinical trial participants, data quality 4 and compliance, it also was explicit in demonstrating that the Agency is open to both greater variety in monitoring methods and broader parameters for monitoring clinical trial conduct. The improvement in data review tools will be incremental going forward. Certainly the advent of EDC was a leap forward that continues to yield benefits in trial management. Standardization of data formatting across the trial industry will facilitate the ready movement of data from patient records into CRFs and databases. Cloud computing will bring other efficiencies as will upgrades in customizable software for analyses. Continued investigations of RBM procedures also will improve data review. TransCelerate, in partnership with operational data available through the Medidata Clinical Cloud, launched in May 2014 a new analysis of 7,000 clinical trials involving more than 120 sponsors to assess the relative contribution of SDV. 11 Their findings most certainly will influence future use of SDV and trial management. 11

12 A Novella Clinical White Paper 1 Tufts Center for the Study of Drug Development, Extraneous Data Collected in Clinical Trials Cost Drug Developers $4 Billion to $6 Billion Annually Press Release, Nov. 6, Accessed at May 24, Funning S, Grahnén A, Eriksson K, Kettis-Linblad A (2009) Quality assurance within the scope of Good Clinical Practice (GCP)-what is the cost of GCP related activities? A survey within the Swedish Association of the Pharmaceutical Industry (LIF) s members. The Quality Assurance Journal. 12(1): 3 7. Accessed at b ed146f3a351e8f9b5c May 29, Eisenstein EL, Lemons PW, Tardiff BE, Schulman KA, Jolly MK, et al. (2005) Reducing the costs of phase III cardiovascular clinical trials. American Heart Journal. 149(3): Accessed at May 29, as cited by Rostami. 4 FDA Guidance for Industry Oversight of Clinical Investigations A Risk-Based Approach to Monitoring August 2011, page 4 Accessed at 5 EMA. A Reflection Paper on Risk Based Quality Management in Clinical Trials, August Revised February Accessed at WC pdf. 6 MHRA. Risk-adapted Approaches to the Management of Clinical Trials of Investigational Medicinal Products. Available at: October Accessed May TransCelerate, Position Paper: Risk-Based Monitoring Methodology, May Accessed at Position-Paper-FINAL-30MAY2013.pdf May 16, Smith, CT, et.al. The value of source data verification in a cancer clinical trial. PLoS One. 2012;7(12):e doi: /journal.pone Epub 2012 Dec 12. Accessed at May 29, Uren SC, et. al. Reducing clinical trial monitoring resource allocation and costs through remote access to electronic medical records. J Oncol Pract Jan;9(1):e13-6. doi: /JOP Accessed at May 29, Keller, Mary Rose. Former VP of clinical operations, Sangart, as quoted in The Future Of Clinical Trials In An Outsourced Model. by Rob Wright. Clinical Leader. March 5, Accessed at June 2, Medidata Press Release, Medidata and TransCelerate BioPharma Inc. Announce Joint Initiative. May 20, Accessed at June 3, 2014.

13 About Novella Clinical Novella Clinical, a Quintiles company, is a full-service clinical research organization specializing in oncology clinical trials. Novella s dedicated oncology division is comprised of more than 300 oncology clinical trial specialists across all functional areas. Headquartered in Research Triangle Park, NC, and Stevenage, UK, and with active oncology work in 40 countries, Novella Clinical provides the experience and insight to bring oncology drugs to market on time and on budget. For more information, visit novellaclinical.com/oncology or contact us at You see beyond the clinical trial. Your goal is to ultimately give people a better quality of life. Novella Clinical is proud to be a part of your mission. Moving Potential. Forward. Novella Clinical s Global Headquarters: 1700 Perimeter Park Drive, Morrisville, NC Phone: European Operations: Richmond House, Walkern Road, Stevenage, Hertfordshire SG1 3QP Phone: +44 (0)