Date/Time Stamped Files and Audit Trails: What Part 11 Compliant SAS Systems are Made of. Carolyn Dougherty, ViroPharma Incorporated, Exton, PA

Transcription

1 PH003 /Time Stamped Files and Audit Trails: What Part 11 Compliant SAS Systems are Made of. Carolyn Dougherty, ViroPharma Incorporated, Exton, PA ABSTRACT Clinical data reporting systems are considered high risk with respect to 21 CFR Part 11 compliance. The FDA s recent change in scope for Part 11 is simply a narrower interpretation; Part 11 is still a requirement and therefore, clinical data reporting systems are still subject to Part 11. Analysis of clinical trial data is usually conducted using SAS software. s extracted from a clinical data management database (raw datasets) and submission/analysis datasets are created with SAS. In addition, data are summarized using SAS. How can processes followed to produce these datasets and analyses be made compliant? /time stamped files and audit trails are two key components of Part 11 that, if maintained in the process of reporting clinical trial data, will greatly improve compliance. Maintaining an audit trail for dataset and table generation, and tracking date/time stamps of these files and the SAS programs used to create them are not small tasks. This tracking, including who created what and when, is often a manual process or a non-existent process. This paper describes an automated system that tracks generation of all SAS programs, datasets and analyses; a capability not inherently available from SAS. This system improves Part 11 compliance in a clinical reporting environment. INTRODUCTION Reporting of clinical trial data for a clinical study report requires a large amount of programming to produce analyses of the data. Beginning with raw datasets, programs are created to generate submission or analysis datasets followed by programs created to produce analyses as required in the statistical analysis plan. This means, for a single clinical study, there could be one hundred or more programs used to produce multiple datasets and sometimes hundreds of summary tables. 21 CFR Part 11 requires data controls be implemented within systems that create, modify, maintain, or transmit electronic records. Therefore, reporting of clinical trial data clearly falls within Part 11 requirements. The requirements further state: Such procedures and controls should include use of secure, computer generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records. Record changes shall not obscure previously recorded information. Such audit trail documentation shall be retained for a period at least as long as that required for the subject electronic records and shall be available for agency review and copying. Therefore, it is important per Part 11 to maintain an audit trail of programs, datasets and output included in the process of reporting clinical trial data. Per the FDA Guidance on Computerized Systems Used in Clinical Trials, an audit trail is a secure, time stamped record that allows reconstruction of the course of events relating to the creation, modification and deletion of an electronic study record. This paper will describe three macros created to produce an audit trail containing information about summary tables in a clinical study report, including information about the datasets used to generate the summary tables. We took the approach described below to generate an audit trail because SAS and PDF (Portable Document Format; the file format of summary tables) did not have an easy mechanism for storing specific metadata about submission datasets and tables. CLINICAL REPORTING DELIVERABLES INCLUDED IN THE AUDIT TRAIL First, we began with identifying what information and physical files could be tracked as part of the audit trail from our existing process for reporting clinical trial data. As mentioned, analysis programming for a clinical trial begins with raw datasets and continues through production of summary tables. The process is depicted in the following basic figure and identifies the main deliverables for tracking in an audit trail: 1

2 Clinical Data Management Database Raw s DataSetram.sas.sas7bdat ram.sas Summary.pdf ram Audit Trail File (DataSetram.sas7bdat) ram Audit Trail File (ram.sas7bdat) The above process is easier to understand by focusing on only one specific CRF panel. For example, the figure below describes the reporting process for medical history data. Medical History Database Mhx_r.sas7bdat u_medhis.sas Medhis.sas7bdat smhx.sas smhx.pdf Summary of Medical History Audit trail file: u_medhis.sas7bdat Audit trail file: Smhx.sas7bdat , Summary of Medical History, is only one summary table from a complete table of contents for a clinical study report. The remaining examples in this paper, will use medical history to identify audit trail information. However, the same information collected for the audit trail of medical history is collected for all submission datasets and all summary tables for a study. SUBMISSION OR ANALYSIS DATASETS When producing the submission dataset medhis.sas7bdat, the macro %trackraw is executed in the program u_medhis.sas. %trackraw produces the audit trail file, u_medhis.sas7bdat, with the following information: 2

3 Audit trail file: u_medhis.sas7bdat: Name of raw dataset(s) used to generate the submission dataset Raw dataset(s) date/time stamp Name of program (DataSetram.sas) used to generate the submission dataset Userid of programmer who generated the submission dataset (i.e. executed the program) Location/server on which the submission dataset was generated The audit trail file for each submission dataset includes one record per raw dataset used to generate the submission dataset. Below is an example of an audit trail file for a submission dataset. In this example, only one raw dataset was required to generate the submission dataset medhis.sas7bdat. Contents of the dataset program audit trail file, u_medhis.sas7bdat: Raw Raw /Time ram User ID Location ID mhx_r 20DEC2002:10:39:00 u_medhis cdougherty statsrv01 In addition, when a submission dataset is generated, the name of the program used to generate the submission dataset is stored as the label of the submission dataset. This allows us to easily identify what program generated the dataset; an important piece of the audit trail information noted later in the paper when producing the audit trail reports. SUMMARY TABLES The macro %mtltrack is executed within the table program, smhx.sas, to generate the summary table, smhx.pdf. %mtltrack produces the audit trail file smhx.sas7bdat with the following information: Audit trail file: smhx.sas7bdat Number, Title and Subset (extracted from an electronic table of contents) Name of submission dataset(s) used to generate the table Name of program (DataSetram.sas) used to generate the submission dataset (taken from the label on the submission dataset) dataset date/time stamp Name of program (ram.sas) used to generate the table (Summary.pdf) ram.sas date/time stamp /time the table (Summary.pdf) was generated Total number of pages in the table Userid of programmer who generated the table (Summary.pdf) Location/server on which the table (Summary.pdf) was generated The audit trail file for the each summary table includes one record per submission dataset used to generate the summary table. Below is an example of an audit trail file for a summary table. In this example, two submission datasets were required to generate the summary table smhx.pdf. 3

4 Contents of the table program audit trail file, smhx.sas7bdat: # Title Summary of Medical History Subset All Randomized ram. sas /Time Stamp medhis u_medhis 11MAR2003: 12:44:11 ram. sas smhx /Time Stamp ram.sas 22JAN2003: 11:32:00 /Time Stamp Generated 11MAR2003: 12:47:00 Total Pages User ID Location ID 1 cdougherty statsrv Summary of Medical History All Randomized demo u_demo : smhx 22JAN2003: 11:32:00 11MAR2003: 12:47:00 1 cdougherty statsrv01 4

5 AUDIT REPORT Although some of the above information which is stored in audit trail files may seem redundant, the files are human readable and therefore can be referenced easily at any time in the process. Storing these files makes the following information readily available when the generation of summary tables for a clinical study report is completed. Execution of the macro %audittrl combines all audit trail data and real-time date/time stamps into one dataset and audit_trail.pdf is created. Audit Trail File: u_medhis.sas7bdat Audit Trail File: smhx.sas7bdat Real-Time Data: Current date/time of submission dataset Current date/time of Summary.sas Current date/time of raw dataset Audit_Trail.pdf The three macros executed in this audit trail process can be validated to confirm that expected results are provided. However, it is important to note that a piece of the system relies on programming practices. If the submission dataset program macro (%trackraw) or the summary table program macro (%mtltrack) is not called in the appropriate programs, the Audit_Trail.pdf would be a failing audit and appropriate submission dataset programs and summary table programs would have to be re-executed to produce a passing audit. The following reports are included in Audit_Trail.pdf. Examples of each report are included on subsequent pages. AUDIT TRAIL REPORT #1: This report identifies whether the audit trail is a passing or failing audit trail. It confirms that the datasets and tables are acceptable (date/time stamps are in the appropriate order and the sequence of events can be traced) or will identify where problems exist. Note that the parenthetical letters for each bullet item below correspond to the letters in the example on the next page. (A) The report confirms the submission dataset(s) has not changed since the Summary.pdf was generated (medhis.sas7bdat is dated before the smhx.pdf and date/time stamp of medhis.sas7bdat used to create smhx.sas is equal to current date/time stamp of medhis.sas7bdat). (B) The report confirms the ram.sas has not changed since the Summary.pdf was generated (smhx.sas is dated before smhx.pdf and date/time stamp of smhx.sas used to create smhx.pdf is equal to current date/time stamp of smhx.sas). Notice the example audit fails this test because the table program (SDISP) is currently dated (:14:48) after the table was generated (:11:59). (C) The report confirms the ram.sas has not been changed since the submission dataset was generated (u_medhis.sas is dated before medhis.sas7bdat). This is why attaching the name of the submission dataset program as the label for the submission dataset was necessary. (D) The report confirms the raw data has not changed since submission datasets and tables were generated (current mhx_r.sas7bdat must be dated before current medhis.sas7bdat and smhx.pdf). AUDIT TRAIL REPORT #2: The report identifies the order in which submission datasets were generated as well as the user ID of the programmer who generated each dataset and the server name on which each dataset was generated. AUDIT TRAIL REPORT #3: The report provides a complete list of tables and the date/time of each table for medical writers as well as identification of who generated each table and the server name on which the table was generated. AUDIT TRAIL REPORT #4: The report identifies the submission dataset(s) used when generating each summary table. This report easily identifies all ram.sas programs that must be re-run if a specific submission dataset is changed. 5

6 CONCLUSION Three macros incorporated into a process from CRF data to final summary tables for a clinical study report can help maintain a thorough audit trail for SAS deliverables and improve Part 11 compliance. First, the macro %trackraw is executed during the generation of each submission dataset to store metadata of the raw datasets and submission dataset. Second, the macro %mtltrack is executed in each table program to store metadata for the summary table when the summary table is generated. Last, %auditrtl is executed when tables are complete, or at an interim timepoint, to combine all of the audit trail data into audit trail reports. Integration of the audit trail data for the audit trail reports enables any programmer or manager to identify who generated what and when. In addition, it allows individuals to verify that the sequence of editing programs, creating datasets and generating tables is appropriate and that the sequence of these events can be easily traced. All of this information gathered from the audit trail can be summarized in only a few minutes; much less time than if checked manually. Most importantly, if audited by the FDA to confirm compliance with 21 CFR Part 11, the audit trail is readily available to the FDA auditors. 6

7 SAS Conference Protocol DUMMY123 Jun 6, :36 Audit Trail AUDIT_TRAIL Audit Trail #1 - All Tracking Information from Raw s to s # Title Name Run Name Name Sub-mission (A) = (B) = (C) < (D) Raw = Raw for ADS Summary of Subject Disposition All Enrolled SDISP 11:59:00 14:48:00 12:00:00 A_AE U_A_AE 10MAR :26:40 10MAR :26:40 24FEB :06:00 ERROR DISP U_DISP 11MAR :22:44 11MAR :22:44 11MAR :21:00 ERROR DEMO U_DEMO 10:24:00 ERROR Summary of Subject Enrollment All Enrolled SENRL 12:08:00 12:08:00 12:08:00 DEMO U_DEMO 10:24: Summary of Subject Demographics All Enrolled SDEM 11:58:00 11:58:00 11:59:00 DEMO U_DEMO 10:24: Summary of Medical History All Enrolled SMHX 12:13:00 12:13:00 12:13:00 MEDHIS U_MEDHIS 11MAR :44:11 11MAR :44:11 11MAR :43:00 DEMO U_DEMO 10:24:00 [Note: cells for the four columns on the right are green if the criteria for acceptance are met, or red if the criteria for acceptance are not met] 7

8 SAS Conference Protocol DUMMY123 Jun 6, :36 Audit Trail AUDIT_TRAIL Audit Trail #2 - Order s Generated Name Name User ID Location ID DEMO U_DEMO : cdougherty statsrv01 EXCL U_EXCLUS :10:58:37 cdougherty statsrv01 COMMENTS U_COMM :17:01:31 cdougherty statsrv01 VAS U_VAS 14FEB2003:11:29:41 cdougherty statsrv01 A_VITAL U_AVITAL 14FEB2003:14:22:09 cdougherty statsrv01 VITALS U_VITALS 20FEB2003:12:09:59 cdougherty statsrv01 PE U_PHYSEX 24FEB2003:16:54:02 cdougherty statsrv01 A_AE U_A_AE 10MAR2003:16:26:40 cdougherty statsrv01 ECG U_ECG 11MAR2003:10:13:20 cdougherty statsrv01 A_ECG U_A_ECG 11MAR2003:10:17:56 cdougherty statsrv01 LAB U_LAB 11MAR2003:11:31:15 cdougherty statsrv01 A_LAB U_A_LAB 11MAR2003:11:36:37 cdougherty statsrv01 LABRANGE U_LABRNG 11MAR2003:11:40:41 cdougherty statsrv01 CONMEDS U_CONMED 11MAR2003:12:08:14 cdougherty statsrv01 DISP U_DISP 11MAR2003:12:22:44 cdougherty statsrv01 INCL U_INCLUS 11MAR2003:12:37:19 cdougherty statsrv01 MEDHIS U_MEDHIS 11MAR2003:12:44:11 cdougherty statsrv01 PKSAMP U_PKSAMP 11MAR2003:12:51:35 cdougherty statsrv01 8

9 SAS Conference Protocol DUMMY123 Jun 6, :36 Audit Trail # Title Audit Trail #3 - Final /Time Stamps Subset Name Run rammer AUDIT_TRAIL Summary of Subject Disposition All Enrolled SDISP :12:00:00 cdougherty statsrv Summary of Subject Enrollment All Enrolled SENRL :12:08:00 cdougherty statsrv Summary of Subject Demographics All Enrolled SDEM :11:59:00 cdougherty statsrv Summary of Medical History All Enrolled SMHX :12:13:00 cdougherty statsrv Summary of Abnormal Physical Examination Findings at Baseline All Treated SPE :12:15:00 cdougherty statsrv Summary of Study Drug Exposure All Treated SSMED :12:16:00 cdougherty statsrv Overall Summary of Treatment Emergent Adverse Events All Treated SAE :11:41:00 cdougherty statsrv Overall Summary of Treatment Emergent Study Drug Related Adverse Events Summary of Treatment Emergent Adverse Events and Their Relationship to Study Drug: No. Observed and Severity Summary of Treatment Emergent Adverse Events By System Organ Class, Preferred Term and Severity Summary of Treatment Emergent Study Drug Related Adverse Events By System Organ Class, Preferred Term All Treated SAE :11:41:00 cdougherty All Treated SAEPID :11:42:00 cdougherty All Treated SAES :11:44:00 cdougherty All Treated SAES :11:44:00 cdougherty Location ID statsrv01 statsrv01 statsrv01 statsrv01 9

10 SAS Conference Protocol DUMMY123 Jun 6, :36 Audit Trail Name Audit Trail #4 - Identify Which rams Use Which s A_AE Adverse Events Causing Death, and Other Serious or Significant Adverse Events LAE Overall Summary of Treatment Emergent Adverse Events Summary of Treatment Emergent Adverse Events and Their Relationship to Study Drug: No. Observed and Summary of Treatment Emergent Adverse Events By System Organ Class, Preferred Term and Severity Summary of Treatment Emergent Adverse Events by Frequency Summary of Treatment Emergent Adverse Events of Interest Summary of Treatment Emergent Laboratory Adverse Events Summary of Subject Disposition A_ECG 12-Lead ECG: Change from Baseline QTc Intervals LECGC A_LAB Summary of All Borderline and Prolonged QT Intervals Summary of Abnormal 12-Lead ECG Intervals at End of Study Summary of Changes in 12-Lead ECG Intervals Title Summary of Clinically Significant 12-Lead ECG Intervals per Investigator Listing of Clinical Laboratory Tests for with Treatment Emergent Clinically Significant Abnormal Laboratory Values per Investigator Summary of Treatment Emergent Clinically Significant Abnormal Laboratory Values per Investigator Summary of Clinical Laboratory Evaluations: Change from Baseline Summary of Laboratory Values >= 3 Times the Upper Limit of Reference Range MEDHIS General Medical History LMEDH Summary of Medical History AUDIT_TRAIL Pr og Name SAE SAEPID SAES SAESF SAESH SAESL SDISP SECG SECGA SECGC SECGCS LLABS SLABC SLABS SLABT SMHX 10

11 ACKLEDGMENTS SAS is a Registered Trademark of the SAS Institute, Inc. of Cary, North Carolina. Thanks to Spencer Hudson, whose idea it was to generate this audit trail system. Additional thanks to those who helped review this paper so that it could be possible to present at NESUG: Spencer Hudson, ViroPharma Incorporated; Heidi Shea, ViroPharma Incorporated; Dan Wang, ViroPharma Incorporated; Louisa Feeley, ViroPharma Incorporated; Souma Chattopadhyay, ViroPharma Incorporated; and Daphne Ewing, Synteract, Inc. CONTACT INFORMATION Your comments and questions are greatly valued and appreciated. Please contact the author at: Carolyn Dougherty ViroPharma Incorporated 405 Eagleview Boulevard Exton, PA Work Phone: (610) Fax: (610) Carolyn.Dougherty@ViroPharma.com Web: 11