MINI-SENTINEL COMMON DATA MODEL DATA QUALITY REVIEW AND CHARACTERIZATION PROCESS AND PROGRAMS

Transcription

1 MINI-SENTINEL COMMON DATA MODEL DATA QUALITY REVIEW AND CHARACTERIZATION PROCESS AND PROGRAMS Program Package version: 3.2 Prepared by the Mini-Sentinel Operations Center May 2014 Mini-Sentinel is a pilot project sponsored by the U.S. Food and Drug Administration (FDA) to inform and facilitate development of a fully operational active surveillance system, the Sentinel System, for monitoring the safety of FDA-regulated medical products. Mini-Sentinel is one piece of the Sentinel Initiative, a multi-faceted effort by the FDA to develop a national electronic system that will complement existing methods of safety surveillance. Mini-Sentinel Collaborators include Data and Academic Partners that provide access to health care data and ongoing scientific, technical, methodological, and organizational expertise. The Mini-Sentinel Coordinating Center is funded by the FDA through the Department of Health and Human Services (HHS) Contract number HHSF I.

2 Table of Contents I. OVERVIEW II. DATA QUALITY REVIEW PROCESS A. DISTRIBUTED PROGRAMMING AND DATA QUALITY CHECKS B. DATA PARTNER QUALITY REVIEW AND CHARACTERIZATION REPORT III. PROGRAM PACKAGE A. CORE TABLES QUALITY REVIEW B. LABORATORY TABLE QUALITY REVIEW C. VITAL SIGNS TABLE QUALITY REVIEW IV. PROGRAM EXECUTION V. PROGRAM PACKAGE OUTPUT A. CORE QUALITY REVIEW OUTPUT EXAMPLE B. LAB QUALITY REVIEW OUTPUT EXAMPLE C. VITAL SIGNS QUALITY REVIEW OUTPUT EXAMPLE VI. APPENDIX A: LIST OF ALL QA ERR CODES VII. APPENDIX B: LIST OF OUTPUT DATASETS Mini-Sentinel Common Data Model i Data Quality Review Process and Programs

3 Modification History Version Date Modification By 3.2 5/30/2014 Integrated Common Components Mini-Sentinel Operations Center Added MSCDM v4.0 compliance checks, minor bug fixes /03/2014 Implemented minor bug fixes Mini-Sentinel Operations Center /25/2013 Implemented minor bug fix Mini-Sentinel Operations Center /16/2013 Implemented minor bug fix Mini-Sentinel Operations Center /05/2013 Added/modified lab and vitals Mini-Sentinel Operations Center checks, fixed bugs /01/2013 Updated for UNIX compatibility Mini-Sentinel Operations Center /12/2012 Added PatID and ID Mini-Sentinel Operations Center matching, enhanced valid date checks, fixed bugs Added clinical data (labs and vitals) programs /14/2010 Added duplicate record checks, Mini-Sentinel Operations Center modified dataset names, fixed bugs /20/2010 Initial published version Mini-Sentinel Operations Center Mini-Sentinel Common Data Model ii Data Quality Review Process and Programs

4 I. OVERVIEW This document describes the process and version 3.2 program package used by the Mini-Sentinel Operations center (MSOC) for data quality review and characterization of the Mini-Sentinel Distributed Database (MSDD). To create the MSDD, each Data Partner transforms local source data into the Mini- Sentinel Common Data Model (MSCDM) format. The MSOC uses a set of data quality review and characterization programs (i.e., a program package ) to ensure that the MSDD meets reasonable standards for data transformation consistency and quality. II. DATA QUALITY REVIEW PROCESS A data quality review is conducted by MSOC when a Mini-Sentinel Data Partner updates their MSCDM database to include additional data. Following a database update, or refresh, a distributed program package developed by the MSOC is executed locally by the Data Partner, and aggregate output tables are returned to MSOC for review. MSOC analysts perform two independent reviews of the output and generate a consensus report of findings. MSOC and the Data Partner work closely together to resolve outstanding issues and approve the data for use in Mini-Sentinel data requests. Updated data are not used in requests until the refresh has been approved by MSOC. A. DISTRIBUTED PROGRAMMING AND DATA QUALITY CHECKS To evaluate data characteristics and quality, MSOC developed distributed code to query the content of MSCDM formatted tables. The distributed code generates aggregate output tables that help determine whether the data conform to MSCDM specifications, maintain integrity across variables and across tables, and trend as expected over time. Output tables are designed to evaluate one or more data checks, i.e., pre-defined data quality measures or characterizations. Each data check is designated a level 1, level 2, level 3, or level 4 data quality check depending on the complexity and assigned an Err Code for tracking and reporting purposes. An Err Code can represent a data characteristic or a data issue (see Section IIB for more information on Err Codes). Level 1 data checks review the completeness and content of each variable in each table to ensure that the required variables contain data and conform to the formats specified by the MSCDM specifications. For each MSCDM variable, level 1 data checks verify that data types, variable lengths, and SAS formats are correct and reported values are acceptable. For example, ensuring that the variable SEX in the Demographic table has a value of F, M, A, or U is a level 1 data check. Level 2 data checks assess the logical relationship and integrity of data values within a variable or between two or more variables within and between tables. For example, the MSCDM requires that the variable ADMITTING_SOURCE in the table is populated only for inpatient and institutional encounters (i.e., ENCTYPE= IP or IS ). A level 2 data check would ensure that ADMITTING_SOURCE is populated only when ENCTYPE= IP or IS. Level 3 data checks examine data distributions and trends over time, both within a Data Partner s database (by examining output by year and year/month) and across a Data Partner s databases (by Mini-Sentinel Common Data Model Data Quality Review Process and Programs

5 comparing updated MSCDM tables to previous versions of the tables). For example, a level 3 data check would ensure that there are no large, unexpected increases or decreases in diagnosis records over time. Level 4 data checks examine the occurrence and prevalence of nonsensical diagnoses and examine variations in care practices across data partners. Level 4 checks are designed to provide more targeted data analyses and profiling of Data Partner data, and are not necessarily designed to detect and correct errors. An example level 4 data check would examine the proportion of ovarian cancer diagnoses among men. B. DATA PARTNER QUALITY REVIEW AND CHARACTERIZATION REPORT Following review of the output tables and evaluation of the data checks, MSOC shares with the Data Partner a summary QA review report containing a list of questions, possible issues, and data characteristics. If there are data issues, the Data Partner will investigate and provide a written response in the report either explaining the results or proposing corrective action. All decisions and discussions are documented in the report in order to develop a knowledge repository about the Data Partner s data. Each data characteristic or issue in the report is identified by an Err Code. Err Codes are constructed of (in this order): the MSCDM table name abbreviation, the data check level (1, 2, 3, or 4), the variable number (as it appears in MSCDM specifications), and the data check number. For example, the Err Code DEM1.3.2 denotes the second level 1 check performed on the variable SEX in the Demographic table. This check ensures that the variable SEX in the Demographic table is 1 character in length. Using Err Codes for tracking allows for easy assessment of data trends over time and across Data Partners. III. PROGRAM PACKAGE The data quality review and characterization program package contains three sub-packages that query MSCDM tables. The Core quality review sub-package queries the Enrollment, Demographic, Dispensing,, Diagnosis, Procedure, Death, and Cause of Death tables. The Laboratory quality review sub-package queries the table, and the Vital Signs quality review sub-package queries the Vital Signs table. All programs are based on MSCDM version 4.0. A. CORE TABLES QUALITY REVIEW The Core quality review sub-package is executed first by all Data Partners via the Master SAS Program. A total of 20 SAS QA programs are included in the sub-package: Master SAS Program (00.0_mscdm_data_qa_review_master_file.sas). Selectively executes programs included in the sub-package in accordance to an MSOC-provided control flow input file (control_flow.sas7bdat). Requires editing by the Data Partner to identify location of the MSCDM tables in staging, as well as the location of the Common Components program. QA Programs: o Enrollment table QA program (01.1_mscdm_data_qa_review-enrollment.sas). Queries the number of members and records in the Enrollment table, and outputs information on medical and drug coverage indicators, enrollment start, end, and length. Mini-Sentinel Common Data Model Data Quality Review Process and Programs

6 o Demographic table QA program (02.1_mscdm_data_qa_review-demographic.sas). Queries the number of members and records in the Demographic table, and outputs information on age, sex, and race. o Dispensing table QA program (03.1_mscdm_data_qa_review-dispensing.sas). Queries the number of members and records in the Dispensing table, and outputs information on dispensing date, dispensings over time, dispensings per member over time, days supply and dispensed amount. o table QA program (04.1_mscdm_data_qa_review-encounter.sas). Queries the number of members, records, encounters, and providers in the table, and outputs information on admission and discharge date, encounters over time, encounter type, length of stay, facility location, admitting source, discharge status and disposition, DRG and DRG type, and number of encounters per member. o Diagnosis table QA program (04.2_mscdm_data_qa_review-diagnosis.sas). Queries the number of members, records, encounters, and providers in the Diagnosis table, and outputs information on encounter type, admission date, diagnosis code and type, principal diagnosis indicators, number of diagnoses per encounter, and number of diagnoses over time. o Procedure table QA program (04.3_mscdm_data_qa_review-procedure.sas). Queries the number of members, records, encounters, and providers in the Procedure table, and outputs information on encounter type, admission date, procedure code and type, number of procedures per encounter, and number of procedures over time. o Death table QA program (05.1_mscdm_data_qa_review-death.sas). Queries the number of members and records in the Death table, and outputs information on the source of and confidence in death information, number of deaths over time, and if the death date has been imputed. o Cause of death table QA program (05.2_mscdm_data_qa_review-causeofdeath.sas). Queries the number of members and records in the Cause of Death table, and outputs information on cause of death codes and cause type, and source of and confidence in cause of death information. o Level 1 and 2 checks QA program (99.1_mscdm_data_qa_review-l1_l2.sas). Queries all Core MSCDM tables to perform level 1 and level 2 checks not performed elsewhere and sets all level 1 and level 2 flags into one dataset. This program also produces metadata. This module will always execute if a QA module for any Core tables are executed. o Level 4 checks QA program (08.1_mscdm_data_qa_review-level4). Queries all Core MSCDM tables to perform level 4 checks. Must be run independently of the Master SAS Program. o Min-Max Dates program (99.2_mscdm_data_qa_review_minmax_dates.sas). Queries all Core MSCDM tables and outputs minimum and maximum dates in each, and calculates DP Min (calculated as the maximum of the Min Dates) and DP Max (calculated as the minimum of the Max dates). Control Flow Program (00.0_mscdm_control_flow.sas). Contains a program that allows selective and sequential execution of QA modules. Standard Macros Program (00.1_mscdm_standard_macros.sas). Contains macros used across all Core QA programs. Standard Formats Program (00.1_mscdm _formats.sas). Contains formats used across all Core QA programs. Log Checker Program (00.3_mscdm_sas_log_checker_directory_cc.sas). Checks program log and summarizes notes, warnings, and errors in an output PDF file. Mini-Sentinel Common Data Model Data Quality Review Process and Programs

7 Signature Request Program (00.4_mscdm_qasignaturerequest.sas). Creates signature files containing metadata for each MSCDM table. The Core quality review sub-package also contains three input files: Error code crosswalk (flags_xwalk.sas7bdat). Assigns appropriate Err Codes to all checks performed in the QA programs. See Appendix A for a list of Core quality review Err Codes. Level 4 codes (level4lookup.sas7bdat). Includes the code lists for conditions of interest included in Level 4 data checks. Control flow (control_flow.sas7bdat). Controls selective and sequential execution of QA modules. B. LABORATORY TABLE QUALITY REVIEW The Laboratory quality review sub-package is executed second and only by Data Partners who have lab data available. A total of 3 SAS programs are included in the sub-package: Lab QA Program (06.1_mscdm_data_qa_review-labs.sas). Queries the number of members and records in the table and outputs information on lab tests included, result units, and available dates (i.e., date of lab vs. order date vs. result date). Many of the checks included in the program check compliance with MSCDM specifications (e.g., appropriate length, type, and format) and assess across-variable integrity (e.g., if individual tests have valid information populated for specimen source and test subtype; if quantitative tests only contain numeric results). Lab Standard Macros (06.1_mscdm_laboratory_macros.sas). Contains macros used in the Lab QA program. Log Checker Program (00.3_mscdm_sas_log_checker_directory_cc.sas). Checks Lab QA program log and appends notes, warnings, and errors to the output.pdf file generated by the Core QA program. The Laboratory quality review sub-package also contains two input files: - LOINC lookup (loinc_lkp.sas7bdat). Contains a lookup table for specimen source and LOINC codes for each laboratory test and sub category. - Procedure lookup (px_lkp.sas7bdat). Includes procedure code list associated with laboratory tests. C. VITAL SIGNS TABLE QUALITY REVIEW The Vital Signs quality review sub-package is executed third and only by Data Partners who have vital signs data available. A total of 2 SAS programs are included in the sub-package: Vital Signs QA Program (07.1_mscdm_data_qa_review-vitals.sas). Queries the number of members and records in the Vital Signs table and outputs the frequency of height, weight, diastolic blood pressure, systolic blood pressure, and tobacco use measurements by age group, sex, and year. Mini-Sentinel Common Data Model Data Quality Review Process and Programs

8 Log Checker Program (00.3_mscdm_sas_log_checker_directory_cc.sas). Checks Vital Signs QA program log and appends notes, warnings, and errors to the output.pdf file generated by the Core QA and (if applicable) Lab QA program. IV. PROGRAM EXECUTION When implementing programs within the MSDD, the MSOC uses a uniform folder structure across Data Partners to facilitate communications between MSOC and Data Partners and to streamline file management. Data Partners create a folder named after the quality review package and several subfolders to organize program inputs and outputs. One of the folders contains output to be sent to MSOC and another contains intermediate files that remain with the Data Partner, but could be used to facilitate follow-up queries if necessary. Table 1 defines the variables that must be initialized by the user to execute the quality review and characterization program package (i.e., defined by the Data Partner before execution of the programs). Please note that these values cannot be left blank. Each Data Partner is required to enter user inputs at the beginning of the applicable data quality program. These inputs are unique to each Data Partner. Table 1. Master Program Variable Definitions Label Field Name Description Common Components Path MSCC Path to directory containing the Common Components file (ms_common_components.sas) Staging MSCDM Path STAGING Path to directory containing the MSCDM datasets pending QA review V. PROGRAM PACKAGE OUTPUT Execution of all sections of the data quality review and characterization program package generates up to 279 output files: 154 Core output tables, 73 Lab output tables, and 21 Vital Signs output tables. In addition, 15 logs, 13 signature files, 1 lab output summary PDF file, and 1 master output summary PDF file are generated. See Appendix B for a complete list of output files. An example output table for each sub-package is provided below to demonstrate how output tables are used during data quality review. A. CORE QUALITY REVIEW OUTPUT EXAMPLE The output dataset dia_l3_pdx_et.sas examines the frequency of records in the Diagnosis table by principal diagnosis flag (PDX) and encounter type (ENCTYPE), and helps evaluate compliance with MSCDM specifications. Table 2: Example Dia_l3_pdx_et Output PDX EncType Count Percent AV 100, ED 25, OA 40, Mini-Sentinel Common Data Model Data Quality Review Process and Programs

9 PDX EncType Count Percent P IP 1, P IS 6, S IP 3, S IS 12, Per MSCDM specifications, only inpatient and institutional stay encounters (ENCTYPE= IP and IS ) should have PDX values populated. Ambulatory and emergency department visits (ENCTYPE= AV, OA, and ED ) should have blank PDX values. The table above represents a Diagnosis table with compliant PDX*ENCTYPE values. In addition to evaluating MSCDM compliance, this table also helps MSOC characterize the data. For example, the output indicates that only 4.0% of diagnoses are flagged as principal (PDX= P ), 8.0% are flagged as secondary (PDX= S ), and no diagnoses are flagged as unable to classify (PDX= X ). These percentages can inform investigator decisions to construct cohorts based on PDX values. B. LAB QUALITY REVIEW OUTPUT EXAMPLE The output dataset lab_l2_test_rslttyp_specimen.sas examines the frequency of records in the table by test name (MS_TEST_NAME), test subcategory (MS_TEST_SUB_CATEGORY), specimen source (SPECIMEN_SOURCE), and result type for lab tests with quantitative results. Table 3: Example lab_l2_test_rslttyp_specimen Output MS Test Name MS Test Subcategory Specimen Source Result Type Number of Tests GLUCOSE FST SR_PLS N 400 GLUCOSE RAN BLOOD N 600 HGBA1C BLOOD N 500 HGBA1C BLOOD M 25 PLATELETS PLASMA N 300 PG_QN BHCG SERUM N 200 PG_QN BHCG URINE N 300 PG_QN HCG SERUM N 450 A result type value of N indicates a numeric test result and M indicates a missing numeric test result. Since allowable MS_TEST_SUB_CATEGORY and SPECIMEN_SOURCE values are conditional on the test (e.g., MS_TEST_SUB_CATEGORY should be blank for HGBA1C tests but equal to either FST or RAN for GLUCOSE tests), this output table allows MSOC to evaluate compliance with MSCDM specifications. In addition, result type indicates which quantitative tests have missing result values. In the example output above, all test name, test subcategory, and specimen source values comply with the MSCDM. However, there are 25 HGBA1C lab records that have missing results that should be investigated. Mini-Sentinel Common Data Model Data Quality Review Process and Programs

10 C. VITAL SIGNS QUALITY REVIEW OUTPUT EXAMPLE The output dataset vit_l3_num_wt_age_sex_y.sas examines the frequency of weight measurements taken by age group, sex and year. Given the amount of data included in this table, the example output below is partially populated and includes information only for year old females in 2006 and 2007: Table 4: Example vit_l3_num_wt_age_sex_y Output Number of Weight Year Age Category Sex Count Percent Measurements F F F F F 2, F 2, F F F 3, F 4, F 3, F 3, This table is useful to understand how the number of weight measurements varies by age group, sex, and year. Unlike the other two example output tables, this output is used for characterization purposes only (i.e., it is not evaluating compliance with the MSCDM or looking for errors). In the table above, there are 2, year old female members with six weight measurements in 2006, compared to 50 female members with only one weight measurement in the same year. In 2007, there are 3, year old female members with six weight measurements, compared to 68 female members with only one weight measurement in the same year. Mini-Sentinel Common Data Model Data Quality Review Process and Programs

11 VI. APPENDIX A: LIST OF ALL QA ERR CODES Full Err Code ALL2.1.1 PatID variable has inconsistent lengths across tables ENR_DEM At least one PatID in the ENR table is not in the DEM table ENR_DIS At least one PatID in the ENR table is not in the DIS table ENR_ENC At least one PatID in the ENR table is not in the ENC table ENR_DIA At least one PatID in the ENR table is not in the DIA table ENR_PRO At least one PatID in the ENR table is not in the PRO table ENR1.0.0 Table does not exist ENR1.1.1 PatID variable is not character type ENR1.1.2 PatID variable has missing values ENR1.1.3 PatID variable has non-missing values that are not left-justified ENR1.1.4 PatID variable contains special characters ENR1.2.1 Enr_Start variable is not SAS date value of numeric data type ENR1.2.2 Enr_Start variable is not of length 4 ENR1.2.3 Enr_Start variable has missing values ENR1.3.1 Enr_End variable is not SAS date value of numeric data type ENR1.3.2 Enr_End variable is not of length 4 ENR1.3.3 Enr_End variable has missing values ENR1.3.6 Enr_End variable has values after the maximum dispensing and utilization dates ENR1.4.1 MedCov variable is not character type ENR1.4.2 MedCov variable is not exactly 1 character in length ENR1.4.3 MedCov variable has missing values ENR1.4.4 MedCov variable has values other than "Y", "N", or "U" ENR1.5.1 DrugCov variable is not character type ENR1.5.2 DrugCov variable is not exactly 1 character in length ENR1.5.3 DrugCov variable has missing values ENR1.5.4 DrugCov variable has values other than "Y", "N", or "U" ENR1.6.1 Chart variable is not character type ENR1.6.2 Chart variable is not exactly 1 character in length ENR1.6.3 Chart variable has missing values ENR1.6.4 Chart variable has values other than "Y" or "N" ENR2.0.0 Record(s) have duplicate key value combinations (with respect to table definition) ENR2.2.1 Enr_Start is after Enr_End ENR2.2.3 Enr_Start occurs more than once in the file in combination with PatID, MedCov, and DrugCov Mini-Sentinel Common Data Model Data Quality Review Process and Programs

12 ENR2.3.4 ENR2.6.1 ENR3.0.1 ENR3.1.1 ENR3.2.1 ENR3.3.4 ENR3.4.1 ENR3.4.2 ENR3.4.3 ENR3.4.4 ENR3.4.5 ENR3.4.6 ENR3.5.1 ENR3.5.2 ENR3.5.3 ENR3.5.4 ENR3.5.5 ENR3.5.6 ENR3.6.1 ENR3.6.2 ENR3.6.3 ENR3.6.4 ENR3.6.5 ENR3.6.6 ENR3.6.7 ENR3.6.8 ENR_DEM2.1.1 Enr_End occurs more than once in the file in combination with PatID, MedCov, and DrugCov At least one PatID has records with inconsistent MedCov and DrugCov indicators with the same year/month/day of enrollment Significant change in number of records between Problem with number of unique PatIDs Problem with distribution of Enr_Start variable Problem with distribution of Enr_End variable Problem with distribution of enrollment months per patient for patients with MedCov = "Y" Problem with distribution of MedCov indicators Problem with distribution of MedCov indicators per year Problem with distribution of MedCov indicators per year-month within the Problem with distribution of MedCov indicators per year across Problem with distribution of MedCov indicators per year-month across Problem with distribution of enrollment months per patient for patients with DrugCov = "Y" Problem with distribution of DrugCov indicators Problem with distribution of DrugCov indicators per year Problem with distribution of DrugCov indicators per year-month within the Problem with distribution of DrugCov indicators per year across Problem with distribution of DrugCov indicators per year-month across Problem with distribution of enrollment months per patient for patients with MedCov = "Y" and DrugCov = "Y" Problem with distribution of MedCov*DrugCov indicators Problem with distribution of MedCov*DrugCov indicators per year within the Problem with distribution of MedCov*DrugCov indicators per year-month Problem with distribution of MedCov*DrugCov indicators per year across Problem with distribution of MedCov*DrugCov indicators per year-month across Problem with distribution of total number of patients with MedCov = "Y" and DrugCov = "Y" and at least 1 overlapping enrollment span Problem with distribution of total number of overlapping enrollment spans for all patients with MedCov = "Y" and DrugCov = "Y" At least one PatID in the DEM table is not in the ENR table Mini-Sentinel Common Data Model Data Quality Review Process and Programs

13 DEM1.0.0 Table does not exist DEM1.1.1 PatID variable is not character type DEM1.1.2 PatID variable has missing values DEM1.1.3 PatID variable has non-missing values that are not left-justified DEM1.1.4 PatID variable contains special characters DEM1.2.1 Birth_Date variable is not SAS date value of numeric data type DEM1.2.2 Birth_Date variable is not of length 4 DEM1.2.3 Birth_Date variable has missing values DEM1.2.4 Birth_Date variable has values before 1/1/1885 DEM1.3.1 Sex variable is not character type DEM1.3.2 Sex variable is not exactly 1 character in length DEM1.3.3 Sex variable has missing values DEM1.3.4 Sex has values other than "F", "M", "A", or "U" DEM1.4.1 Hispanic variable is not character type DEM1.4.2 Hispanic variable is not exactly 1 character in length DEM1.4.3 Hispanic variable has missing values DEM1.4.4 Hispanic variable has values other than "Y", "N", or "U" DEM1.5.1 Race variable is not character type DEM1.5.2 Race variable is not exactly 1 character in length DEM1.5.3 Race variable has missing values DEM1.5.4 Race variable has values other than "0", "1", "2", "3", "4", or "5" DEM1.6.1 ZIP variable is not character type DEM1.6.2 ZIP variable is not exactly 5 characters in length DEM1.6.3 ZIP variable has missing values DEM1.7.1 ZIP_Date variable is not SAS date value of numeric data type DEM1.7.2 ZIP_Date variable is not length of 4 DEM1.7.3 ZIP_Date variable has missing values DEM1.7.4 ZIP_Date variable has values before MinDate DEM1.7.5 ZIP_Date variable has values after MaxDate DEM2.0.0 Record(s) have duplicate key value combinations (with respect to table definition) DEM3.0.1 Significant change in number of records between DEM3.1.1 Problem with number of unique PatIDs DEM3.2.2 Problem with distribution of age DEM3.2.3 Problem with distribution of age groups (0-1 yrs, 2-4 yrs, 5-9 yrs, yrs, yrs, yrs, yrs, yrs, yrs, 75+ yrs) DEM3.3.1 Problem with distribution of Sex variable DEM3.4.1 Problem with distribution of Race variable DEM3.5.1 Problem with distribution of Hispanic variable ENR_DIS2.1.2 At least one PatID in the DIS table is not in the ENR table Mini-Sentinel Common Data Model Data Quality Review Process and Programs

14 DIS1.0.0 Table does not exist DIS1.1.1 PatID variable is not character type DIS1.1.2 PatID variable has missing values DIS1.1.3 PatID variable has non-missing values that are not left-justified DIS1.1.4 PatID variable contains special characters DIS1.2.1 RxDate variable is not a SAS date value of numeric data type DIS1.2.2 RxDate variable is not of length 4 DIS1.2.3 RxDate variable has missing values DIS1.3.1 NDC variable is not character data type DIS1.3.2 NDC variable is not exactly 11 characters in length DIS1.3.3 NDC variable has missing values DIS1.3.4 NDC variable contains special characters or non-digits DIS1.4.1 RxSup variable is not numeric type DIS1.4.2 RxSup variable is not of length 4 DIS1.4.3 RxSup variable has negative, missing, or zero values DIS1.4.4 RxSup variable has a format length (no format length should be specified) DIS1.5.1 RxAmt variable is not numeric type DIS1.5.2 RxAmt variable is not of length 4 DIS1.5.3 RxAmt variable has negative, missing or zero values DIS1.5.4 RxAmt variable has a format length (no format length should be specified) DIS2.0.0 DIS3.0.1 DIS3.1.1 DIS3.2.1 DIS3.2.2 DIS3.2.3 DIS3.2.4 DIS3.2.5 DIS3.2.6 DIS3.4.1 DIS3.5.1 DIS3.5.2 ENR_ENC2.1.3 ENC_DIA_PRO2.2.1 Record(s) have duplicate key value combinations (with respect to table definition) Significant change in number of records between Problem with number of unique PatIDs Problem with distribution of RxDate (i.e. total number of dispensings per year) Problem with distribution of RxDate (i.e. total number of dispensings per year-month) Significant change in number of records per RxDate (year) across Significant change in number of records per RxDate (year-month) across Problem with distribution of RxDate (overall) Problem with distribution of RxDate (overall) across Problem with distribution of RxSup Problem with distribution of RxAmt Problem with average number of prescriptions per PatID by year At least one PatID in the ENC table is not in the ENR table ID variable has inconsistent lengths across tables Mini-Sentinel Common Data Model Data Quality Review Process and Programs

15 ENC_DIA_PRO2.2.6 ENC_DIA_PRO2.5.1 ENC_DIA2.2.4 ENC_PRO2.2.5 ENC1.0.0 ENC1.1.1 ENC1.1.2 ENC1.1.3 ENC1.1.4 ENC1.2.1 ENC1.2.2 ENC1.2.3 ENC table has ID values which are not found in the DIA and PRO tables Provider variable has inconsistent lengths across tables ENC table has ID values not found in the DIA table ENC table has ID values not found in the PRO table Table does not exist PatID variable is not character type PatID variable has missing values PatID variable has non-missing values that are not left-justified PatID variable contains special characters ID variable is not character type ID variable has missing values ID variable has non-missing values that are not left-justified ENC1.2.4 ID variable contains special characters ENC1.3.1 ADate variable is not SAS date value of numeric data type ENC1.3.2 ADate variable is not of length 4 ENC1.3.3 ADate variable has missing values ENC1.4.1 DDate variable is not SAS date value of numeric data type ENC1.4.2 DDate variable is not of length 4 ENC1.5.1 Provider variable is not character type ENC1.5.2 Provider variable has missing values and should be populated for all records of the table (part of the definition of the table) ENC1.5.3 Provider variable has non-missing values that are not left-justified ENC1.5.4 Provider variable contains special characters ENC1.6.1 Facility_Location variable is not character type ENC1.6.2 Facility_Location variable is not exactly 3 characters in length ENC1.6.3 Facility_Location variable contains non-digits ENC1.6.4 Facility_Location has missing values ENC1.6.5 Facility_Location variable has non-missing values that are not left-justified ENC1.7.1 ENC1.7.2 ENC1.7.3 ENC1.7.4 ENC1.8.1 ENC1.8.2 ENC1.8.3 ENC1.8.4 ENC1.9.1 EncType variable is not character type EncType variable is not exactly 2 characters in length EncType variable has missing values EncType variable has values other than "IP", "IS", "ED", "AV", or "OA" Facility_Code variable is not character type Facility_Code variable contains special characters Facility_Code variable has non-missing values that are not left-justified Facility_Code has missing values Discharge_Disposition variable is not character type Mini-Sentinel Common Data Model Data Quality Review Process and Programs

16 ENC1.9.2 ENC1.9.3 ENC ENC ENC ENC ENC ENC ENC ENC ENC ENC ENC ENC ENC2.0.0 ENC2.2.1 ENC2.2.7 ENC2.3.1 ENC2.3.2 ENC2.4.1 ENC2.4.2 ENC2.9.1 ENC2.9.2 ENC ENC ENC ENC ENC ENC ENC Discharge_Disposition variable is not exactly 1 character in length Discharge_Disposition variable has values other than "A", "E", "U", or missing Discharge_Status variable is not character data type Discharge_Status variable is not exactly 2 characters in length Discharge_Status variable has values other than "AF", "AL", "AM", "AW", "EX", "HH", "HS", "HO", "IP", "NH", "OT", "RS", "RH", "SH", "SN", UN, or missing DRG variable is not character type DRG variable is not exactly 3 characters in length DRG variable contains non-digits DRG_Type variable is not character type DRG_Type variable is not exactly 1 character in length DRG_Type variable contains values other than "1", "2", or missing Admitting_Source variable is not character type Admitting_Source variable is not exactly 2 characters in length Admitting_Source variable contains values other than "AV", "ED", "AF", "AL", "HH", "HS", "HO", "IP", "NH", "OT", "RS", "RH", "SN", "UN", or missing Record(s) have duplicate key value combinations (with respect to table definition) ID values occur more than once ID variable does not contain (in this order) PatID, ADate, Provider, EncType ADate is after DDate (for IP and IS only) ADate and DDate variables have values before DP_MinDate DDate variable is missing for EncType value "IP" DDate variable is populated for records with EncType values other than "IP" or "IS" Discharge_Disposition variable is missing for EncType values "IP" or "IS" Discharge_Disposition variable is populated for records with EncType values other than "IP" or "IS" Discharge_Status variable is missing for EncType values "IP" or "IS" Discharge_Status variable is populated for records with EncType values other than "IP" or "IS" DRG variable is missing for EncType values "IP" or "IS" DRG variable is populated for records with EncType values other than "IP", "IS", or "ED" DRG_Type variable is missing for EncType values "IP" or "IS" DRG_Type variable is populated for records with EncType values other than "IP", "IS", or "ED" Admitting_Source variable is missing for EncType values "IP" or "IS" Mini-Sentinel Common Data Model Data Quality Review Process and Programs

17 ENC ENC3.0.1 ENC3.1.1 ENC3.2.1 ENC3.3.1 ENC3.3.2 ENC3.3.3 ENC3.3.4 ENC3.3.5 ENC3.3.6 ENC3.4.1 ENC3.4.5 ENC3.4.6 ENC3.4.7 ENC3.4.8 ENC3.4.9 ENC ENC ENC ENC ENC3.5.1 ENC3.7.4 ENC3.7.5 ENC3.7.6 ENC3.7.7 ENC3.7.8 ENC3.7.9 ENC3.9.1 Admitting_Source variable is populated for records with EncType values other than "IP" or "IS" Significant change in number of records between Problem with number of unique PatIDs Problem with number of unique IDs Problem with distribution of ADate (i.e. total number of records per year) Problem with distribution of ADate (i.e. total number of records per yearmonth) Significant change in number of records per ADate (year) across Significant change in number of records per ADate (year-month) across Problem with distribution of ADate (overall) Problem with distribution of ADate (overall) across Problem with distribution of DDate (i.e. total number of records per year) Problem with distribution of DDate (i.e. total number of records per yearmonth) Significant change in number of records per DDate (year) across Significant change in number of records per DDate (year-month) across Problem with distribution of DDate (overall) Problem with distribution of DDate (overall) across Problem with distribution of DDate variable by EncType per year Problem with distribution of DDate variable by EncType per year-month Problem with distribution of length of stay (DDate-ADate + 1) by EncType Problem with distribution of length of stay (DDate-ADate + 1) by EncType per year Problem with number of unique Provider values Problem with distribution of number of encounters per member by year and EncType Problem with distribution of number of encounters per member by yearmonth and EncType Problem with distribution of number of records by EncType per year within the Problem with distribution of number of records by EncType per year-month Significant change in number of records by EncType per year across Significant change in number of records by EncType per year-month across Problem with distribution of Discharge_Disposition variable (overall) Mini-Sentinel Common Data Model Data Quality Review Process and Programs

18 ENC3.9.2 ENC ENC ENC ENC ENC ENC ENC ENC ENR_DIA2.1.4 ENC_DIA2.2.2 DIA1.0.0 DIA1.1.1 DIA1.1.2 DIA1.1.3 DIA1.1.4 DIA1.2.1 DIA1.2.2 DIA1.2.3 Problem with distribution of Discharge_Disposition variable (by EncType) Problem with distribution of Discharge_Status variable (overall) Problem with distribution of Discharge_Status variable (by EncType) Problem with distribution of DRG variable Problem with distribution of DRG_Type variable (overall) Problem with distribution of DRG_Type variable (by year to confirm switch from old to new system) Problem with distribution of DRG_Type variable (by EncType) Problem with distribution of Admitting_Source variable (overall) Problem with distribution of Admitting_Source variable (by EncType) At least one PatID in the DIA table is not in the ENR table DIA table has ID values not found in the ENC table Table does not exist PatID variable is not character type PatID variable has missing values PatID variable has non-missing values that are not left-justified PatID variable contains special characters ID variable is not character type ID variable has missing values ID variable has non-missing values that are not left-justified DIA1.2.4 ID variable contains special characters DIA1.3.1 ADate variable is not SAS date value of numeric data type DIA1.3.2 ADate variable is not of length 4 DIA1.3.3 ADate variable has missing values DIA1.4.1 Provider variable is not character type DIA1.4.2 Provider variable has missing values and should be populated for all records of the table (part of the definition of the table) DIA1.4.3 Provider variable has non-missing values that are not left-justified DIA1.4.4 Provider variable contains special characters DIA1.5.1 EncType variable is not character type DIA1.5.2 EncType variable is not exactly 2 characters in length DIA1.5.3 EncType variable has missing values and should be populated for all records of the table (part of the definition of the table) DIA1.5.4 EncType variable has values other than "IP", "IS", "ED", "AV", or "OA" DIA1.6.1 DX variable Is not character type DIA1.6.2 DX variable is not exactly 18 characters in length DIA1.6.3 DX variable has missing values and should be populated for all records of the table (part of the definition of the table) DIA1.6.4 DX variable contains special characters other than a decimal point Mini-Sentinel Common Data Model Data Quality Review Process and Programs

19 DIA1.6.5 DIA1.7.1 DIA1.7.2 DIA1.7.3 DIA1.8.1 DIA1.9.1 DIA1.9.2 DIA1.9.3 DIA2.0.0 DIA2.2.6 DIA2.9.2 DIA2.9.1 DIA3.0.1 DIA3.1.1 DIA3.2.1 DIA3.3.1 DIA3.3.2 DIA3.3.3 DIA3.3.4 DIA3.3.5 DIA3.3.6 DIA3.4.1 DIA3.5.1 DIA3.5.2 DIA3.5.3 DIA3.5.4 DIA3.5.5 DIA3.6.1 DIA3.6.2 DIA3.7.1 DIA3.7.2 Problem with the length of DX values Dx_Codetype variable is not character type Dx_Codetype variable is not exactly 2 characters in length Dx_Codetype variable contains values other than "09", "10", "11", "SM", or "OT" OrigDX variable is not character type PDX variable is not character type PDX variable is not exactly 1 character in length PDX variable contains values other than "P", "S", "X", or missing Record(s) have duplicate key value combinations (with respect to table definition) ID variable does not contain (in this order) PatID, ADate, Provider, EncType PDX variable is populated for EncType values other than "IP" or "IS" PDX variable has missing values for EncType values "IP" or "IS" Significant change in number of records between Problem with number of unique PatIDs Problem with number of unique IDs Problem with distribution of ADate (i.e. total number of records per year) Problem with distribution of ADate (i.e. total number of records per yearmonth) Significant change in number of records per ADate (year) across Significant change in number of records per ADate (year-month) across Problem with distribution of ADate (overall) Problem with distribution of ADate (overall) across Problem with number of unique Provider values Problem with distribution of number of records by EncType per year within the Problem with distribution of number of records by EncType per year-month Significant change in number of records by EncType per year across Significant change in number of records by EncType per year-month across Problem with distribution of number of unique diagnoses per encounter visit (overall) Problem with distribution of DX variable DX variable is not consistent with Dx_Codetype variable Problem with distribution of Dx_Codetype (by year) Problem with distribution of Dx_Codetype (by EncType) Mini-Sentinel Common Data Model Data Quality Review Process and Programs

20 DIA3.9.1 DIA3.9.2 ENR_PRO2.1.5 ENC_PRO2.2.3 PRO1.0.0 PRO1.1.1 PRO1.1.2 PRO1.1.3 PRO1.1.4 PRO1.2.1 PRO1.2.2 PRO1.2.3 Problem with distribution of PDX variable. Problem with distribution of PDX*EncType At least one PatID in the PRO table is not in the ENR table PRO table has ID values not found in the ENC table Table does not exist PatID variable is not character type PatID variable has missing values PatID variable has non-missing values that are not left-justified PatID variable contains special characters ID variable is not character type ID variable has missing values ID variable has non-missing values that are not left-justified PRO1.2.4 ID variable contains special characters PRO1.3.1 ADate variable is not SAS date value of numeric data type PRO1.3.2 ADate variable is not of length 4 PRO1.3.3 ADate variable has missing values PRO1.4.1 Provider variable is not character type PRO1.4.2 Provider variable has missing values and should be populated for all records of the table (part of the definition of the table) PRO1.4.3 Provider variable has non-missing values that are not left-justified PRO1.4.4 Provider variable contains special characters PRO1.5.1 EncType variable is not character type PRO1.5.2 EncType variable is not exactly 2 characters in length PRO1.5.3 EncType variable has missing values PRO1.5.4 EncType variable has values other than "IP", "IS", "ED", "AV", or "OA" PRO1.6.1 PX variable Is not character type PRO1.6.2 PX variable is not exactly 11 characters in length PRO1.6.3 PX variable has missing values PRO1.6.4 PX variable contains special characters other than a decimal point PRO1.6.5 Problem with the length of PX values PRO1.7.1 PX_Codetype variable Is not character type PRO1.7.2 PX_Codetype variable is not exactly 2 characters in length PRO1.7.3 PX_Codetype variable has missing values PRO1.7.4 PX_Codetype variable contains values other than "09", "10", "11", "C2", "C3", "C4", "H3", "HC", "LC", "LO", "ND", "OT", or "RE" PRO1.8.1 OrigPX variable is not character type PRO2.0.0 Record(s) have duplicate key value combinations (with respect to table definition) PRO2.2.1 ID variable does not contain (in this order) PatID, ADate, Provider, EncType Mini-Sentinel Common Data Model Data Quality Review Process and Programs

21 PRO3.0.1 PRO3.1.1 PRO3.2.1 PRO3.3.1 PRO3.3.2 PRO3.3.3 PRO3.3.4 PRO3.3.5 PRO3.3.6 PRO3.4.1 PRO3.5.1 PRO3.5.2 PRO3.5.3 Significant change in number of records between Problem with number of unique PatIDs Problem with number of unique IDs Problem with distribution of ADate (i.e. total number of records per year) Problem with distribution of ADate (i.e. total number of records per yearmonth) Significant change in number of records per ADate (year) across Significant change in number of records per ADate (year-month) across Problem with distribution of ADate (overall) Problem with distribution of ADate (overall) across Problem with number of unique Provider values Problem with distribution of number of records by EncType per year within the Problem with distribution of number of records by EncType per year-month Significant change in number of records by EncType per year across PRO3.5.4 Significant change in number of records by EncType per year-month across PRO3.5.5 Problem with distribution of number of unique procedures per encounter visit (overall) PRO3.6.1 Problem with distribution of PX variable PRO3.6.2 PX variable is not consistent with PX_Codetype variable PRO3.7.1 Problem with distribution of PX_Codetype (year) PRO3.7.2 Problem with distribution of PX_Codetype (by EncType) ENR_DTH2.1.6 At least one PatID in the DTH table is not in the ENR table DTH_COD2.1.1 At least one PatID in the DTH table is not in the COD table (if both tables exist) DTH1.1.1 PatID variable is not character type DTH1.1.2 PatID variable has missing values DTH1.1.3 PatID variable has non-missing values that are not left-justified DTH1.1.4 PatID variable contains special characters DTH1.2.1 DeathDt variable is not SAS date value of numeric data type DTH1.2.2 DeathDt variable is not of length 4 DTH1.2.3 DeathDt variable has missing values DTH1.2.4 DeathDt variable has values before 1/1/1885 DTH1.3.1 DtImpute variable is not character type DTH1.3.2 DtImpute variable is not exactly 1 character in length DTH1.3.3 DtImpute variable has missing values DTH1.3.4 DtImpute variable has values other than "B", "D", "M", or "N" Mini-Sentinel Common Data Model Data Quality Review Process and Programs

22 DTH1.4.1 Source variable is not character type DTH1.4.2 Source variable is not exactly 1 character in length DTH1.4.3 Source variable has missing values DTH1.4.4 Source variable has values other than "L", "N", "S", or "T" DTH1.5.1 Confidence variable is not character type DTH1.5.2 Confidence variable is not exactly 1 character in length DTH1.5.3 Confidence variable has missing values DTH1.5.4 Confidence variable has values other than "E", "F", or "P" DTH2.0.0 Record(s) have duplicate key value combinations (with respect to table definition) DTH3.0.1 Significant change in number of records between DTH3.1.1 Problem with number of unique PatIDs DTH3.2.1 Problem with distribution of DeathDt variable by year DTH3.2.2 Problem with distribution of DeathDt variable by year-month DTH3.3.1 Problem with distribution of DtImpute DTH3.4.1 Problem with distribution of Source DTH3.5.1 Problem with distribution of Confidence ENR_COD2.1.7 At least one PatID in the COD table is not in the ENR table DTH_COD2.1.2 At least one PatID in the COD table is not in the DTH table (if both tables exist) COD1.1.1 PatID variable is not character type COD1.1.2 PatID variable has missing values COD1.1.3 PatID variable has non-missing values that are not left-justified COD1.1.4 PatID variable contains special characters COD1.2.1 COD variable is not character type COD1.2.2 COD variable has missing values COD1.2.3 COD variable is not exactly 8 characters in length COD1.2.6 COD variable has non-missing values that are not left-justified COD1.2.7 Problem with the length of COD values COD1.3.1 CodeType variable is not character type COD1.3.2 CodeType variable is not exactly 2 characters in length COD1.3.3 CodeType variable has missing values COD1.3.4 CodeType variable has values other than "09" or "10" COD1.3.5 CodeType variable has non-missing values that are not left-justified COD1.4.1 CauseType variable is not character type COD1.4.2 CauseType variable is not exactly 1 character in length COD1.4.3 CauseType variable has missing values COD1.4.4 CauseType variable has values other than "C", "I", "O", or "U" COD1.5.1 Source variable is not character type COD1.5.2 Source variable is not exactly 1 character in length Mini-Sentinel Common Data Model Data Quality Review Process and Programs

23 COD1.5.3 COD1.5.4 COD1.6.1 COD1.6.2 COD1.6.3 COD1.6.4 COD2.0.0 COD3.0.1 COD3.1.1 COD3.2.1 COD3.2.2 COD3.4.1 COD3.5.1 COD3.6.1 DIA4.6.1 DIA4.6.2 DIA4.6.3 DIA4.6.4 DIA4.6.5 DIA4.6.6 DIA4.6.7 DIA4.6.8 DIA4.6.9 DIA DIA DIA DIA DIA Source variable has missing values Source variable has values other than "L", "N", "S", or "T" Confidence variable is not character type Confidence variable is not exactly 1 character in length Confidence variable has missing values Confidence variable has values other than "E", "F", or "P" Record(s) have duplicate key value combinations (with respect to table definition) Significant change in number of records between Problem with number of unique PatIDs Problem with distribution of COD variable COD variable is not consistent with CodeType variable Problem with distribution of CauseType Problem with distribution of Source Problem with distribution of Confidence Problem with number of ovarian cancer encounters (per year-month) within Problem with number of ovarian cancer encounters (overall) within Problem with number of ovarian cancer encounters by sex (per year-month) within Problem with number of ovarian cancer encounters by sex (overall) within Significant change in number of ovarian cancer encounters (per year-month) across Significant change in number of ovarian cancer encounters (overall) across Significant change in number of ovarian cancer encounters by sex (per yearmonth) across Significant change in number of ovarian cancer encounters by sex (overall) across Problem with number of prostate cancer encounters (per year-month) within Problem with number of prostate cancer encounters (overall) within Problem with number of prostate cancer encounters by sex (per year-month) within Problem with number of prostate cancer encounters by sex (overall) within Significant change in number of prostate cancer encounters (per yearmonth) across Significant change in number of prostate cancer encounters (overall) across Mini-Sentinel Common Data Model Data Quality Review Process and Programs