Texas Newborn Screening Performance Measures Project

Texas Newborn Screening Performance Measures Project Susan Tanksley, PhD MSGRCC Annual Meeting July 14, 2011 The Texas Newborn Screening Performance Measure Project (TNSPMP) is funded through a cooperative agreement with the Centers for Disease Control and Prevention (CDC). Agreement Number 5U47CI000573-03. 1

Texas Newborn Screening Performance Measures Project (TNSPMP) 3 year CDC Cooperative Agreement September 2007 May 2011 Overall project goal Develop and identify evidence-based performance measures to improve patient care for newborns identified with disorders through the newborn screening program Focus on pre- and post-analytical aspects of NBS 2 2

Specimen Prepared ANALYTICAL X Screening Performed Results Verified P R E A N A L Y T I C A L Specimen Received Specimen Transported Specimen Collected Infant Born Parent & Provider Education Results Reported Physician and Parent Notified of Abnormal Result Action Taken Follow up with patient Effect on Patient Care P O S T A N A L Y T I C A L PATIENT CARE

TNSPMP Goals Goal 1 - Formalize a Steering Committee to guide project Goal 2 - Develop and define performance measures that may reveal gaps Goal 3 - Pilot key performance measures for effectiveness Goal 4 - Identify, recommend, and document evidence-based interventions 4

Project Goal 1 Project Goal 1 - Formalize a Steering Committee to guide the project Developed external stakeholder team representative of the NBS System Created DSHS Project Team from Laboratory and Follow-up 14 meetings held with external stakeholder team 5

TNSPMP Team 6

Project Goal 2 Project Goal 2 - Develop and define performance measures that may reveal gaps in the system. 1) System assessment where are we now? Conducted Internal Program Evaluation and Assessment Scheme (PEAS) for Texas Newborn Screening Program 2005 external review of the program by NNSGRC Stakeholder input - Brainstorming sessions at 1 st two meetings Texas NBS Program Gaps and Barriers Summary Report (May 2008) Cross walk of findings between PEAS, NNSGRC Consultative Report, and stakeholder observations 7

Stakeholder Feedback 8

Existing Performance Measures Investigated existing measures in the NBS community Conducted a national survey to identify existing performance measures used by state NBS programs Collected report cards (quality reports) from states Best Evidence Clinical Expertise Evidence-Based Patient Focused 9 9

Project Goal 2 2) Reviewed literature to identify evidence-based performance measures Time to treatment and its affect on patient outcome Potential performance measures suggested by experts Disorders CAH Galactosemia (GALT) MCADD Congenital Hypothyroidism (CH) Difficult to find solid evidence MSUD PKU Sickle Cell Disease Identified approximately 50 performance measures Hosted 9 Focus Groups with healthcare professionals and consumers to assist with defining the measures and developing standards 10

Universal Measure: Time To Treatment Time to initiate treatment Series of steps that can be measured discretely or as a group Help pinpoint where a breakdown has occurred Pre-analytic Analytic Post-analytic 11

Measures with Evidence for Evaluation Sickle Cell Time to Initiate Penicillin Treatment (HbSS) Compliance with Oral Prophylactic Prescription of Penicillin (HbSS) Age of First Prevnar Vaccination (PCV-7) Parent Education on Assessing Enlarged Spleen/Monitoring Episodes of Fever (SCD) Clinical Evaluation at Age 5 for Disease Management Genetic Counseling of Parents CAH Time to Initiate Treatment for SW CAH Time to Treatment for SW & SV CAH: By Gender Time to Gender Assignment for SW CAH Frequency of Medical Evaluations that Assess Growth Timeliness Turnaround Times (All not listed) Specimen Collection Time Specimen Transit Turnaround Days from Abnormal Screen Resulting until Physician Notification Days from Physician Notification until Parent Notification Days from Parent Notification until Physician Specialist Visit for Confirmatory Testing Days from Physician Specialist Visit until Receipt of Confirmatory Testing Results, etc. Percent Missing Birth Weight Percent Missing Date of Birth Percent Missing Date of Collection Percent Missing PCP Information Percent with Incorrect PCP Information Unsatisfactory Specimen Rate PKU Time to Initiate Treatment Frequency of Phenylalanine Monitoring for Metabolic Control Monitoring Average Phenylalanine Levels Dietary Compliance Galactosemia Time to Initiate Treatment Dietary Compliance MSUD Time to Initiate Treatment Time to Reduce Plasma Leucine Concentration Levels Mean Annual Leucine Levels for Long Term Metabolic Control MCADD Time to Confirmed Diagnosis Hospitalization for Severe Episodes related to MCADD Parent Understanding Post Physician Notification Adherence to Dietary Treatment (Avoid Fasting) Screening/Diagnosis of At-Risk Family Members Normal Developmental & Cognitive Outcome by Age 4 CH Time to Initiate Treatment Initial Dosage of L-Thyroxine Normalization of Serum TSH, T4 and FT4 Concentration within One Month of Treatment Evaluation for Transient/Permanent CH by Age 4 12 12

TNSPMP Task at Hand Proposal Process Identification Process Selection Process Brainstormed Evidence-Based Actionable YEAR 1 ACTIVITY YEAR 2 ACTIVITY 13

Impact Assessment What overall impact will the measure have on health outcomes? # Evaluation Criterion Scoring 1 The performance measure is scientifically sound 1-10 2 The performance measure is relevant to the stakeholders in the newborn screening system 1-5 3 The performance measure can reveal health care disparities 1-5 4 The performance measure has great potential for improving health care quality 1-10 5 The performance measure has significant health importance 1-5 6 The performance measure can be used to improve the newborn screening system 1-5 14

Feasibility Assessment Can the measure be implemented/maintained with existing staff and funding and can the data be collected for a 6 month pilot? Data elements Are the data elements identified and clear? Data collection methods Are there collection methods in place for all of the data elements? Ease of data collection Will it be easy or difficult to collect and report the data? The ease of querying the data from the database The response rate for data requested via case management forms The need to add a database field to capture data The need for a collection method Data entry needs 15

Selection of Performance Measures Impact What difference will the measure have on patient outcomes? Online survey completed by full team Feasibility What are the logistics required to implement the measure during a 6 month pilot? Online survey completed by DSHS staff Affinity Exercise August 2009 Stakeholders voted on each performance measure. Low Feasibility Quadrant II Qualified candidate to pilot in year three but may require infrastructure needs Quadrant III Weak candidate Will not be considered to pilot High Impact Quadrant I Strong candidate to pilot Quadrant IV Weak candidate Will not be considered to pilot Low Impact High Feasibility 16

Sample Performance Measures CAH Time to Correct Gender Assignment Time to Treatment for SW CAH Time to Treatment for SW & SV CAH; Female Vs Male Frequency of Developmental Evaluations thru Age 18 CH Time to Treatment Appropriate Dosage of L-Thyroxine Number of Newborns with Normal Serum TSH, T4 and FT4 Concentration within One Month of Treatment Number of Newborns Receiving Assessment for Co-morbidities Number of Patients Receiving Evaluation for Transient/Permanent CH at Age 3 17

Next Steps Submitting performance measures for national consideration MCAD Time to Diagnosis Hospitalization Rate Parent Understanding Post Physician Notification Adherence to Dietary Treatment (Avoid Fasting) Screening/Diagnosis of Family Members Frequency of Developmental Evaluation thru Age 4 Sickle Cell Diseases Time to Treatment of SCA or SCD Compliance of Prophylactic Prescription of Penicillin thru Age 5 Age of First Prevnar Vaccination (PCV-7) Number of Parents Educated on Assessing Enlarged Spleen/Monitoring Episodes of Fever Number of Patients Receiving Detailed Assessment & Parent Consultation at Age 5 Years Genetic Counseling for Parents 18

Selected Measures to Pilot Universal: Specimen Quality Percent NBS specimens missing date/time of collection, date/time of birth, birth weight, physician information Unsatisfactory NBS Specimen Rate Universal: Timeliness Time from birth to NBS specimen collection for the initial screen NBS specimen transit time from collection to receipt in the laboratory Time from laboratory out-of-range NBS result to notification of follow-up staff Time from out-of-range NBS result until physician notification Time from birth until physician notification of out-of-range NBS Sickle Cell Time to Penicillin treatment for infants with sickle cell anemia Time of pneumococal vaccination for infants with sickling hemoglobinopathies PKU Time to initiate treatment for infants with PKU Monitoring phenylalanine levels in infants with PKU for metabolic control Galactosemia Time to initiate treatment (Soy Based Diet) for infants with galactosemia MSUD Time to Initiate Treatment Time to Reduce Plasma Leucine Concentration Levels MCADD Time to treatment for infants with MCADD Congenital Adrenal Hyperplasia Time to initiate treatment for SW & SV CAH: By Gender Congenital Hypothyroidism Time to Initiate Treatment for primary CH Initial Dosage of thyroid replacement therapy for newborns with primary congenital hypothyroidism 19 19

Definitions. Time to Initiate Treatment for Primary Congenital Hypothyroidism Significance of Measure If untreated, congenital hypothyroidism (CH) can lead to mental retardation and poor physical growth. When thyroid hormone replacement treatment begins in the neonatal period, normal growth and development can be expected. Conceptual and Operational Definitions Conceptual Measure the time it takes from birth to initial thyroid hormone replacement treatment for an infant with primary CH. Operational Numerator: For a specified reporting period and physician or pediatric subspecialist, the number of infants with primary CH receiving thyroid hormone replacement treatment where the date of birth minus the date of treatment is 14 days or less. Denominator: The number of infants with primary CH documented by the Texas NBS Program within the specified reporting period per physician or pediatric subspecialist. Performance Standard Disorder specific treatment initiated within 14 days of age. Performance Goal 100% compliance with the performance standard. 20

Project Goal 3 Project Goal 3 - Pilot key performance measures for effectiveness in improving time to treatment for infants with newborn screening disorders. Each of the final performance measures has been defined, evaluated and data summary reports developed. Pre- and post-analytical report cards have been developed. Refined pre-analytical NBS report card format Provide a gold standard and state average for comparison Produced through automated process/to be available online Retrospective data analysis has been completed for July to December 2009. July to December 2010 data being collected and analyzed for comparison to retrospective data. New report card process in final phases of validation. 21

Percentage Specimen Transit Time from Collection to Receipt in the Lab 100 90 80 70 44% 43% 40% 44% 40% 48% 48% 60 50 40 30 20 10 56% 57% 60% 56% 60% 52% 52% 0 TOTAL (N=406,629) July (n=71,991) August (n=68,536) September (n=70,197) October (n=68,364) November (n=60,724) December (n=66,817) NBS Specimens Received by the State Public Health Laboratory Data: July December 2009 Outside the performance standard Received within 3 days of collection 22

Percentage Percent NBS Specimen Missing DOB 100.00 0.16 0.18 0.20 0.12 0.19 0.16 0.12 99.00 98.00 99.84 99.82 99.80 99.88 99.81 99.84 99.88 97.00 96.00 95.00 TOTAL (N=406,629) July (n=71,991) August (n=68,536) September (n=70,197) October (n=68,364) November (n=60,724) December (n=66,817) NBS Specimens Received by the State Public Health Laboratory Data: July December 2009 DOB available DOB missing 23

Percentage Percent NBS Specimen Missing TOB 100 22 21 22 22 23 22 22 80 60 40 78 79 78 78 77 78 78 20 0 TOTAL (N=406,629) July (n=71,991) August (n=68,536) September (n=70,197) October (n=68,364) November (n=60,724) December (n=66,817) NBS Specimens Received by the State Public Health Laboratory Data: July December 2009 TOB available TOB missing/invalid 24

Percentage Percent NBS Specimen Missing Birth Weight 100.00 5.03 4.63 5.10 5.05 5.22 5.24 4.99 90.00 80.00 94.97 95.37 94.90 94.95 94.78 94.76 95.01 70.00 60.00 50.00 TOTAL (N=406,629) July (n=71,991) August (n=68,536) September (n=70,197) October (n=68,364) November (n=60,724) December (n=66,817) NBS Specimens Received by the State Public Health Laboratory Data: July December 2009 Birth weight available Birth weight missing 25

Report Card: Original Format 26

Report Card: New Format TX NBS Report Card New Format 27

No. Cases Time to Initiate Treatment for Salt Wasting Congenital Adrenal Hyperplasia: by Gender Disorder - specific treatment initiated within 7 days of age. Minimum 9 days to treatment for these 9 cases. 4 CAH Salt Waster - Time to Treatment Females Males Unknown 3 2 2 1 1 2 1 1 1 1 0 1 to 10 Days 11 to 20 Days 21 to 30 Days 31 to 40 Days 41 to 50 Days Days Data: July December 2009

No. Cases Time to Initiate Treatment for Infants with Phenylketonuria Disorder - specific treatment initiated within 30 days of age 6 5 4 PKU - Time to Treatment 5 All 10 cases for this time period were treated within 30 days of age. 3 3 2 2 1 0 1 to 10 Days 11 to 20 Days 21 to 30 Days Days

No. Cases Time to Penicillin Treatment for Infants with Sickle Cell Anemia - Twice-daily prophylactic penicillin therapy initiated by 2 months of age 16 14 24 contained no treatment initiation 12 date. 8 Hemoglobin - Time to Treatment SS Sickle Cell Anemia SC Sickle C Disease S Beta Zero Thalassemia 94 cases for this time period. 28 cases were treated within 2 10 months of age 1 1 5 2 5 6 4 2 0 1 1 to 10 Days 1 11 to 20 Days 1 2 3 3 21 to 30 Days 31 to 40 Days Days 9 41 to 50 Days 2 4 51 to 60 Days 6 61 to 70 Days 9 71 to 80 Days 1 4 81 to 100 Days 7 < 101 Days

Report Cards for Specialists 31

Project Goal 4 Identify, recommend, and document evidence-based interventions Project Goal 4 Brainstorming sessions held with stakeholders to gather intervention ideas and score feasibility of those ideas. Explored evidence-based interventions through an assessment of available literature and linked to the gaps & barriers when possible. Distributed survey to other US NBS Programs asking for ideas on intervention strategies. Interventions to be included in updated gaps/barriers document. Publications regarding outcomes of project being drafted. 32

Target Intervention Areas Education Long-Term Follow-Up Provider Evaluation Information Systems Courier Programs Collaboration w/ Peers 33

PLAN DEVELOP REPLACE Life Cycle of a Performance Measure TEST MODIFY EVALUATE IMPLEMENT 34 34

Accomplishments Texas NBS Program Gaps and Barriers Summary Report (May 2008) Summary Evidence Report (May 2009) Performance Measures Selection Process and Development of Pilot Plans (January 2010) Awarded 8 month no cost extension (September 2010-May 2011) Key Deliverables Final report due to CDC August 28, 2011 35

Lessons Learned Get to know the NBS system in your state Work to develop and nurture strong relationships with system stakeholders Program evaluation is essential for improvement Listen to stakeholders Ask why? Maintain continuing relationship with system stakeholders 36

Lessons Learned Strong evidence is hard to find in NBS and few evidencebased measures exist Pay for performance & other evidence-based initiatives will require assessments via evidence-based measures Investing in ongoing data collection & analysis will help build stronger evidence-base Obtaining buy-in and consensus when defining terms in NBS processes and measures is essential Assessing data system access and utility is critical and should be started earlier in the process 37

The performance measures: Lessons Learned Validated what does not work in the present system and why Identified challenges in the present system Set goals for moving forward and aligning Texas with other NBS programs Performance measures must be carefully defined, understood and consistently reported. The reporting should be streamlined Definitions need to be explained to providers Performance measures need to be shared with providers. 38

Extension of Project Concept Applicable to other areas of public health Association of Public Health Laboratories (APHL) grant awarded November 2010 Innovations in Quality Public Health Laboratory Practice Project team will use strategies and lessons learned from TNSPMP to improve the Blood Lead Screening Program Focus is on pre-analytical performance measures and improving the report card process 39

Thank you! 40

Susan Tanksley susan.tanksley@dshs.state.tx.us 512.776.3106 41