Roll-Out and Preliminary Evaluation of a Novel Birth Registration E-learning, New York City, 2014

Transcription

1 Roll-Out and Preliminary Evaluation of a Novel Birth Registration E-learning, New York City, CityMatCH Leadership and MCH Epidemiology Conference September 18, 2014 Erica Lee, MPH Quality Improvement Coordinator Bureau of Vital Statistics New York City Department of Health and Mental Hygiene

2 Outline Birth Registration Data Quality and Public Health Impact Training Development Evaluation Roll-Out Results Challenges and Next Steps

3 Birth Registration All live births in New York City are registered with the local Health Department ~120,000 births annually from 42 facilities Electronic registration system (EBRS) Birth registrars collect and enter information Over 250 items: demographic and medical Data sources: clinicians, parents, and medical records

4

5 Data Quality Demographic items generally high quality Sensitivity of some medical items poor Example: pre-pregnancy DM sensitivity ~50% on BC compared to medical records. 1 Variation in data quality by facility 1 Delvin, H.M., Desai, J., Walaszek, A. (2009). Reviewing Performance of Birth Certificate and Hospital Discharge Data to Identify Births Complicated by Maternal Diabetes. Matern Child health J. 13:

6 Reasons for Data Quality Variations EBRS implementation priorities Variation in facility characteristics, staff training and reporting practices Burdensome process 1 to 2 registrars responsible for 800-7,000 births a year Low motivation Not Health Department staff No additional money for hospital

7 Public Health Impact Cost Amendments and corrections Data quality assurance and cleaning Local & state MCH programs Research National statistics

8 Birth Registration elearning Internet-based training Self administered Geared towards non-clinical staff Interactive, narrated Chapters: importance of birth registration and data, beginning a new record, and prenatal care/pregnancy history Part of larger certification process

9

10 Evaluation: Questions and Design Evaluation Questions: 1. Was the training effective? 2. Was the training functional? 3. Did the training improve data quality? Design = Pre/post training survey and analysis of birth record data quality

11 1a: Training Efficacy Construct Reaction Survey question I feel like my understanding of the purpose of birth registration has increased after taking this module Perceived Knowledge Before/after taking the chapter, how would you rate your level of knowledge about birth registration? Satisfaction I would recommend the e-learning training to new registrars Self-Efficacy Engagement Perceived behaviors Workplace Value I feel like I have the skills and knowledge necessary to complete my job I think that the work that I do is important to the health of women and infants The information I learned in this training will change the way that I complete the birth certificate My coworkers and supervisors value the work that I do

12 1a: Training Efficacy Construct Reaction Survey question I feel like my understanding of the purpose of birth registration has increased after taking this module Perceived Knowledge Before/after taking the chapter, how would you rate your level of knowledge about birth registration? Satisfaction I would recommend the e-learning training to new registrars Self-Efficacy Engagement Perceived behaviors Workplace Value I feel like I have the skills and knowledge necessary to complete my job I think that the work that I do is important to the health of women and infants The information I learned in this training will change the way that I complete the birth certificate My coworkers and supervisors value the work that I do

13 1b: Knowledge Test 20 questions Covers 3 chapters True/False and multiple choice Piloted with various staff for difficulty Example: True/False: If the mother only had one previous live birth, the date of her first live birth and her last live birth is the same.

14 2: Training Functionality Construct Content Accessibility (Literacy level) Usability Time Survey question The elearning training was presented in a clear and thoughtful manner There were words I didn t understand The elearning training was easy to navigate The chapters were too long Resource Availability Technology I was able to find a computer to complete the elearning training chapters The use of interactive elements (graphics, animation, and audio) was effective

15 2: Training Functionality Construct Content Accessibility (Literacy level) Usability Time Survey question The elearning training was presented in a clear and thoughtful manner There were words I didn t understand The elearning training was easy to navigate The chapters were too long Resource Availability Technology I was able to find a computer to complete the elearning training chapters The use of interactive elements (graphics, animation, and audio) was effective

16 3: Data Quality Improvement Composite measure of 7 birth certificate variables (% unknown/conflicting values) Developed 4 specialized EBRS reports Number of duplicate records Errors to the mother s address Changes to the child s name Changes to marital status

17 elearning Roll-Out December 2013 piloted with 2 hospitals (10 registrars) July/August all users with EBRS accounts New training program Requirement Initial letter + conference call Reminder blasts Specific outreach to noncompliant EBRS users anyone who might benefit Certificate of Completion

18 Results and Evaluation

19 Participants 232 completed training, pre and post-surveys 93% from Hospitals, 7.3% midwives Mean: 6.7 years in their role (0 37 years) Roles Collect information for parent or facility worksheets Attending birth Entering or amending certificate information Department heads

20 Knowledge Test (20 questions) 100.0% 80.0% Mean change: +15.8%, p< Min: -10% Max: +45% 69.7% 85.5% 60.0% 40.0% 20.0% 0.0% Pretest Posttest

21 Most Improved Importance of Birth Registration Data True or False: Some information collected on NYC birth certificates is available to members of the public, like expectant mothers, to help them decide where to receive care. TRUE Pre-Test Post-Test 47% 76% Improving timeliness of birth registration If the parents have not yet selected a first name for their child, what should you do? 1-Enter No Name in the Child s First Name field. 2-Tell the parents they need to give their child a first name in order to complete the birth record. 3-Leave the Child s First Name field blank. 4-Enter Baby Girl or Baby Boy in the Child s First Name field. Pre-Test Post-Test 56% 92%

22 Most Improved Sources and accuracy of Birth Registration Data You are completing the mother s date of last normal menses (LMP) on the facility worksheet. Who/what is the best source of this information? 1 A pregnancy wheel 2 The prenatal care record 3 The birth attendant 4 The mother Pre-Test Post-Test 53% 72% You receive a mother s prenatal care record when she is 36 weeks along in her pregnancy. In the record, it says she had 12 prenatal care visits. The mother delivers at 40 weeks, and she tells you she thinks she had 4 additional prenatal care visits between 36 and 40 weeks. How many prenatal care visits should you report for birth registration? 1 99 (Unknown) 2 12 Pre-Test Post-Test % 66% 4 13

23 Attitudes (post training) High self-efficacy and sense of purpose Value Purpose Moderate sense of Self-Efficacy workplace value 70% 80% 90% 100% Post Pre >95% found the training clear, easy to use, and would recommend to others ~40-50% felt training chapters were too long, had difficulties finding time to take the training

24 Challenges Limited health department resources Large number of potential data items Variation in hospital procedures and staff roles (e.g. I interview the mother vs. I attend the birth vs. key entry staff)

25 Conclusions Training was effective and functional. Data quality evaluation TBD. Training uptake was better than expected Feedback was generally very positive

26 Next Steps Analyze data quality before and after training rollout Build additional chapters (delivery questions) Training tailored for clinical staff Feedback to hospitals Training refreshers

27 Acknowledgements Melissa Gambatese, MPH Renata Howland, MPH (former CSTE* fellow) James Buss Tara Das, PhD, MPH Ann Madsen, PhD, MPH Shelley Li *This study was supported in part by an appointment to the Applied Epidemiology Fellowship Program administered by the Council of State and Territorial Epidemiologists (CSTE) and funded by the Centers for Disease Control and Prevention (CDC) Cooperative Agreement Number 5U38HM Thank you! Erica Lee, MPH Quality Improvement Coordinator Office of Vital Statistics

28 Feedback This training should be mandatory for all new birth registrars. Excellent training material to introduce to new staff and encourage active staff to take. A constant reminder of if's and no's. This is the training that I needed when I stepped in to cover the birth registration office.