Validating EHR Reports

Transcription

1 Validating EHR Reports October 2012 Cardiac Learning & Action Network: Improving Cardiac Population Health through Primary Care Redesign Introduction The use of electronic health records (EHRs) has greatly improved the ability of medical practices to access clinical quality data through registries and clinical reports. Although these functionalities can seem straightforward in some EHR systems, a number of steps are necessary in order to make sure that practice staff can trust the data in the reports. Various team members have important roles in the reporting process: A clinical person should confirm that all staff follow a protocol for putting clinical information related to the report into the correct fields of the EHR. A quality improvement person should confirm that the data included in the numerator and denominator reflect the information generated in the report. A database/software person should confirm that the report is pulling data from the correct fields in the EHR and that there are no potential software errors. After these steps have been taken, the validation process assures a practice that the report is accurate (or at least contains minimal error). There are many reasons that reports can generate incomplete or inaccurate information; report validation identifies errors so that the practice can correct any mistakes. Types of Reporting Errors There are two primary types of errors to look for when validating your report: 1. Inclusion errors: Patients are in your report but should not be Patients may be erroneously included in your report for a number of reasons, including outdated problem lists or providers using ICD codes that are not part of the practice protocol. Please see the Finding Inclusion Errors worksheet, page 4, for assistance in documenting your efforts to resolve these types of errors. 2. Exclusion errors: Patients are not in your report but should be Patients may be erroneously excluded from your report due to reasons such as rounding of vital signs that inappropriately exclude patients from the numerator or denominator, or patients being prescribed medications that are not pulled in the report. Please see the Finding Exclusion Errors worksheet, page 5, for an outline of the steps required to uncover these types of errors. 1

2 Methods for Finding Errors Chart reviews and complementary reports, outlined below, are two useful methods for finding potential inclusion and exclusion errors. The worksheets provided on pages 4 and 5 can help your team systematically review each round of reports. It may take several iterations of the report before everyone is satisfied that the data are accurate and complete. Staff may want to include a review of clinic workflows to prevent future data-entry errors. Chart Review Although chart review is time-consuming, it is worth the effort to ensure that the information in the report is trustworthy. To make the best use of your time, review a sample of records rather than the entire data set included in the report. The first step in a chart review is to determine how many samples you must take (i.e., how many charts you must review) to get an accurate representation of your report s entire data set. One shorthand method is to select the inverse of the desired error rate. For example, to find at least one instance of an error that occurs in 5% of the charts, you need to review 20 1 charts that have been randomly selected. Although there is no magic number of charts to review, most experts find reviewing 30 charts to be a good target that will find errors that occur in 3 4% percent of records in a report and at the same time does not represent an impossible task. There are far more scientific ways to determine sample size; however, this basic method will provide a place to start. For further information on more robust techniques, please contact your Cardiac LAN consultant. 1 We arrive upon the sample size of 20 charts by selecting the inverse of the desired error rate (which in the example is 5%, so 1/5% = 20). If you want to find an error that occurs in 3% of the charts, you will review 1/3%, or 33 charts. If you want to find an error that occurs in 1% of the charts, you need to review 1/1% or 100 charts. 2

3 Complementary Reports An additional way to validate that a report is accurate is to run a complementary report that looks for the data in an alternative way. Validating a chronic disease diagnosis: 1. Reporting from claims data for the past two years If the patient has a chronic disease diagnosis coded for two or more encounters, and this diagnosis does not appear on the EHR problem list, have the PCP review the chart for accuracy. 2. Reporting on common procedures Patients with two or more A1c tests without a diagnosis of diabetes Patients with elevated LDLs without a diagnosis of hyperlipidemia 3. Reporting from the medication list Patient on cholesterol-lowering medications without a diagnosis of hyperlipidemia Patients on antipsychotics without a mental health diagnosis Patients on bronchodilator inhalers and steroid inhalers without a diagnosis of asthma Validating vaccination reports: 1. Write a report from claims data for vaccination CPT codes and compare to the data in the initial EHR report. 2. Compare results in your vaccination reports to purchasing orders to see if they are in close proximity. Conclusion Generating reports from your EHR is an important part of quality improvement. Validating those reports, using the suggestions outlined here, will help ensure that your quality improvement strategies are based on accurate information. This document is based, in part, on the whitepaper authored by Jeff Hummel, MD MPH for Deep Domain entitled Get More from your EHR: An expert guide to EHR Reporting and is used here with permission. No part of this document may be reused without the prior written permission of Deep Domain, Inc. ( The content was adapted by Qualis Health, the Medicare Quality Improvement Organization for Idaho and Washington, under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy. ID/WA-C9-QH-759R

4 Finding Inclusion Errors Chart Review The purpose of these steps is to find patients who are in your report but should not be. Report Name Run Date From the records included in your report, systematically select patient charts until reaching the agreed upon sample size. Fully review each chart before moving to the next one. Reviewer(s) Patient ID Does patient belong in the report s denominator? Does patient belong in the report s numerator? Compare the data in the report to relevant data in the chart. For example, confirm that the patient was diagnosed with IVD and had the LDL result indicated in the report. List all discrepancies, then discuss with the clinical team and the database analyst/report writer. 4

5 Finding Exclusion Errors The purpose of these steps is to find patients who are not in your report but should be. Step 1: Chart Review Report Name Run Date Reviewer(s) Steps Notes 1. Locate all of the patients on the report that belong to the panel of (or are attributed to) the provider who is participating in the validation process. 2. Open the validating provider s clinic schedule in the most recent months for which data should be in the report. Working backwards one day at a time, select the patients who would be expected to be included in the report based on the provider s knowledge of their medical condition. 3. Verify that each patient found on the schedule who should be in the report actually is in the report, and that the data in the report match the data in the electronic record. 4. For those patients who are missing from the report (but should be included) review the chart to determine if the criteria for inclusion in the a) denominator, and b) numerator are met in the chart. 5. List all discrepancies, then discuss with the clinical team and database analyst/report writer. Step 2: Complementary Reports Complete the following steps for each complementary report. Steps 1. Write a new report that seeks the same information in a different way. 2. Validate the report with a chart review. 3. Identify any patients who are included in the complementary report but are missing from the initial report. Determine whether they actually belong in the initial report. 4. List all discrepancies, then discuss with the clinical team and database analyst/report writer. Notes 5

6 Cardiac Learning & Action Network: Improving Cardiac Population Health through Primary Care Redesign Suggested Workflow for Ensuring High Risk Patients Receive Annual Lipid Panels Best practice indicates that some chronic care patients such as those with diabetes, hyperlipidemia, and ischemic vascular disease need at least one lipid panel ordered each year. This workflow is designed to help organizations identify and reach out to these high risk patients. Members of the Cardiac LAN report that the most common errors are not having a system to track open orders, and entering the information as free text or scanned data. IT and QI Departments Generate a report for each provider: Numerator = Of patients in the denominator, those who have not received a lipid panel in the past 12 months Denominator = Active patients, in the provider s panel, with [diagnosis], aged 18 years and older Validate the report (see and correct errors. Provider Care Team Reserve time to review the report. Pt received lipid panel in last 12 mos? No Yes Is lipid panel appropriate? No Yes Documented correctly? Yes No Correct documentation. Contact patient to schedule lab. Document. Document. This material was prepared by Qualis Health, the Medicare Quality Improvement Organization for Idaho and Washington, under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy. ID/WA-C9-QH Page 1 of 1

7 The Problem List: Overview Cardiac Learning & Action Network: Improving Cardiac Population Health through Primary Care Redesign DEFINITION AND PURPOSE The problem list is often thought of as a simple record of current medical conditions or symptoms for a particular patient, but a well maintained problem list serves many other important roles: 1. Functions as a table of contents and an index to a patient s medical record 2. Facilitates coordinated management across medical specialties 3. Serves as the inclusion criteria for population management and case management work. This, in turn, supports a wide array of decision support tools ranging from alerts to registry functionality that are used in chronic disease population management. 4. Allows care teams to quickly identify chronic illness problems in patients who present for unrelated acute issues, and address gaps in the care for those chronic problems during the visit Problem lists are key drivers for much quality improvement work, and therefore must be maintained using a standardized, organization-wide approach. GUIDELINES FOR HEALTHCARE ORGANIZATIONS It is important for healthcare organizations to have policy and procedural guidelines for their providers regarding how to create and maintain patient problem lists. Guidelines should include information such as who is expected to maintain problem lists, workflows recommended to develop and maintain problem lists, the role of the patient in creating the problem list, and proposed audit processes to check for accuracy and completeness. Most organizations agree that, at a minimum, the problem list should document: 1. Chronic or recurring medical conditions likely to affect patient care for multiple visits 2. Active problems likely to require ongoing medication or diagnostic monitoring over time 3. Persistent symptoms that affect patient care over time Organizational policies (see an example, page 2) may be needed to guide consistent documentation for additional characteristics, such as: 1. The presence of risk factors (tobacco use, familial cancer risk, etc.) 2. The existence of a patient s controlled substance agreement or advance directive (perhaps using dummy codes as flags) 3. Behavioral health issues, including social circumstances or personality issues that may affect care management (e.g., lack of housing) 4. Conditions that are perceived as important by patients that could influence clinical decision-making, such as dietary preference or preference for non-allopathic medical practices. 5. Episodic conditions (e.g., pregnancy) The Problem List page 1 of 2

8 The Problem List: Sample Policy and Procedures Use the following example, including the discussion points shown in gray, to help guide your development of an organization-wide policy. POLICY STATEMENT Active patients must have a problem list as part of their medical record. The problem list contains a list and date of onset of the patient s diagnoses, unexplained physiological abnormalities, and symptoms requiring medical management. The problem list should be accurate and actively maintained. PURPOSE The purpose of this policy is to provide guidance on problem list content as well as maintenance procedures including who, how, and when the problem list should be updated. Standardized use of the problem list across the organization is a foundational step for better population management and quality improvement initiatives. DEFINITIONS Active patient: A patient who has had a visit to [organization name] within [specified time frame] Problem list: At a minimum, the problem list should include: Chronic or recurring medical conditions that are likely to affect patient care for multiple visits Active problems that are likely to require ongoing medication or diagnostic monitoring over time Persistent symptoms that affect patient care over time Other elements required by your organization PROBLEM LIST PROCEDURES Authority to update the problem list: Describe under what circumstances non-provider staff may add or resolve problems Describe when providers must sign-off or approve changes to the problem list Updating the problem list: Describe the frequency with which the problem list must be updated (every visit, every physical, every diagnosis) Describe how staff will know when the problem list was last updated Content of the problem list: Describe which types of problems/diagnoses/symptoms should be included in the problem list Consider special situations like smoking status, homeless status, or pregnancy Consider any special codes your organization might be using to track certain patient populations Patient role in problem list maintenance Consider adding a policy on how and when to review problem lists with patients Consider adding a policy on how patients can make changes to problem lists REVIEW / REVISION DATES State how frequently the problem list policy should be reviewed This material was prepared by Qualis Health, the Medicare Quality Improvement Organization for Idaho and Washington, under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy. ID/WA-C9-QH The Problem List page 2 of 2

9 For Cervical Cancer Screening For NURSING TEAM 1 Pre check scheduled patients female age for documentation for 1) hysterectomy then 2) Pap smear in Allscripts and Noteworthy or Next Gen 2 If there is data for either and it is within guidelines STOP Guidelines: a. cervical cancer screening starts at 21 years of age b. From age screening is recommended every 3 years c. From age option to screen every 5 years *This may vary based on history and results* If there is no data, when rooming patient: 1. Ask patient 1) Have you had a hysterectomy? Was it an abdominal or Vaginal Hysterectomy? a. If yes hysterectomy: 1) Why did you have a hysterectomy? 2) When? (can be approximate month/year) 3) Do you still have your ovaries? 4)Do you still have your cervix? If yes: When was your last pap? To Document: 1 Go to History on the left 2 Click Past Surgical (Blue HX) 3 Click + for Genitourinary Female 4 Double click Hysterectomy 5 Box opens go to Date and enter date (can be approximate) 6 In comment box write in either vaginal or abdominal and if it was partial or complete. 7 Click OK b. If no hysterectomy 1)Ask patient: When was your last Pap smear? 2. Check HPV Immunization status for all patients ages a. If yes then make sure it is documented in chart and series is completed b. If no then provide the patient with the VIS and offer immunization If pap screening within guideline but no records in chart 1. Ask the patient where it was done? 2. Have patient fill out the record request for the site and send request to medical recordsthey will follow up and document the information into Health Maintenance so it will be there in the future when you see this patient again. If outside guidelines Schedule pap

10 For Cervical Cancer Screening For MEDICAL RECORDS 1. When receive record request from TEAM send to appropriate specialist. When receive returned documentation, enter information in Allscripts. To Document: 1 Go to History on the left 2 Click Health Maintenance (Blue HX) 3 Click + for Adult Health Maintenance 4 Double click Pap Smear 5 Box opens go to Date and enter date pap smear was performed 6 If Normal double click Normal on left and it will go to the right click OK 7 When scanning document that says normal no need to have provider review. 8 If Abnormal double click Abnormal and it will go to the right 9 In comment box type in comment from the report then click OK 10 When scanning document that says abnormal have provider review 2. When you receive pap results If Normal result 1. document under Health Maintenance 2. Scanned Document goes to Provider for approval If Result says anything abnormal 1. Send scanned document to Provider for approval 2. Provider will send message with instructions to the nurse for patient notification

11 Cervical Cancer Screening Guidelines for Average-Risk Women 1 American Cancer Society (ACS), American Society for Colposcopy and Cervical Pathology (ASCCP), and American Society for Clinical Pathology (ASCP) U.S. Preventive Services Task Force (USPSTF) American College of Obstetricians and Gynecologists (ACOG) When to start screening 5 Age 21. Women aged <21 years should not be screened regardless of the age of sexual initiation or other risk factors. (Strong recommendation) Age 21. (A recommendation) Recommend against screening women aged <21 years. (D recommendation) Age 21 regardless of the age of onset of sexual activity. Women aged <21 years should not be screened regardless of age at sexual initiation and other behavior-related risk factors. (Level A evidence) Statement about annual screening Women of any age should not be screened annually by any screening method. (Strong recommendation) Individuals and clinicians can use the annual Pap test screening visit as an opportunity to discuss other health problems and preventive measures. Individuals, clinicians, and health systems should seek effective ways to facilitate the receipt of recommended preventive services at intervals that are beneficial to the patient. Efforts also should be made to ensure that individuals are able to seek care for additional health concerns as they present. In women aged years, annual cervical cancer screening should not be performed. (Level A evidence) Patients should be counseled that annual well-woman visits are recommended even if cervical cancer screening is not performed at each visit. Screening method and intervals 6 Cytology (conventional or liquid based) years of age Every 3 years. 7 (Strong recommendation) Every 3 years. (A recommendation) Every 3 years. (Level A evidence) years of age Every 3 years. 7 (Strong recommendation) Every 3 years. (A recommendation) Every 3 years. (Level A evidence) HPV co-test (cytology + HPV test administered together) years of age HPV co-testing should not be used for women aged <30 years years of age Every 5 years (Strong recommendation) ; this is the preferred method (Weak recommendation). Recommend against HPV co-testing women aged <30 years. (D recommendation) For women who want to extend their screening interval, HPV cotesting every 5 years is an option. (A recommendation) HPV co-testing 8 should not be performed in women aged < 30 years. (Level A evidence ) Every 5 years; this is the preferred method. (Level A evidence) Primary HPV testing 9 For women aged years, screening by HPV testing alone is not recommended in most clinical settings. (Weak recommendation) 10 Recommend against screening for cervical cancer with HPV testing (alone or in combination with cytology) in women aged <30 years. (D recommendation) Not addressed. When to stop screening Aged >65 years with adequate screening history. 11,12 Aged >65 years with adequate screening history. (D recommendation) 11 Aged >65 years with adequate screening history 11, 13 (Level A evidence) Screening post-hysterectomy Women who have had a total hysterectomy (removal of the uterus and cervix) should stop screening. 14 Women who have had a supra-cervical hysterectomy (cervix intact) should continue screening according to guidelines. (Strong recommendation) Recommend against screening in women who have had a hysterectomy (removal of the cervix). 13 (D recommendation) Women who have had a hysterectomy (removal of the cervix) should stop screening and not restart for any reason. 13 (Level A evidence) 15 The need for a bimanual pelvic exam Not addressed in 2012 guidelines but was addressed in 2002 ACS guidelines. 16 Addressed in USPSTF ovarian cancer screening recommendations (draft). 17 Addressed in 2012 well-woman visit recommendations. 18 Aged <21 years, no evidence supports the routine internal examination of the healthy, asymptomatic patient. An external-only genital examination is acceptable. Aged 21 years, no evidence supports or refutes the annual pelvic examination or speculum and bimanual examination. The decision whether or not to perform a complete pelvic examination should be a shared decision after a discussion between the patient and her health care provider. Annual examination of the external genitalia should continue. 19 Screening among those immunized against HPV 16/18 Women at any age with a history of HPV vaccination should be screened according to the age specific recommendations for the general population. The possibility that vaccination might reduce the need for screening with cytology alone or in combination with HPV testing is not established. Given these uncertainties, women who have been vaccinated should continue to be screened. Women who have received the HPV vaccine should be screened according to the same guidelines as women who have not been vaccinated. (Level C evidence)

12 HPV = human papillomavirus; CIN = cervical intraepithelial neoplasia 1 These recommendations do not apply to women who have received a diagnosis of a high-grade precancerous cervical lesion (CIN 2 or 3) or cervical cancer, women with in utero exposure to diethylstilbestrol, or women who are immunocompromised, or are HIV positive. 2 Saslow D, Solomon D, Lawson HW, et al. American Cancer Society, American Society for Colposcopy and Cervical Pathology, and American Society for Clinical Pathology screening guidelines for the prevention and early detection of cervical cancer. CA Cancer J Clin May-Jun;62(3): doi: /caac Available at 3 USPSTF. Screening for Cervical Cancer Available at These recommendations apply to women who have a cervix, regardless of sexual history. 4 ACOG Practice Bulletin No. 131: Screening for Cervical Cancer. ACOG Committee on Practice Bulletins-Gynecology. Obstet Gynecol Nov;120(5): doi: 5 Since cervical cancer is believed to be caused by sexually transmissible human papillomavirus infections, women who have not had sexual exposures (e.g., virgins) are likely at low risk. Women aged >21 years who have not engaged in sexual intercourse may not need a Pap test depending on circumstances. The decision should be made at the discretion of the women and her physician. Women who have had sex with women are still at risk of cervical cancer % of women aged years in the United States report no vaginal intercourse (Saraiya M, Martinez G, Glaser K, et al Obstet Gynecol Dec;114(6): doi: /AOG.0b013e3181be3db4.). Providers should also be aware of instances of non-consensual sex among their patients. 6 Conventional cytology and liquid-based cytology are equivalent regarding screening guidelines, and no distinction should be made by test when recommending next screening. 7 There is insufficient evidence to support longer intervals in women aged years, even with a screening history of consecutive negative cytology tests. 8 All ACOG references to HPV testing are for high risk HPV testing only. Tests for low risk HPV should not be performed. 9 Primary HPV testing (HPV testing alone) is defined as conducting the HPV test as the first screening test. It may be followed by other tests (like a Pap) for triage. 10 No further explanation of which clinical settings HPV testing should be used to screen women aged years as a stand alone test. 11 Current guidelines define adequate screening as three consecutive negative cytology results or two consecutive negative co-tests within 10 years before cessation of screening, with the most recent test performed within 5 years, and are the same for ACS, ACOG, and USPSTF. 12 Women aged >65 years with a history of CIN2, CIN3, or AIS should continue screening for at least 20 years after spontaneous regression or appropriate management. (Weak recommendation) 13 And no history of CIN 2 or higher. 14 Unless the hysterectomy was done as a treatment for cervical pre-cancer or cancer. 15 Women should continue to be screened if they have had a total hysterectomy and have a history of CIN 2 or higher in the past 20 years or cervical cancer ever. Continued screening for 20 years is recommended in women who still have a cervix and a history of CIN 2 or higher. Therefore, screening with cytology alone every 3 years for 20 years after the initial post-treatment surveillance for women with a hysterectomy is reasonable. (Level B evidence) guidelines statement: The ACS and others should educate women, particularly teens and young women, that a pelvic exam does not equate to a cytology test and that women who may not need a cytology test still need regular health care visits including gynecologic care. Women should discuss the need for pelvic exams with their providers. Saslow D, Runowicz CD, Solomon D, et al. American Cancer Society Guideline for the Early Detection of Cervical Neoplasia and Cancer. CA Cancer J Clin 2002; 52: The bimanual pelvic examination is usually conducted annually in part to screen for ovarian cancer, although its effectiveness and harms are not well known and were not a focus of this review. No randomized trial has assessed the role of the bimanual pelvic examination for cancer screening. In the PLCO Trial, bimanual examination was discontinued as a screening strategy in the intervention arm because no cases of ovarian cancer were detected solely by this method and a high proportion of women underwent bimanual examination with ovarian palpation in the usual care arm. 18 ACOG Committee Opinion No. 534: Well-Woman Visit. Committee on Gynecologic Practice. Obstet Gynecol Aug;120(2)1: doi: /AOG.0b013e For women aged 21 years, annual pelvic examination is a routine part of preventive care even if they do not need cervical cytology screening, but also lacks data to support a specific time frame or frequency of such examinations. The decision to receive an internal examination can be left to the patient if she is asymptomatic and has undergone a total hysterectomy and bilateral salpingo oophorectomy for benign indications, and is of average-risk.

13 Diagnostic Test Tracking Audit Tool Diagnostic Test Tracking Policy/Process: Medical record number: Date: Type of test Date of Test Physician: Physician name: Diagnostic report filed in patient EMR under Lab Physician signed off Lab in EMR Physician notified patient of abnormal results when an explanation of medical condition was necessary Treatment plan noted in the dictation Nurse: Nurse appropriately notified patient of normal results or abnormal results that do not require an explanation of medical condition Pap result filed under Diagnostic HX in patient EMR Recall placed in Centricity Follow Up: Patient notified within 14 days of service Method of notification documented Patient followed through with recommended treatment plan (documented in EMR) Certified letter sent to patient that could not be reached with abnormal results Test tracking log checked on a daily basis (OB Coordinator) Comments: Original Doc 7/17/09 Revised

14 Resources for Improving Cervical Cancer Screening Outcomes Source Description Link General Sites CDC Cervical cancer screening overview vical/basic_info/screening.htm NIH Overview of cervical cancer neplus/cervicalcancer.html HHS Cervical cancer awareness month resources O/JanuaryToolkit.aspx PCMH Programs NCQA PCMH Standards 0/patient centered medicalhome case study featuringmary howard health center/ Accreditation Association for Ambulatory Health Care Safety Net Medical Home Initiative Joint Commission USPTF NIH NCI CDC CDC PCMH Accreditation Safety Net Medical Home Initiative Implementation Guides Overview of BPHC Joint Commission accreditation process Screening Guidelines Public Health Task Force cervical cancer screening guidelines updated Data and Statistics Overview of past, present, and future advances in cervical cancer Overview of cervical cancer screening rates in US women aged from Cervical cancer screening trends in US women with a hysterectomy >age 65 from editation/fqchcs/ ome.org/practicetransformation/implementatio n guides rg/accreditation/bureau_of_pri mary_health_care.aspx estaskforce.org/uspstf/uspscer v.htm opics/factsheet/canceradvances in focus/cervical eview/mmwrhtml/mm6151a2. htm eview/mmwrhtml/mm6151a3. htm