Validating EHR Reports

Transcription

1 Validating EHR Reports October 2012 Cardiac Learning & Action Network: Improving Cardiac Population Health through Primary Care Redesign Introduction The use of electronic health records (EHRs) has greatly improved the ability of medical practices to access clinical quality data through registries and clinical reports. Although these functionalities can seem straightforward in some EHR systems, a number of steps are necessary in order to make sure that practice staff can trust the data in the reports. Various team members have important roles in the reporting process: A clinical person should confirm that all staff follow a protocol for putting clinical information related to the report into the correct fields of the EHR. A quality improvement person should confirm that the data included in the numerator and denominator reflect the information generated in the report. A database/software person should confirm that the report is pulling data from the correct fields in the EHR and that there are no potential software errors. After these steps have been taken, the validation process assures a practice that the report is accurate (or at least contains minimal error). There are many reasons that reports can generate incomplete or inaccurate information; report validation identifies errors so that the practice can correct any mistakes. Types of Reporting Errors There are two primary types of errors to look for when validating your report: 1. Inclusion errors: Patients are in your report but should not be Patients may be erroneously included in your report for a number of reasons, including outdated problem lists or providers using ICD codes that are not part of the practice protocol. Please see the Finding Inclusion Errors worksheet, page 4, for assistance in documenting your efforts to resolve these types of errors. 2. Exclusion errors: Patients are not in your report but should be Patients may be erroneously excluded from your report due to reasons such as rounding of vital signs that inappropriately exclude patients from the numerator or denominator, or patients being prescribed medications that are not pulled in the report. Please see the Finding Exclusion Errors worksheet, page 5, for an outline of the steps required to uncover these types of errors. 1

2 Methods for Finding Errors Chart reviews and complementary reports, outlined below, are two useful methods for finding potential inclusion and exclusion errors. The worksheets provided on pages 4 and 5 can help your team systematically review each round of reports. It may take several iterations of the report before everyone is satisfied that the data are accurate and complete. Staff may want to include a review of clinic workflows to prevent future data-entry errors. Chart Review Although chart review is time-consuming, it is worth the effort to ensure that the information in the report is trustworthy. To make the best use of your time, review a sample of records rather than the entire data set included in the report. The first step in a chart review is to determine how many samples you must take (i.e., how many charts you must review) to get an accurate representation of your report s entire data set. One shorthand method is to select the inverse of the desired error rate. For example, to find at least one instance of an error that occurs in 5% of the charts, you need to review 20 1 charts that have been randomly selected. Although there is no magic number of charts to review, most experts find reviewing 30 charts to be a good target that will find errors that occur in 3 4% percent of records in a report and at the same time does not represent an impossible task. There are far more scientific ways to determine sample size; however, this basic method will provide a place to start. For further information on more robust techniques, please contact your Cardiac LAN consultant. 1 We arrive upon the sample size of 20 charts by selecting the inverse of the desired error rate (which in the example is 5%, so 1/5% = 20). If you want to find an error that occurs in 3% of the charts, you will review 1/3%, or 33 charts. If you want to find an error that occurs in 1% of the charts, you need to review 1/1% or 100 charts. 2

3 Complementary Reports An additional way to validate that a report is accurate is to run a complementary report that looks for the data in an alternative way. Validating a chronic disease diagnosis: 1. Reporting from claims data for the past two years If the patient has a chronic disease diagnosis coded for two or more encounters, and this diagnosis does not appear on the EHR problem list, have the PCP review the chart for accuracy. 2. Reporting on common procedures Patients with two or more A1c tests without a diagnosis of diabetes Patients with elevated LDLs without a diagnosis of hyperlipidemia 3. Reporting from the medication list Patient on cholesterol-lowering medications without a diagnosis of hyperlipidemia Patients on antipsychotics without a mental health diagnosis Patients on bronchodilator inhalers and steroid inhalers without a diagnosis of asthma Validating vaccination reports: 1. Write a report from claims data for vaccination CPT codes and compare to the data in the initial EHR report. 2. Compare results in your vaccination reports to purchasing orders to see if they are in close proximity. Conclusion Generating reports from your EHR is an important part of quality improvement. Validating those reports, using the suggestions outlined here, will help ensure that your quality improvement strategies are based on accurate information. This document is based, in part, on the whitepaper authored by Jeff Hummel, MD MPH for Deep Domain entitled Get More from your EHR: An expert guide to EHR Reporting and is used here with permission. No part of this document may be reused without the prior written permission of Deep Domain, Inc. ( The content was adapted by Qualis Health, the Medicare Quality Improvement Organization for Idaho and Washington, under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy. ID/WA-C9-QH-759R

4 Finding Inclusion Errors Chart Review The purpose of these steps is to find patients who are in your report but should not be. Report Name Run Date From the records included in your report, systematically select patient charts until reaching the agreed upon sample size. Fully review each chart before moving to the next one. Reviewer(s) Patient ID Does patient belong in the report s denominator? Does patient belong in the report s numerator? Compare the data in the report to relevant data in the chart. For example, confirm that the patient was diagnosed with IVD and had the LDL result indicated in the report. List all discrepancies, then discuss with the clinical team and the database 4

5 Finding Exclusion Errors The purpose of these steps is to find patients who are not in your report but should be. Step 1: Chart Review Report Name Run Date Reviewer(s) Steps Notes 1. Locate all of the patients on the report that belong to the panel of (or are attributed to) the provider who is participating in the validation process. 2. Open the validating provider s clinic schedule in the most recent months for which data should be in the report. Working backwards one day at a time, select the patients who would be expected to be included in the report based on the provider s knowledge of their medical condition. 3. Verify that each patient found on the schedule who should be in the report actually is in the report, and that the data in the report match the data in the electronic record. 4. For those patients who are missing from the report (but should be included) review the chart to determine if the criteria for inclusion in the a) denominator, and b) numerator are met in the chart. 5. List all discrepancies, then discuss with the clinical team and database Step 2: Complementary Reports Complete the following steps for each complementary report. Steps 1. Write a new report that seeks the same information in a different way. 2. Validate the report with a chart review. 3. Identify any patients who are included in the complementary report but are missing from the initial report. Determine whether they actually belong in the initial report. 4. List all discrepancies, then discuss with the clinical team and database Notes 5