How to Develop a Data Validation Agency (EDV)

Transcription

1 Preparing to Conduct an Encounter Data Validation Sara Hallvik, MPH Ricci Rimpau, RN, CHC, CPHQ Jody Carson, RN, MSW, CPHQ June 5, 2013

2 Preparation Steps* 1. Electronic data checks 2. Frequency of the EDV 3. Drawing a random sample 4. Communication with network providers 5. Staff training 6. Developing an EDV tool *Document every step! 2

3 1. Electronic Data Checks Set the foundation for EDV Are data complete and within an expected range? Missing data Out-of-range data (e.g. CPT codes not in SERI, unknown ethnicity code, etc.) Run report to show percentage of electronic data errors according to data element Document process and include in report 3

4 Example: Electronic Data Check Table Table 6. Results of 2013 Electronic Data Checks. Outpatient Encounter Data Field State Standard % Complete a RSN ID 100% complete (non-missing values), with values known to DBHR Consumer ID 100% complete (non-missing values), with values known to DBHR Agency ID 100% complete (non-missing values) Primary diagnosis 100% complete (non-missing values), one diagnosis must be present Service date 100% complete (non-missing values), must be in valid date format Service location Provider type Procedure code 100% complete (non-missing values), with values specified in data dictionary 100% complete (non-missing values), with values specified in data dictionary 100% complete (non-missing values), with values specified in service instructions Claim number 100% complete (non-missing values) Minutes of service 100% complete for records with no per diem CPT/HCPCS codes a Due to rounding, some fields showing percent completeness may have had a small number of missing data values. 4

5 Example: Electronic Data Check Table (Continued) Table 6. Results of 2013 Electronic Data Checks (Continued). Demographic Data Field State Standard % Complete a RSN ID 100% complete (non-missing values), with values known to DBHR Consumer ID 100% complete (non-missing values) First name 100% complete (non-missing values) Last name 100% complete (non-missing values) Date of birth Optional per the state s Data Dictionary Gender Optional per the state s Data Dictionary % complete (non-missing values), with values specified in data 80.4 Ethnicity dictionary Hispanic origin 100% complete (non-missing values) Language preference 100% complete (non-missing values) Social Security number Optional per the state s Data Dictionary 88.1 Sexual orientation 100% complete (non-missing values) a Due to rounding, some fields showing percent completeness may have had a small number of missing data values. 5

6 2. Frequency of the EDV Required annually More frequent review allows for timely identification of any potential problems Consider the following when deciding how frequently to conduct the EDV: RSN size Number of network providers Results of the risk assessment Results of previous EDV Staff time and other resources Burden on network providers Document decision and explain in report 6

7 3. Drawing a Random Sample: Population Definitions Time period: October September Data source: data submitted to the state Population: all clients with any service paid for by Medicaid Include all services rendered within the time period, regardless of intake/discharge dates 7

8 3. Drawing a Random Sample: Client Sample Size Client sample size is dependent on the size of the RSN (see table). Age distribution of clients selected for the sample must be proportional to the distribution of children (age 0-20) and adults (age 21+) served by the RSN within the 12-month period. # Network Providers Minimum # Client Charts 8

9 3. Drawing a Random Sample: Encounter Sample Size Encounter sample size is dependent on the size of the RSN (see table). Document process and include in the report # Network Providers * * 1,233 Minimum # Encounters *Consider rotating network providers each year. 9

10 3. Drawing a Random Sample: Random Number Generator Random selection of clients should utilize a random number generator that begins with a new seed each time Prevents duplicate lists from being run Especially important for frequent audits so client charts are not reviewed multiple times Computer programs such as Excel, SAS, or SPSS, or websites such as have adequate random number generators. 10

11 3. Drawing a Random Sample: Proportional Age Distribution Determine the proportion of kids and adults served in the study period For example, 1,300 kids served 2,500 clients served = 0.52 = sample includes 52% kids Determine number of kids and adults records to include in the sample For example, 0.52 x 100 client charts = 52 kids charts 11

12 3. Drawing a Random Sample: Over-Sampling Consider over-sampling to capture minority populations, small community providers, etc. Pull stratified random samples (e.g., over-sample non-white adults so that non-white adults make up half of their sample population) 12

13 3. Drawing a Random Sample: Over- Sampling Example Over-sample non-white adults so that non-white adults make up half of the sample population Total population served in the RSN is 1,000 (500 kids, 500 adults) 100 (20%) of adults are non-white Select all 100 non-white adults, then pull a random sample of 100 white adults Add these populations together (n=200) to make the new sample pool Suppose the required sample size is 100 (50 kids, 50 adults); using the process above, randomly select 50 adults from the new sample pool of

14 4. Communication with Network Providers The network providers should receive 30 days notice of EDV visit 3-5 days notice of client list Network providers need enough time to pull client charts, but not too much time to allow charts to be updated Document communication and include in the report 14

15 5. Staff Training: Selecting EDV Staff Involved staff should Be experienced with clinical codes, and be able to determine whether particular code matches service described Have time to dedicate to EDV, so that reviewers are consistent for all network providers 15

16 5. Staff Training: Gold Standard Chart Review Staff should be trained on: How to use the EDV tool Past EDV results for the network providers Agreed upon standards Resource materials (SERI, etc.) Gold Standard chart review prior to the first audit Ensure the tool works properly Reviewers agree on the process Document training and include in the report 16

17 5. Staff Training: Documented Process Training and scoring decisions should be documented so that process can be duplicated even if different staff are involved Scoring decisions document should be updated at every review, and re-visited often 17

18 5. Staff Training: Documented Process Examples Where to look in the record for certain elements (e.g. comb through the entire record, or only look in expected section?) Impact of reviewer familiarity with providers Reviewer assumptions about scoring 18

19 5. Staff Training: Inter-Rater Reliability Inter-rater reliability is part of every EDV review. A second reviewer scores an entire record and compares their scores against the scores of the first reviewer to ensure consistency. What proportion of records will be entered by a second reviewer? (e.g., 10%, 5% after good agreement is established) What is an acceptable level of agreement? (e.g., 95% agreement on demographics and encounters; 80% agreement on golden thread clinical record review) 19

20 6. Developing an EDV Tool: Required Elements Must capture the following elements: Date of service Name of service provider Procedure code Service units/duration Service location Provider type Whether the service code agrees with treatment described 20

21 6. Developing an EDV Tool: Optional Elements RSNs are also encouraged to include Golden Thread elements: Does the assessment substantiate the diagnosis? Is the treatment plan consistent with the diagnosis? Do progress notes address the treatment plan? Consider reviewing other elements while in the client record. Added elements need to be consistent across all network providers. 21

22 6. Developing an EDV Tool: Tool Options The EDV tool must be portable and must not rely on an Internet connection (a tool that can be run from an encrypted laptop s hard drive is ideal). Access database Pre-load encounter data for each agency Automated download procedure, no copy/paste, no manual work, reduces risk for error Excel worksheet Must be locked so that reviewers can only modify the data entry cells Write formulas so that calculations are done automatically (double check formulas, verify results) Paper, with subsequent data entry Use data entry validation process 22

23 6. Developing an EDV Tool: Scoring Methods Match exact match between the electronic data and the clinical record No Match electronic data do not match the clinical records Erroneous encounters that occurred and are presented by the clinical record but contain incorrect data or missing any of the minimum data elements Unsubstantiated (not in the medical record) encounters submitted by the subcontractor but either cannot be verified in the medical record or are duplicated. Missing (not in the encounter record) clinical record contains evidence of a service but it is not represented by the encounter record 23

24 6. Developing an EDV Tool: More About Missing Consider easing the burden of assessing missing scores by: Restricting the time period of the encounter dataset (e.g., one month) Exclude certain codes (e.g., per-diem and crisis services) The process: compare services documented in the clinical record with services in the encounter data. Any service documented in the clinical record that is not represented by an encounter record should be scored Missing. 24

25 Questions? Sara Hallvik Ricci Rimpau Jody Carson 25