Introduction to the Home Health Care CAHPS Survey Webinar Training Session. Session I. January 2016

Transcription

1 Introduction to the Home Health Care CAHPS Survey Webinar Training Session Session I January 2016

2 Session I 2

3 Introduction to the Home Health Care CAHPS Survey Welcome This training session will cover the following: Background and development of the CAHPS Surveys and the Home Health Care CAHPS Survey, in particular: purpose and goals of the Home Health Care CAHPS Survey, and the questionnaire HHA and vendor roles, responsibilities, and participation requirements 3

4 Introduction to the Home Health Care CAHPS Survey (cont d) Sample selection Survey administration procedures and survey protocols for each approved mode The Exceptions Request and Discrepancy Notification Report process Quality assurance requirements 4

5 Training Session Logistics and Reminders Training Session logistics We expect to take a 15-minute break about halfway through this session. If you leave the session at any time, please do not disconnect from either the web or telephone connections. If you get disconnected and have problems accessing the session, call the webinar provider at:

6 Training Session Logistics and Reminders (cont d) Your phone will be muted during the presentation. We will stop for questions after each major topic is presented. The webinar host will explain what you need to do to ask a question. You can submit questions via the Chat feature or ask over the phone. 6

7 Training Session Logistics and Reminders (cont d) Designated HHCAHPS Project Managers should make sure that they have registered for Session II. Each vendor s designated Project Manager must complete the Training Certification Form, which will be available on the HHCAHPS website via an link you will receive, on Wednesday, January 27, Some of the information that this training will cover is not included in the Protocols and Guidelines Manual, Version 8.0, so please be attentive throughout the training session. 7

8 Background and Development of the Home Health Care CAHPS Survey 8

9 The Home Health Care CAHPS Survey Overview The CAHPS Surveys The Home Health Care CAHPS Survey Mode Experiment HHA and Vendor Participation Public Reporting HH PPS Rule for CY 2016 Annual Payment Update Requirements 9

10 Overview of CAHPS Surveys What is CAHPS? Consumer Assessment of Healthcare Providers and Systems Family of surveys designed to collect data from patients and consumers about their experiences with the care they receive from their health care providers Developed by the Agency for Healthcare Research and Quality (AHRQ) CAHPS is a registered trademark of AHRQ, a U.S. government agency 10

11 Overview of CAHPS Surveys (cont d) CAHPS development methods include the following: public call for measures, literature reviews, focus groups with patients, cognitive interviews, stakeholder input, public response to Federal Register notices, and field tests. 11

12 Overview of CAHPS Surveys (cont d) Goals of CAHPS Surveys Conduct the survey in a standardized manner. Analyze and adjust data. Publicly report survey results. Survey results used by consumers to choose a health care provider, by providers to improve the quality of care they deliver to their patients, and to monitor performance of health care providers. 12

13 The Quality Initiative November 2001, the Department of Health and Human Services announced the Quality Initiative. Spring 2003, the Quality Initiative expanded to include home health care agencies (the Home Health Quality Initiative). OASIS measures publicly reported on Home Health Compare at 13

14 The Home Health Care CAHPS Survey Development Timeline September 2006, call for measures issued. In 2007, several rounds of cognitive testing conducted. In 2008, a field test was conducted to evaluate the draft Home Health Care CAHPS Survey instrument. March 2009, the survey was approved by the National Quality Forum. July 2009, the survey was approved by the U.S. Office of Management and Budget. 14

15 The Home Health Care CAHPS Survey Development Timeline (cont d) August 2009, proposed rule CY 2010 HH PPS published. Fall 2009, HHCAHPS mode experiment conducted. October 2009, voluntary participation in the Home Health Care CAHPS Survey began. November 10, 2009, Home Health Prospective Payment System Rate Update for Calendar Year 2010, Final Rule. November 17, 2010, Home Health Prospective Payment System Rate Update for Calendar Year 2011, Final Rule. November 4, 2011, Home Health Prospective Payment System Rate Update for Calendar Year 2012, Final Rule. 15

16 Overview of the Home Health Care CAHPS Survey Measuring experiences of Medicare and Medicaid patients receiving home health care from Medicare-certified HHAs. HHCAHPS is important because it is used to produce comparable data on patients perspectives of care that will allow comparisons of HHAs on domains important to consumers, create incentives for HHAs to improve the quality of care they provide through public reporting of results, and enhance public accountability in health care by increasing transparency of the quality of care provided in return for public investment. 16

17 Overview of the Home Health Care CAHPS Survey (cont d) HHCAHPS is conducted by independent approved survey vendors working under contracts with HHAs. Survey vendors must meet specific requirements to be an approved Home Health Care CAHPS Survey vendor. Sampling and data collection are conducted on a monthly basis. Three modes of data collection are allowed mail only, telephone only, and mail with telephone follow-up of nonrespondents (called mixed mode). 17

18 The Home Health Care CAHPS Survey Questionnaire Contains 34 questions and is available in English, Spanish, Traditional and Simplified Chinese, Russian, and Vietnamese. Includes questions about access to care and communication and interactions with agency staff. Asks patients to rate the care received from the HHA and to indicate willingness to recommend the HHA. Includes demographic questions (self-reported health status, education, whether patient lives alone). 18

19 The Home Health Care CAHPS Survey Questionnaire (cont d) Core Home Health Care CAHPS Survey items (Q1 Q25) Your Home Health Care Q3. When you first started getting home health care from this agency, did someone from this agency talk with you about how to set up your home so you can move around safely? Your Care From Home Health Providers in the Last 2 Months Q10. In the last 2 months of care, did you and a home health provider from this agency talk about pain? Your Home Health Agency Q24. In the last 2 months of care, did you have any problems with the care you got through this agency? 19

20 The Home Health Care CAHPS Survey Questionnaire (cont d) Global Rating Questions Q20. We want to know your rating of your care from this agency s home health providers. Using any number from 0 to 10, where 0 is the worst home health care possible and 10 is the best home health care possible, what number would you use to rate your care from this agency s home health providers? Q25. Would you recommend this agency to your family or friends if they needed home health care? 20

21 The Home Health Care CAHPS Survey Questionnaire (cont d) About You Home Health Care CAHPS Survey questions (Q26 Q34) Q26. In general, how would you rate your overall health? Q28. Do you live alone? Q29. What is the highest grade or level of school that you have completed? Q32. What language do you mainly speak at home? Q33. Did someone help you complete this survey? 21

22 HHA Participation in the Home Health Care CAHPS Survey Overview HHAs interested in participating must be a Medicare-certified home health agency, contract with an approved Home Health Care CAHPS Survey vendor (a list of approved survey vendors is available at Register on the HHA private and secure side of and authorize an approved survey vendor on to submit data for that HHA. 22

23 Vendor Participation in the Home Health Care CAHPS Survey Overview Vendors interested in participating must submit a Vendor Participation Form (available on the project s website), attend the introduction training session and all update training sessions, collect data adhering to the Protocols and Guidelines Manual specifications, prepare and submit a Quality Assurance Plan (QAP), and participate in oversight activities. 23

24 Public Reporting CMS began publicly reporting HHCAHPS Survey results in April A rolling four-quarter average is publicly reported. Data are updated quarterly with the earliest quarter being replaced by data from the most current quarter. Results are not publicly reported for an HHA until that HHA has four quarters or 12 months of data. The HHCAHPS Survey results for an agency are posted with the comparable state and national averages. Data are reported on on the Home Health Compare website along with other home health care quality data (OASIS). 24

25 HH PPS Rule for CY 2016 and the CY 2017 APU Requirements For the CY 2017 Annual Payment Update, HHAs must Participate if their HHA was Medicare-certified prior to April 1, 2015; and Determine if they had 59 or fewer patients from April 1, 2014, through March 31, If so, then file a CY 2017 HHCAHPS APU Participation Exemption Request (PER) form on by March 31, If not, then collect HHCAHPS data from April 2015 through March 2016 on an ongoing basis. 25

26 HHCAHPS APU Participation Periods APU Year CY 2016 CY 2017 CY 2018 HHCAHPS Survey Participation Period Submit an HHCAHPS Survey data file for each month from April 2014 to March 2015 Submit an HHCAHPS Survey data file for each month from April 2015 to March 2016 Submit an HHCAHPS Survey data file for each month from April 2016 to March 2017 Deadline to Submit a PER form March 31, 2015 March 31, 2016 March 31,

27 CY 2016 HH PPS Final Rule for Data Submissions Data must be submitted to the Home Health Care CAHPS Data Center by the data submission deadline for each quarter as specified in the HH PPS Final Rule. These dates are on There will be no exceptions to this requirement. Therefore, all survey vendors are advised to submit data well in advance of the quarterly data submission deadlines. Data submission deadlines for CY 2017 APU and CY 2018 APU are in the CY 2016 HH PPS Final Rule. 27

28 CY 2016 Final Rule and Data Submission Deadlines Beginning with April 2012 data submissions and moving forward, HHCAHPS quarterly data submission deadline dates will always be the third Thursday of the month (in the months of April, July, October, and January), and they are stated in the annual HH PPS Final Rules. Here are some upcoming data submission deadlines: April 21, 2016, for 4th quarter 2015 data July 21, 2016, for the 1st quarter 2016 data October 20, 2016, for the 2nd quarter 2016 data January 19, 2017, for the 3rd quarter 2016 data April 20, 2017, for the 4th quarter 2016 data 28

29 Questions? 29

30 Home Health Care CAHPS Survey Participation Requirements 30

31 Participation Requirements Overview Roles and Responsibilities (CMS, Home Health Care CAHPS Survey Coordination Team, Home Health Agencies, Survey Vendors) Vendor Participation Requirements Vendor Business Requirements Vendor Approval Process 31

32 Roles and Responsibilities Role of CMS and the Home Health Care CAHPS Survey Coordination Team: Provide training to vendors. Provide technical assistance to HHAs and vendors. Ensure integrity of data collection by conducting oversight and quality assurance of survey vendors. Analyze and publicly report Home Health Care CAHPS Survey results. 32

33 Roles and Responsibilities (cont d) Role of Home Health Agencies: Contract with an approved Home Health Care CAHPS Survey vendor to conduct the survey. Compile and deliver to the survey vendor each month a file with information about patients served during the sample month. Monitor survey vendor data submissions by reviewing reports. Preview public reporting results. 33

34 Roles and Responsibilities (cont d) Role of Survey Vendors: Submit a completed Vendor Participation Form. Participate in this training session and any subsequent update trainings. Vendor s project manager must attend the entire session and complete the training certification following the session. We strongly recommend that vendor s sampling and data managers attend. We also recommend that staff from any subcontractors used by survey vendors attend. 34

35 Vendor Participation Requirements Review, agree to, and follow the Participation Requirements listed in Section VI of the Vendor Participation Form. Work with client HHAs to create monthly patient information files. Receive and process monthly patient information files from HHAs. Administer the survey following the specifications in the Home Health Care CAHPS Survey Protocols and Guidelines Manual. 35

36 Vendor Participation Requirements (cont d) Prepare and submit HHCAHPS Survey data files to the Home Health Care CAHPS Survey Data Center. Review data submission reports. Develop and submit a Quality Assurance Plan. Participate and cooperate in all oversight activities conducted by the Coordination Team. 36

37 Vendor Business Requirements Survey vendors must have proven experience conducting surveys using requested mode of data collection, minimum of 3 years in business, and minimum of 2 years conducting surveys of individuals. 37

38 Vendor Business Requirements (cont d) In the CY 2014 HH PPS Final Rule, we finalized this: Defines what is meant by survey experience A survey is defined as the collection of data from individuals selected by statistical sampling methods and the data collected are used for statistical purposes. Applicant vendors must have experience conducting surveys of individuals responding about their own experiences, not of individuals responding on behalf of a business or organization (establishment or institution surveys). 38

39 Vendor Business Requirements (cont d) Vendors must demonstrate that prior survey experience includes statistical sampling experience. The applicant organization has 2 years of experience conducting surveys in which statistical samples of individuals were selected. If staff within the organization have relevant experience while employed by another organization, that experience may not count toward the 2-year minimum. 39

40 Vendor Business Requirements (cont d) Examples of Data Collection Activities that do not satisfy the requirement of valid survey experience: Polling questions administered to trainees or participants of training sessions or educational courses, seminars, or workshops Focus groups, cognitive interviews, or any other qualitative data collection activities Surveys of fewer than 600 individuals Surveys that did not involve statistical sampling methods Internet or web-based surveys Interactive Voice Recognition surveys 40

41 Vendor Business Requirements (cont d) Organizational capability and capacity to collect and process all survey-related data following standardized procedures: personnel, data collection and processing systems, survey management and tracking systems, data submission, and technical assistance/customer support. 41

42 Vendor Business Requirements (cont d) Quality Control Procedures: Train survey personnel. Have procedures and methods to ensure the quality of the data collected. Document and maintain records of quality control activities. 42

43 Vendor Approval Process (This week) vendors must attend Webinar training. Vendors should complete and submit the Vendor Participation Form to the Coordination Team. Assuming that all business requirements are met and training certification is complete, vendors will receive interim approval status. Within 6 weeks after the first quarterly data submission, vendors should submit their Quality Assurance Plan (QAP) to the Coordination Team. Assuming that the QAP meets requirements, vendor will receive final approval status. 43

44 Questions? 44

45 Sampling Procedures Protocols and Guidelines Manual, Chapter IV 45

46 Sampling Overview Sampling Timeframe Overview of the Steps in the Sampling Process 1. Obtain patient file 2. Examine file for completeness 3. Construct frame 4. Select sample 5. Update contact information 6. Assign ID numbers 7. Finalize sample file and initiate data collection 46

47 Sampling Timeframe The survey must be initiated within 21 days after the end of the sample month. If an HHA cannot provide the monthly patient information file in time for data collection to begin within 21 days after the sample month ends, the survey can be initiated within the 22nd and 26th day after the sample month ends. Vendors must submit a Discrepancy Notification Form (DNR) if the survey is initiated within 22 to 26 days after the sample month ends. 47

48 Sampling Timeframe (cont d) CMS may grant approval to initiate the survey more than 26 days after the sample month ends. Vendors must obtain approval from CMS before initiating the survey more than 26 days after the sample month ends. Request approval for initiating the survey more than 26 days after the sample month ends by sending an to the HHCAHPS Coordination Team. The should include the agency s name, CCN, affected sample month, and the reason for the late start. The vendor should copy the HHA on these requests. 48

49 Steps in the Sampling Process 49

50 Steps in the Sampling Process 50

51 Step 1: Obtain Patient File From HHA Vendors must work with HHAs to obtain all required data elements for patients served during the sample month. HHAs are required to provide the vendor with a monthly patient information file for each sample month, even if the HHA did not serve any patients who are eligible for the survey during the sample month. It is also acceptable for the HHA to send the vendor an in lieu of a 0-eligible file. 51

52 Step 1: Obtain Patient File From HHA (cont d) If an HHA does not submit a monthly patient information file for a sample month, the vendor must submit a Discrepancy Notification Report (DNR) for that HHA. This applies for HHAs that have already begun their participation in the HHCAHPS Survey. For new HHA clients, do not submit a DNR to indicate that the HHA missed a month unless the HHA has already administered the survey in a prior month. 52

53 Step 1: Obtain Patient File From HHA (cont d) The HHA and the survey vendor decide which organization EXCLUDES patients who are not eligible for the survey from the monthly patient information file. In some instances, the HHA will make the exclusions. If the HHA makes the exclusions, the vendor must provide the exclusion criteria to the HHA. If the vendor makes the exclusions, the HHA must provide all of the patient information that the vendor will need to make the exclusions. 53

54 Step 1: Obtain Patient File From HHA (cont d) The following patients are eligible to be included on the sampling frame. Eligible patients are those who: have Medicare or Medicaid as a payer, are at least age 18 at the end of the sample month, have at least one visit for skilled care in the sample month and two skilled visits during the lookback period, are not known to be deceased, are not currently receiving hospice care, received home visits for reasons other than only routine maternity care, are not no publicity patients, and are not state-regulated patients. 54

55 Step 1: Obtain Patient File From HHA (cont d) Definition of No publicity Patients These are patients who specifically request the HHA not to release any information about them to anyone other than agency personnel. Home health agencies must not ask their patients if they want to be in the survey or if the HHA can provide information about the patient to the survey vendor. 55

56 Step 1: Obtain Patient File From HHA (cont d) Definition of State-regulated Patients State-regulated patients are those who have certain conditions or illnesses and live in a state that prohibits the release of information about patients who have those conditions. 56

57 Step 1: Obtain Patient File From HHA (cont d) There are two additional reasons that a patient may be excluded. 1. Patients who have harmed the health or well-being of a home care provider or threatened to harm the health or well-being of a home health provider. HHAs must alert the vendor when they submit the monthly patient information file that a patient has been excluded from the file for this reason. 57

58 Step 1: Obtain Patient File From HHA (cont d) The final reason for excluding a patient 2. After a patient has been included in a sample, he or she is not eligible to be included in the sample for the next 5 months. The vendor must check its files to identify and exclude patients sampled in the last 5 months. Only the survey vendor can make this exclusion because the HHA will not know which patients were sampled in the last 5 months. For the full set of exclusion criteria, see Chapter IV in the Protocols and Guidelines Manual. 58

59 Step 1: Obtain Patient File From HHA (cont d) For purposes of this survey, the basis for a skilled home health visit is the classification of the agency employee who visited the patient and not the reason for the visit. A skilled visit is a visit made by a registered nurse (RN) or licensed practical nurse (LPN), a physical therapist or physical therapist assistant, an occupational therapist or occupational therapist assistant, or a speech therapist or speech therapist assistant. Skilled visits do not include visits by social workers, home health or personal care aides, or nursing aides. They also do not include visits made to train or evaluate personnel who are not making a skilled visit. 59

60 Step 1: Obtain Patient File From HHA (cont d) The lookback period is the sample month and the month immediately preceding the sample month. For example, if the sample month is February, the lookback period is January 1 through February 28. To be eligible for the survey, the patient must have had at least one skilled visit in the sample month and two such visits in the lookback period. The patient could have had one skilled visit in the sample month and one in the preceding month, or two in the sample month and none in the preceding month, or two or more in the sample month and in the preceding month. 60

61 Step 1: Obtain Patient File From HHA (cont d) Examples of Skilled Visits and Survey Eligibility For the December 2015 sample month, if a patient has three aide visits in December and six speech therapist visits in November, that patient is ineligible. one speech therapist visit in December and one RN visit in November, that patient is eligible. three RN visits in December and one aide visit in November, the patient is eligible. 61

62 Step 1: Obtain Patient File From HHA (cont d) Vendors must ensure that HHA files: Include all information about all patients served during the sample month (except allowable exclusions), including those discharged during the sample month. Contain patients from all units and branches of the HHA filing under the same CMS Certification Number (CCN). Are submitted to their vendor by an agreed-upon date each month. 62

63 Step 1: Obtain Patient File From HHA (cont d) Table 4.1 in the Protocols and Guidelines Manual contains a complete list of data elements the HHA must provide. Some of the patient information that the HHA must provide includes the following: patient name and contact information (address, telephone number); date of birth; medical record number, the unique ID assigned by the HHA to enable the tracking of the patient s care; source(s) of payment for the home health care; and number of skilled home health visits during the sample month and lookback period. 63

64 Step 1: Obtain Patient File From HHA (cont d) The HHA must also provide for each patient in the file primary diagnosis (underlying reason for home care), other diagnoses (if any), and Activities of Daily Living (ADLs). If necessary, the HHA can send diagnosis and other information to the vendor after the survey begins for the sample month. 64

65 Step 1: Obtain Patient File From HHA (cont d) Activities of Daily Living Five ADLs are tracked and the maximum number to report is five. The valid range for the Bathing ADL in OASIS-C is 0 6. The XML file will accommodate values from both OASIS-B and OASIS-C. 65

66 Step 1: Obtain Patient File From HHA (cont d) Activities of Daily Living (cont d) Vendors must include on the XML data file the same ADL information that the HHA provides on the monthly patient information file. HHAs can provide either the total number of ADL deficits or the number of deficits for each individual ADL. They do not have to provide both. Vendors are not permitted to calculate the total number of ADL deficits using the number of deficits provided for the individual ADLs. The only acceptable values of the ADL Deficit Count are 0-5 and M. If an HHA submits a value exceeding 5, the vendor should recode the total to 5. 66

67 Step 1: Obtain Patient File From HHA (cont d) HHAs should know and be able to identify patients with ESRD. ICD-10 diagnosis codes that are strong indicators are N18.6, Z91.15, and Z

68 Step 1: Obtain Patient File From HHA (cont d) Source of Payment If the HHA does not provide the source of payment, the vendor should try to retrieve it from the HHA before sampling. If the HHA cannot provide the source of payment in time for the vendor to select the sample, the vendor should include the patient on the sample frame if the patient meets all other survey eligibility criteria. Vendors must indicate on the XML data file they submit to the HHCAHPS Data Center whether the source of payment is known, assumed, or missing for each patient. Vendors should use A for assumed on their submitted data files. 68

69 Step 1: Obtain Patient File From HHA (cont d) A patient may have multiple sources of payment. Medicare or Medicaid could be a primary or secondary payer. If Medicare or Medicaid is primary or secondary for the sample month, the standard visit count and other exclusions should be applied, even if some visits might be covered by another payer. HHAs must also provide vendors with the total count of all patients served during the sample month, irrespective of exclusions. 69

70 Step 1: Obtain Patient File From HHA (cont d) If OASIS is the source of data: The HHA should obtain patient information from the most recent OASIS Assessment conducted, but sometimes it may be necessary to obtain data from earlier assessments or other sources. For example: Whether the patient was admitted from a recent inpatient setting is on the OASIS Start of care (SOC) and Resumption of care (ROC) assessments only. Diagnoses are on the follow-up as well. 70

71 Step 1: Vendors Take Note The HHA must provide information about all of its patients who received skilled care, except the allowable exclusions. If an HHA wants to survey its non-medicare or non- Medicaid patients, the vendor must remove these patients from the data file before submitting it to the HHCAHPS Data Center (reminder that non-medicare and non-medicaid patients do not count toward sampling targets). If the HHA is conducting other surveys of its patients, it must provide patient information files to the vendor before selecting the sample for its other surveys. 71

72 Steps in the Sampling Process 72

73 Step 2: Examine the File for Completeness Vendors should examine the file to ensure that it contains the information needed to confirm survey eligibility based on information provided by the HHA (date of birth, payment source, number of skilled visits). If the HHA does not provide an address or telephone number for a patient, the vendor should recontact the HHA. HHAs must have an address at which home care is provided. In most cases, the HHA will have the patient s telephone number to schedule visits and to confirm provider visits. Even if no address or telephone number is provided, the patient is still eligible for the survey if he or she meets all other eligibility criteria. 73

74 Step 2: Examine the File for Completeness (cont d) Vendors should contact the HHA to obtain any information needed to determine survey eligibility or to survey sampled cases if information is missing. Information must be requested for all patients included in the sample month file. HHAs can submit data needed for analysis (admission source, diagnosis information, etc.) on a second file after the survey is initiated but soon enough to meet file submission deadlines. If the HHA submits more than one file, it is essential that there is enough patient information on each to link files and to deduplicate patients. 74

75 Step 2: Examine the File for Completeness (cont d) Vendors must select the sample at one point in time; that is, vendors should not select two separate samples for the same month. If an HHA submits a second file with additional patients, and data collection has already begun for that sample month, the vendor should not select a second sample. For this situation, the vendor should submit a Discrepancy Notification Report. 75

76 Steps in the Sampling Process 76

77 Step 3: Identify Eligible Patients and Construct Sample Frame An example of a layout of a sample frame that shows information needed to determine survey eligibility is included in Appendix B of the Protocols & Guidelines Manual. The example layout contains the name, length, etc. of each data element that should be included on the sample frame. It also indicates which data elements are required for data submission. 77

78 Step 3: Identify Eligible Patients and Construct Sample Frame (cont d) The vendor should confirm that the patients included on the file are eligible based on age and number of skilled visits. Verify that all patients included on the file are 18 years old or older at the end of the sample month. Verify that each patient included on the file received one skilled visit in the sample month and two skilled visits during the lookback period. Patients missing some eligibility criteria should be considered eligible as long as all other eligibility criteria are met. Exclude patients who have been included in the survey sample in the past 5 months, whether or not they responded. 78

79 Step 3: Identify Eligible Patients and Construct Sample Frame (cont d) Vendors should construct the sample frame, which will contain information about all patients eligible for the survey. Keep the original monthly patient information file intact. If the monthly patient information file is to create the sample frame, copy the file and work in the copy. Retain both the original monthly patient information file and the sample frame that is created for possible audit during site visits. 79

80 Step 3: Identify Eligible Patients and Construct Sample Frame (cont d) Vendors cannot calculate the number of skilled visits in the lookback period. Contact the HHA if the number of skilled visits in the lookback period is missing. If an HHA or its IT vendor provides the dates of all visits in the lookback period instead of the total number of visits, it is acceptable for the vendor to calculate the total number of visits in the lookback period. If the HHA cannot provide the number of lookback period visits, enter M (for Missing) for this data element on the XML data file. If the HHA cannot provide the number of visits, consider the patient eligible for the survey if he or she meets all other survey eligibility criteria. 80

81 Step 3: Identify Eligible Patients and Construct Sample Frame (cont d) Sampling Documentation Reminders Documentation will be subject to review by the HHCAHPS Survey Coordination Team. For quality assurance purposes, vendors must keep all HHA patient files for at least 18 months. Vendors must also record and retain documentation indicating the reasons patients were excluded from an HHA s sample frame. 81

82 Steps in the Sampling Process 82

83 Step 4: Determine a Sample Size and Select a Sample for Each HHA Sample Requirements The reliability target for the survey ratings and most of the composites is 0.8 or higher. For statistical precision, a targeted minimum of 300 completed surveys per HHA over a 12-month period is needed. This is an average of 25 completed surveys per sample month. Some HHAs will not serve enough patients to yield 300 completed surveys over a 12-month period. 83

84 Step 4: Determine a Sample Size and Select a Sample for Each HHA (cont d) Vendors must calculate and use a sampling rate for each HHA, including small HHAs, to ensure that an even distribution of patients is sampled over a 12-month period. Because there is variability in the number of patients served and changes in an HHA s personnel or operations over time, selecting a sample of patients using a sample rate will ensure that patients perspective of the care received at different points in time is captured. 84

85 Step 4: Determine a Sample Size and Select a Sample for Each HHA (cont d) Select an Appropriate Sample Size Each Month The sampling rate must be sufficient to yield the targeted minimum 300 completed surveys. The vendor should consider the HHA s mix of patients; that is, whether the HHA serves more short- or longterm patients, when estimating a sampling rate. 85

86 Step 4: Determine a Sample Size and Select a Sample for Each HHA (cont d) Select an Appropriate Sample Size Each Month The sampling rate may be adjusted over time, but it should remain approximately the same for each month in a quarter. The rate can be increased to reach the target, but it must not be abruptly changed to avoid exceeding 300 completed surveys. It is preferable that the sampling rate be the same from quarter to quarter. 86

87 Step 4: Determine a Sample Size and Select a Sample for Each HHA (cont d) Estimate a Sampling Rate The first month that an HHA participates in the HHCAHPS Survey, the agency will have a larger number of patients eligible to participate because no patients will have been previously sampled. The sampling rate for the HHA s first sample month should be adjusted to make the sample size more like a typical month. Vendors should work with their new HHAs to estimate the sample frame size for each month, using the previous 3-6 months of information on eligible patients served. The more months included when calculating a sampling rate, the more accurate the estimate will be. 87

88 Step 4: Determine a Sample Size and Select a Sample for Each HHA (cont d) The mode of administration and the expected response rate will be important in determining the sample size Mode Response rate Sample size for 25 responses/month Mail only 30.7% 81 Phone only 34.6% 72 Mixed 39.2% 64 The response rates in this table reflect those received during the HHCAHPS Survey participation period for the calendar year 2016 Annual Payment Update. 88

89 Step 4: Determine a Sample Size and Select a Sample for Each HHA (cont d) Sample Size = (Number of responses needed)/ (Response Rate) = 25/(Response Rate) 89

90 Step 4: Estimating HHA Sample Size Using 300 as the target number of completed surveys and an estimate of a final response rate, calculate the target sample for a year and 1/12 of that per month. An effective response rate of 25%, for example, would require an annual sample of 1,200 (300/.25), which is a target of 100 per month. The estimated sampling rate to achieve the target sample size should be determined from the 3-6 months of data received from HHAs when they sign on with their vendor. Actual data received over time will be used to refine the rate. 90

91 Step 4: Estimating HHA Sample Size (cont d) Apply patient exclusions for each month based on the HHCAHPS survey eligibility criteria. Each patient sampled is excluded from the sample frame for the next 5 months. Depending on the average patient length of stay, this exclusion may reduce the eligible frame considerably. Simulate drawing a sample of approximately the same size from each of the months. 91

92 Step 4: Estimating HHA Sample Size (cont d) Using the second and subsequent months to approximate what the typical frame size will be, determine typical sampling rate each month. Ensure that all of the eligible patients are not selected in the first month, leaving no patients in subsequent months. HHAs will vary in the average length of stay of their patients. The prior sampling exclusion will have a smaller effect on the frame when patients have a short length of stay than when they have a long length of stay. This should become apparent in the simulation process. 92

93 Step 4: Estimating HHA Sample Size (cont d) Simulate creation of random numbers to select the target sample size each month. The proportion sampled each month or quarter may vary to meet the required sample size. The first survey month for any client will generally be sampled at a lower rate than subsequent months. Adjustments may be needed over time to reach the annual target of (a) 300 over each of the rolling four quarter periods; or (b) a sample that approximates the HHA s annual number of patients. All patients sampled in a sample month must be surveyed. The target of 300 is not a quota after which surveying or processing can stop. 93

94 Step 4: Estimating HHA Sample Size (cont d) Sampling at 100% Assume that ABC Home Care had 35 eligible patients the first month it participates in the survey. # Sampled Eligible in Month 1 35 No. Patients Eligible in Months No. Patients Eligible in Month 7 35 If all patients are surveyed in Month 1, there would be no patients to sample in Months 2-6 unless the HHA served some new patients in those months. 94

95 Step 4: Selecting the HHA Sample Use a random number generator to select the sample. Usually simple random sampling will be used. Generate a number from the Uniform Distribution each number having an equal probability of selection. 95

96 Step 4: Selecting the HHA Sample (cont d) Use a random number generator with a seed not set by the user, and retain the seed for quality assurance. Software that meets the requirements: SAS v9 or RAT- STATS. RAT-STATS is a free program available from the HHS Office of Inspector General at Do NOT use a spreadsheet random number generator or similar program that has not passed rigorous testing. 96

97 Step 4: Selecting the HHA Sample: Method 1 Sort the sample frame of N eligible patients. The sort order should be retained in the documentation. Generate N random numbers. Assign the random numbers in the order generated to each patient included on the frame. Re-sort the patients as ordered by the random numbers. Select the first n, the sample size required for the mode used. 97

98 Step 4: Selecting the HHA Sample: Method 2 If the random number generator can produce a set of integers, it may be able to produce n integers between 1 and the frame size, N, that can be used to select n numbered patients in the frame. If, for example, a sample of 100 is needed from a frame of 150, the program may be set to produce 100 integers uniformly distributed between the numbers 1 and 150. Each of the n random numbers is the number of one of the patients in the ordered frame. 98

99 Step 4: Sampling Other Than Simple Random Sampling Simple Random Sampling (SRS) is the standard method of sampling. Two alternative methods can be used: Proportionate Stratified Random Sampling (PSRS) Disproportionate Stratified Random Sampling (DSRS) PSRS and DSRS may be used if there are logical strata with sufficient sample size. The minimum allowable eligible patients for a month in a stratum is 10. The sampling method for an HHA may be changed, but not within a quarter. 99

100 Step 4: Proportionate Stratified Random Sampling PSRS may be a reasonable choice if the HHA would like to keep track of sample size and results from the survey for each stratum, or the HHA wants to designate branches or other aggregates of operating units for tracking, while using the same sampling rate for each. The PSRS sampling method is described in more detail in Chapter IV of the Protocols and Guidelines Manual. 100

101 Step 4: An Example of Proportionate Stratified Random Sampling (PSRS) Assume that an HHA has three branches: Branch A has 300 patients eligible for the survey in a sample month. Branch B has 200 eligible patients. Branch C has 100 eligible patients. Of the 600 patients (from all three branches), assume that 100 are to be sampled. In PSRS, 1/6 (16.7%) will be sampled from each branch with each having its own sample frame. 101

102 Step 4: An Example of Proportionate Stratified Random Sampling (PSRS) (cont d) The sampling rate for each branch would be the same at 16.7%. In this example, the number selected for each branch would be: Branch A = 50 Branch B = 33 Branch C = 17 Simple random sampling may be used in each of the strata (branches) to select the sample. 102

103 Step 4: Disproportionate Stratified Random Sampling (DSRS) DSRS is a good sampling option if an HHA wishes to achieve statistically more precise ratings for branches. Vendors must submit an Exceptions Request Form to use this type of sampling (see Chapter XIV of the Protocols and Guidelines Manual for more information about the Exceptions Request Form). The DSRS sampling method is described in more detail in Chapter IV of the Protocols and Guidelines Manual. 103

104 Step 4: Disproportionate Stratified Random Sampling (DSRS) (cont d) Use the XML template for DSRS when preparing the data submission file when this sampling method is used. The header record on the DSRS XML template requires additional data elements to allow appropriate weighting of the samples. 104

105 Step 4: An Example of Disproportionate Stratified Random Sampling Assume an HHA has three branches. In a sample month, Branch A has 300 eligible patients, Branch B has 200 eligible patients, and Branch C has 100 eligible patients. 105

106 Step 4: An Example of Disproportionate Stratified Random Sampling (cont d) Assume that the HHA wants to keep the sample size for Branch A at 50 patients but wants to increase the sample size of the smallest HHAs (Branch B and Branch C) to 50. In this case the disproportionate sampling rates would be: Branch A: 50/300 = 16.7%, Branch B: 50/200 = 25%, and Branch C: 50/100 = 50%. 106

107 Step 4: An Example of Disproportionate Stratified Random Sampling (cont d) The sample for each branch will be selected using the sampling rate calculated for each branch. Information sufficient to compute the sampling rate for each stratum must be reported when the data are submitted. This includes the total number of patients in the stratum in which the HHA provided services during the sample month, the total number of patients in that stratum who were eligible for the survey during that month, and the total number of patients sampled. To adjust for the differences in the probability of patients being selected from each branch, the inverse of the sampling rates will be applied as weights. 107

108 Steps in the Sampling Process 108

109 Step 5: Verify or Update Sample Contact Information It is strongly encouraged that vendors send all HHAprovided contact information for sampled cases to an outside address service (e.g., National Change of Address) for address verification/update. Vendors using telephone-only or mixed mode are also encouraged to use a commercial telephone number look-up service to verify/update telephone numbers for sampled patients. Performing these steps prior to the start of data collection will result in fewer returned surveys or unsuccessful call attempts. 109

110 Steps in the Sampling Process 110

111 Step 6: Assign Unique Sample Identification Numbers Vendors must assign a unique alphanumeric sample identification (SID) number to each sample member. The SID number cannot include any information that would identify the HHA that served the patient. This includes patient s medical record number, date of birth, street or house number, etc. Do not reuse SID numbers if a patient is sampled more than once, assign a new SID number to him or her each time. 111

112 Steps in the Sampling Process 112

113 Summary of Sample Selection Procedures Sample frame construction, sample selection, and data collection will occur on a monthly basis. Survey vendors must: Calculate a sampling ratio for each HHA using patient information from a 3- to 6-month period. Use the same sampling ratio for all 3 months in a quarter. Submit an Exceptions Request Form for the HHA if the HHA wants to sample more frequently. 113

114 Summary of Sample Selection Procedures (cont d) PSRS and DSRS sampling are permitted, but vendors must submit an Exceptions Request Form prior to using DSRS. There must be sufficient sample sizes to use PSRS and DSRS. The minimum number of eligible patients in each stratum in a sample month is 10. The same sampling method (SRS, PSRS, or DSRS) must be used for an HHA in all 3 months of the quarter. 114

115 Common Sampling Mistakes Patients with missing or incomplete addresses and telephone numbers were being coded as ineligible. The lookback period was not being correctly defined (i.e., 60 days vs. month preceding the sample month and the sample month). All eligible patients were being surveyed each sample month; that is, a sampling rate was not calculated and used to determine the sample size. If there are two or more home health patients in the same household, only one was considered eligible to be included on the sample frame. 115

116 Common Sampling Mistakes (cont d) The vendor was calculating and using the number of skilled visits in the lookback period instead of using the number provided by the HHA. The vendor was calculating the total number of ADLs rather than accepting the information provided by the HHA. If the source of payment data element was missing on the HHA file, the patient was being excluded from the sample frame. The SID number was being used more than once. Once a SID number is assigned, it must never be used again. 116

117 Common Sampling Mistakes (cont d) The medical record number (MRN) was the only data element being used to identify duplicate patients. Vendors calculated Patient Age as of the beginning of the sample month. Vendors did not retain the Seed Number or random numbers used during sampling. Vendors should be able to replicate their sample selection process. Vendors did not retain documentation of why patients were ineligible for sampling. Vendors were excluding patients if any of the eligibility data elements were missing. 117

118 Sampling Reminders Here are some reminders, based on site visit and QAP observations. If the HHA does not provide all of the information needed for administering the survey or for analysis, vendors should work with their HHAs to obtain the missing information. Vendors that have automated the receipt and processing of data files and the sampling process must have quality control procedures in place to ensure that these systems are working properly. The sampling rate can be adjusted each quarter (not monthly). 118

119 Sampling Reminders (cont d) The total number of eligible patients entered on the XML file must include patients who were sampled but later identified during data collection as ineligible. Vendors should include on the XML file the ADL deficit count or the individual ADL values as provided by the HHA rather than compute the ADL deficit count themselves. If an HHA switches vendors, the outgoing vendor should not provide any information about patients sampled in months prior to the new vendor. 119

120 Questions? 120

121 Survey Administration Procedures Protocols and Guidelines Manual, Chapters V-VII 121

122 Survey Administration Overview Survey Instrument and Materials Supplemental Questions Modes of Survey Administration Survey Management Data Confidentiality, Security, and Storage 122

123 Survey Instrument and Materials Survey Content Core Questions (Qs. 1 25) About You Questions (Qs ) Survey Materials Available: English (Appendix C) Spanish (Appendix D) Simplified and Traditional Chinese (Appendices E and F) Russian (Appendix G) Vietnamese (Appendix H) Supplemental Questions (Appendix J in all five languages) 123

124 Survey Instrument and Materials (cont'd) Instrument Translations The HHCAHPS Survey is currently available in five languages: English, Spanish, Chinese (mail only), Russian, and Vietnamese. Vendors must use the CMS-approved translations of the HHCAHPS Survey questions and responses. Send requests for additional languages to the Coordination Team. 124

125 Survey Administration Overview Survey Instrument and Materials Supplemental Questions Modes of Survey Administration Survey Management Data Confidentiality, Security, and Storage 125

126 Supplemental Questions HHAs may add their own questions to the HHCAHPS Survey questionnaire or telephone interview. HHAs may also choose to use some or all of the HHCAHPS supplemental questions. Supplemental questions are under the Survey and Protocols tab at Consent to Share Responses supplemental Question. Supplemental questions must be placed after the core HHCAHPS Survey questions (Questions 1 25). Supplemental questions may be placed either before or after the HHCAHPS Survey About You questions. 126

127 Supplemental Questions (cont d) Vendors must use CMS-approved translations of the HHCAHPS Survey supplemental questions. HHAs or their vendors are responsible for translating any additional questions added to the HHCAHPS Survey instrument. If vendors add their own questions, consider adding some transitional phrasing to help focus the respondent on the new questions. 127

128 Supplemental Questions (cont d) If questions are added, we strongly recommend that vendors be consistent in how these questions are formatted (should be identical to the rest of the questions). Supplemental questions do not need to be approved or reported to CMS. We strongly recommend that agencies/vendors avoid sensitive questions and lengthy additions, because these will likely reduce response. 128

129 Supplemental Questions (cont d) Supplemental questions cannot repeat any HHCAHPS core items. Supplemental questions cannot be used for marketing or promoting HHA services. Supplemental questions cannot ask for identification of people who might need home health services. The Consent to Share Responses supplemental question provides a way for vendors to determine whether the respondent will allow his or her name to be linked with his or her responses. 129