Introduction to CAHPS for Physician Quality Reporting System (PQRS) Survey Survey Vendor Training

Transcription

1 Introduction to CAHPS for Physician Quality Reporting System (PQRS) Survey Survey Vendor Training July 2015

2 Welcome! In today s CAHPS for PQRS Survey training, we will: Review Key Concepts and Protocols Provide Updates on the 2015 Survey Administration Review the 2015 Data Collection Schedule Administer the Quiz 2

3 Post Training Quiz Each survey vendor must complete and pass a Post Training Quiz in order to be approved to administer the 2015 CAHPS for PQRS Survey Quiz must be completed online immediately after training One Quiz Form will be accepted per survey vendor We will notify survey vendors of their Post Training Quiz results by July 24,

4 Overview and Background

5 Topics Overview of the PQRS Program About the CAHPS for PQRS Survey Summary Survey Measures (SSMs) New for 2015 Survey Administration 5

6 About the Survey The Consumer Assessment of Healthcare Providers and Systems (CAHPS) Survey for group practices participating in the Physician Quality Reporting System (PQRS) Group Practice Reporting Option (GPRO) was developed to collect information about patient experience and care within that group practice The CAHPS for PQRS Survey includes the core questions contained in the CAHPS Clinician & Group Survey (Version 2.0), plus additional questions to measure: Access to and use of specialist care Experience with care coordination Patient involvement in decision-making Experiences with a health care team Health promotion and patient education Patient functional status and general health 6

7 Surveyed Patient Population CMS assigns Original Medicare beneficiaries to the group practice based on utilization during the program year Assigned patient population is the basis for the measurement and reporting of CAHPS for the PQRS program Groups can elect to include their (PY) 2015 CAHPS for PQRS performance in the calculation of their 2017 Value-Based Payment Modifier Assignment will not affect beneficiaries guaranteed benefits or choice of doctor or any other provider 7

8 SSMs in CAHPS for PQRS Survey 12 SSMs Source 1. Getting Timely Care, Appointments & Information 2. How Well Your Providers Communicate 3. Patient s Rating of Provider 4. Access to Specialists 5. Health Promotion and Education 6. Shared Decision-making 7. Health Status & Functional Status 8. Courteous & Helpful Office Staff 9. Care Coordination 10. Between Visit Communication 11. Helping You Take Medications as Directed 12. Stewardship of Patient Resources CG CAHPS Core, CG CAHPS Supplemental, New content 8

9 2015 Survey Administration Beginning with the 2015 reporting period (data to be collected in November 2015), CMS will NOT bear the cost of survey administration All group practices of 100+ Eligible Providers (EPs) participating via the PQRS GPRO must report the CAHPS for PQRS Survey CAHPS for PQRS Survey is optional for groups with 2-99 EPs participating via the PQRS GPRO 9

10 Resources For more information about the PQRS program Patient-Assessment-Instruments/PQRS/index.html 10

11 Program Requirements

12 Objectives Communicating with Beneficiaries Roles and Responsibilities Minimum Survey Vendor Business Requirements Technical Assistance and Communication 12

13 Communicating with Beneficiaries If the group practice chooses to notify beneficiaries, all beneficiaries must be notified Group practices and survey vendors are strongly encouraged not to: Ask any CAHPS for PQRS Survey questions of Original Medicare beneficiaries 4 weeks prior to, during or after survey administration Group practices and survey vendors must not: Attempt to influence or encourage beneficiaries to answer survey questions in a particular way Imply that the group practice, its personnel or agents will be rewarded or gain benefits for positive feedback Offer incentives of any kind 13

14 Roles and Responsibilities CMS provides: Survey administration protocols and timeline o Quality Assurance Guidelines Version 1 Training of survey vendors Technical assistance Tools, format and procedures for submitting the collected data Analyses of data and official reports o Group practice reports and Excel files detailing survey scores 14

15 Roles and Responsibilities (cont'd) Group practices will: Contract with a CMS approved survey vendor Authorize the survey vendor to collect and submit data on their behalf o Authorize survey vendor no later than 9/22/

16 Roles and Responsibilities (cont'd) Survey vendors will: Review and acknowledge agreement with the rules of participation Participate in: o Introduction to CAHPS for PQRS Survey Webinar Training Session Successfully complete a quiz measuring comprehension of CAHPS for PQRS Survey protocols o All CAHPS for PQRS Survey update training sessions, as scheduled Follow CAHPS for PQRS Survey administration requirements Complete and submit the Vendor Access to PQRS Data Warehouse Form by 9/8/

17 Roles and Responsibilities (cont'd) Survey vendors will: Execute a Data Use Agreement (DUA) with CMS by 9/30/2015 o DUA restricts use of sample and survey data o Data that are appended to the sample file or beneficiary survey data for reporting to clients must be approved in advance o the CAHPS for PQRS Project Team to get approval Verify the group practice has authorized the survey vendor Receive and perform checks of each group practice s sample file Administer the CAHPS for PQRS Survey o According to protocols and procedures established by CMS o Following the required data collection schedule o Oversee the quality of work of staff and subcontractors Submit data on behalf of the group practice Review CAHPS for PQRS Survey data submission reports 17

18 Roles and Responsibilities (cont'd) Survey vendors will: Attest to the accuracy of their data collection processes Implement security procedures aligned with HIPAA and CMS Privacy Requirements Develop a Quality Assurance Plan (QAP) and submit by 8/12/2015 Submit Mail materials by 9/18/2015 Submit CATI screenshots by 10/9/2015 Participate in oversight activities 18

19 Minimum Survey Vendor Business Requirements Applicant organizations (vendor and subcontractors) must currently possess all required facilities and systems to implement the CAHPS for PQRS Survey Subcontractors will be subject to the same requirements as the applicant vendor Subcontractors with key roles in the survey administration are required to attend all training sessions If approved, survey vendors will be required to maintain at least one active CAHPS for PQRS Survey client for each of two consecutive survey cycles 19

20 Minimum Survey Vendor Business Requirements (cont'd) Possess relevant survey experience Demonstrate survey capability and capacity Participate in all training sessions Adhere to all protocols and specifications Participate in oversight activities 20

21 Technical Assistance and Communication For additional information and technical assistance: Phone (toll free): For data submission issues: To learn more about the CAHPS for PQRS Survey and to see important new updates and announcements, visit the CAHPS for PQRS Survey website at 21

22 Sample Design and Beneficiary Selection

23 Objectives Overview Sample File Layout Delivery of Sample File to Vendors 23

24 Overview CMS selects sample for each group practice Original Medicare beneficiaries assigned to the group practice 18 years or older Live in the United States, Puerto Rico or U.S. Virgin Islands Ineligible beneficiaries include individuals known to be institutionalized 24

25 Overview (cont d) Target sample of 860 beneficiaries will be drawn by CMS for the 2015 survey If the practice has fewer than 860 beneficiaries, there are rules for practices to still participate 25

26 Overview (cont d) For large group practices of 100 or more eligible providers: If the practice has fewer than 860 beneficiaries, but more than 415 beneficiaries, all eligible beneficiaries will be surveyed for the 2015 reporting period If the practice has fewer than 416 beneficiaries, the survey cannot be conducted 26

27 Overview (cont d) For group practices with 25 to 99 eligible providers: If the practice has fewer than 860 beneficiaries, but more than 254 beneficiaries, all eligible beneficiaries will be surveyed for the 2015 reporting period If the practice has fewer than 255 beneficiaries, the survey cannot be conducted 27

28 Overview (cont d) For group practices with 2 to 24 eligible providers: If the practice has fewer than 860 beneficiaries, but more than 124 beneficiaries, all eligible beneficiaries will be surveyed for the 2015 reporting period If the practice has fewer than 125 beneficiaries, the survey cannot be conducted 28

29 Sample Files CMS will provide the most complete and current contact information available for sampled beneficiaries Address as of October 2015 Phone number as of October 2015 Oversample high users of care 25% of each group practice sample Drawn from top 10% of beneficiaries based on visits 29

30 Sample File Layout File Record Layout for the Sample File RAND Field Name Starting Position in Record Field Length Valid Codes Field Contents FINDER 1 8 numeric Unique beneficiary finder number assigned by CAHPS for PQRS Survey Data Coordination Team FNAME 9 30 text CMS beneficiary first name MNAME text CMS beneficiary middle name LNAME text CMS beneficiary last name DOB_C 94 8 yyyymmdd Date of birth ZIP char Mailing address zip code ADDR1FINAL text Mailing address line 1 ADDR2FINAL text Mailing address line 2 CITY text Mailing address city name PR_CD text Puerto Rican urbanization code STATE char Mailing address USPS state code FIPS_STATE char CMS state FIPS code, 2 numbers with leading zeros FIPS_CNTY char CMS county FIPS code, 3 numbers with leading zeros 30

31 Sample File Layout (cont'd) RAND Field Name Starting Position in Record Field Length Valid Codes Field Contents GENDER Gender code: 1 = male, 2 = female PQRSGROUP_ID [G]nnnn Five-character PQRS identifier: begins with a letter G, followed by 4 numbers GROUPPRACTICE_ NAME Free text Group practice name provided by CMS FOCALTYPE numeric Provider type: 1= primary care, 2 = specialist PRTITLE text Type of provider (physician, physician assistant, nurse practitioner, certified nurse midwife, certified clinical nurse specialist) PRFNAME text Provider first name PRLNAME text Provider last name TELEPHONE NUMBER char Beneficiary phone number 31

32 Delivery of Sample File CAHPS for PQRS Survey data coordination team will: Provide a separate file for each survey vendor Authorize survey vendors to access the secure PQRS data warehouse Provide encrypted sample files to the secure data warehouse at o Survey vendor access is restricted to their own files 32

33 Data Collection Protocol

34 Objectives Overview 2015 Data Collection Schedule Mail Protocol Phone Protocol Languages 34

35 Overview Mixed-mode survey administration Survey vendors must be prepared to administer questionnaires in English and one or more of the following languages: Spanish Cantonese Korean Mandarin Russian Vietnamese May include an insert with the pre-notification letter and first survey mailing that includes a number to call to request a translation of the survey 35

36 Overview (cont'd) No supplemental questions permitted Proxy respondents are permitted Sampled beneficiaries who are unable to respond to the phone interview Permission must be received from the beneficiary 36

37 2015 Data Collection Schedule Task Date Vendors must submit QAP 8/12/2015 Vendors must complete and a Vendor Access to the PQRS Data Warehouse Form to pqrs-datasupport@rand.org 9/8/2015 Vendors must submit mail survey materials 9/18/2015 Group practices must submit Survey Vendor Authorization Form for the 2015 CAHPS for PQRS Survey administration Vendors must complete and submit a new DUA to CMS and provide a confirmation to the CAHPS for PQRS Technical Assistance (pqrscahps@hcqis.org) 9/22/2015 9/30/

38 2015 Data Collection Schedule (cont'd) Task Date Vendors provide toll-free customer support phone numbers 10/2/2015 Vendors must submit English CATI Screenshots 10/9/2015 Group practice sample files become available to vendors 10/22/2015 Mail-out pre-notification letter 11/13/ /16/2015 Open customer support toll-free line 11/16/2015 Mail-out first questionnaire 11/20/ /23/

39 2015 Data Collection Schedule (cont'd) Submit interim data Task Mail-out second questionnaire Initiate CATI follow-up (1 st attempt must occur during this time) Conduct additional CATI follow-up Date 12/9/ /11/ /15/ /16/2015 1/6/2016 1/13/2016 1/14/2016 2/3/

40 2015 Data Collection Schedule (cont'd) Task Date Complete the phone follow-up sequence 2/3/2016 Cutoff date for returned mail surveys 2/3/2016 Customer support toll-free line closed 2/3/2016 Submit final data to CMS 2/10/2016 2/12/

41 Mail Protocol Pre-notification letter Two mailings Survey cover letters Questionnaire Data receipt and processing Quality control guidelines 41

42 Mail Protocol (cont'd) Pre-notification letter Full name and address on envelopes Contains the salutation Dear Medicare Beneficiary Is dated November 16, 2015 Include survey vendor s customer service phone number o Provided to CMS in advance CMS logo in the return address section Envelope marked Return Service Requested or Address Service Requested or Change Service Requested Font equal to or larger than o Times New Roman 11 point o Arial 11 point 42

43 Mail Protocol (cont'd) Survey Cover Letters Are dated: o November 23, 2015 (1 st survey mailing) o December 16, 2015 (2 nd survey mailing) Printed on a separate sheet of paper not attached to questionnaire Salutation personalized with beneficiary name Signature of senior employee of survey vendor Survey vendor logo and return address Font size equal to or larger than o Times New Roman 11 point o Arial 11 point 43

44 Mail Protocol (cont'd) Questionnaire Full survey title must be placed at the top of page 1 Name of clinician provided in sample file printed in Question 1 Question and answer category wording must not be changed No changes in the order of the questions or answer categories About you questions must be included All instructions written at top of page 1 Return address for mail processing placed on bottom of the last page Print survey as booklet in black and white o May include a highlight color 44

45 Mail Protocol (cont'd) Questionnaire A form tracking ID linked to the Unique Respondent Finder Number must be printed on each survey o Best practice - form tracking ID linked to the Unique Respondent Finder Number printed on each page o The ID may be printed on the first and/or last page o An internal tracking barcode next to the tracking ID on the survey and other materials is acceptable Font size equal to or larger than Arial 11 point Optional Formatting o Two column format o Wide margins o Boxes or ovals are acceptable for response categories 45

46 Mail Protocol (cont'd) Mail Packet Envelope must be printed with: o Survey vendor s logo May add CMS logo o Survey vendor s return address Use of window envelopes is permissible Include a prepaid business reply envelope addressed to the same address listed on the last page of the survey 46

47 Mail Protocol (cont'd) Data Receipt and Processing Track by date of receipt Key-entry or scanning technology If beneficiary returns more than one completed questionnaire, use the first completed questionnaire received Store returned paper surveys or scanned images of paper surveys o Secure and environmentally controlled location o 3 years Ambiguous responses o Decision rules 47

48 Mail Protocol (cont'd) Decision rules for data capture If a mark falls between two choices and is obviously closer to one choice than another, select the choice to which the mark is closest Example 1 (Mail) Never x Sometimes Usually Always Code as: Sometimes 48

49 Mail Protocol (cont'd) Decision rules for data capture If a mark falls equidistant between two choices, code the value of the item as M Missing Do not impute Example 2 (Mail) Never x Sometimes Usually Always Code as: M Missing 49

50 Mail Protocol (cont'd) Decision rules for data capture If a value is missing, code it as M Missing o Do not impute Note: Dependent questions appropriately skipped should be coded as 88 Not Applicable Example 3 (Mail) Never Sometimes Usually Always Code as: M Missing 50

51 Mail Protocol (cont'd) Decision rules for data capture When more than one response choice is marked, code the value as M Missing o Do not impute Exception: Questions that have instructions to mark one or more may have multiple responses Example 4 (Mail) x x Never Sometimes Usually Always Code as: M Missing 51

52 Mail Protocol (cont'd) Decision rules for data capture When more than one response choice is marked and the beneficiary s intent is obvious, select the obvious response option Example 4 (Mail) x Never x Sometimes Usually Always Code as: Never 52

53 Mail Protocol (cont'd) The QAG quality control guidelines include, but are not limited to: Performing address validation and updates Conducting interval checking of printed mail pieces Conducting seeded mailings 53

54 Phone Protocol Phone interviews must not be conducted via inbound calls until after the phone component of survey administration begins CATI: Program with official phone script Program skip pattern questions appropriately Link electronically to survey management system Automated dialing may be used Interviewer records respondent answers electronically 54

55 Phone Protocol (cont'd) Eligible beneficiaries Did not respond to mail surveys Returned a mail survey that does not fulfill the rules defining a completed or partially completed survey No valid address available after reasonable attempts to obtain 55

56 Phone Protocol (cont'd) Where possible, CMS will provide phone numbers for beneficiaries as part of the sample file In addition, vendors must attempt to obtain phone numbers for beneficiaries using one of the following: Directly from the group practice o Via a file that contains all beneficiaries o Sample must not be shared with the group practice o Share no information with the group practice that might identify a beneficiary Commercial software Internet directories Directory assistance Other tested methods 56

57 Phone Protocol (cont'd) Survey vendors must attempt to reach every beneficiary identified for phone follow-up until the beneficiary is contacted, found ineligible or six attempts have been made No further attempts are to be made after attempting to reach the beneficiary by phone six times Definition of a phone attempt Phone rings six times with no answer Beneficiary requests call back Phone answered by someone other than beneficiary who is unavailable Busy signal for each of three consecutive attempts (made approximately at 20 minute intervals, if possible) Answering machine/privacy manager reached Disconnect/out of service 57

58 Phone Protocol (cont'd) Phone Script Standardized phone script provided by CMS Text must NOT be modified Script must be read verbatim o All questions and response choices must be read exactly as they appear in the CMS provided CATI script o Text that is underlined, bolded, highlighted, in uppercase lettering, or italicized must be emphasized 58

59 Phone Protocol (cont'd) Phone interviewer training Phone script and CATI program Guidelines for reaching beneficiaries Survey introduction Identifying possible ineligible beneficiaries Definition of phone attempts Interviewing guidelines and conventions o System conventions, e.g. CATI screens, interim disposition codes o Avoiding refusals o Probing for complete answers Customer Support FAQs 59

60 Phone Protocol (cont'd) Monitoring and Oversight 10% of all interviews through silent monitoring o Attempts and completed interviews o All interviewers o All times of day o Different days of the week Interviewers who consistently fail to follow the phone script verbatim, fail to employ proper probes, fail to remain neutral, objective and courteous, have difficulty understanding or using the computer, must be identified and retrained or replaced, if necessary Monitor subcontractors, if applicable 60

61 Languages The CAHPS for PQRS Survey will be administered in English and additional languages from the list below Spanish Cantonese Mandarin Korean Russian Vietnamese Group practices in Puerto Rico must administer the survey in Spanish and offer English if requested 61

62 Languages (cont'd) Three options for implementing data collection in a language other than English 1. Group practices provide language preference for the entire Original Medicare population to survey vendor to support language-specific survey mailings 2. Dual language survey mailings 3. Include an insert that contains instructions for the beneficiary to request a survey in the optional language Vendors must be prepared to conduct phone surveys in the same languages offered for the mail survey Use of optional languages is at the request of the group practice 62

63 Data Specifications and Coding

64 Objectives File Encryption File Specifications Decision Rules and Coding Guidelines Mail Surveys Survey Disposition Codes Survey Completion Guidelines 64

65 File Encryption Data files must be encrypted prior to data submission Survey vendors required to use PGP Use Public Key encryption Data files submitted by survey vendors that are not encrypted will be rejected and must be resubmitted 65

66 File Specifications Survey vendors must use flat ASCII file format to submit survey data files Survey vendors will submit all group practice s sampled beneficiary records in one file No substitutions for valid data elements are acceptable 66

67 File Specifications (cont'd) Survey data will contain one record for each sampled beneficiary Each record will consist of two parts: Survey Status Section (found in Appendix J) Beneficiary Survey Data Section 67

68 File Specifications (cont'd) Survey File Record Layout - Survey Status Section: RAND Field Name Field Contents Starting Position in Record Field Length Valid Codes FINDER Unique Beneficiary Finder Number Assigned by CAHPS for PQRS Data Coordination Team PQRSGROUP_ID Five character group practice identifier: begins with the letter G, followed by 4 numbers DISPOSITN Final Disposition Code , 31, 11, 20, 22, 24, 32, 33, 34, 35, Numeric From sample file 9 5 [G]nnnn From sample file Coding Notes 10 = Completed survey 31 = Partially completed survey 11 = Institutionalized 20 = Deceased 22 = Language barrier 24 = Mentally or physically unable to respond 32 = Refusal 33 = Non-response when there is no indication of bad address or telephone number 34 = Blank survey or Incomplete survey returned 35 = Bad address and/or bad telephone number 40 = Excluded from survey 68

69 File Specifications (cont'd) Survey File Record Layout - Survey Status Section (cont d): RAND Field Name Field Contents Starting Position in Record Field Length Valid Codes Coding Notes MODE Survey Completion Mode , 8 1 = Mail; 2 = Inbound CATI; 3 = Outbound CATI; 8 = Not applicable DISPO_LANG Survey Language , 8 Language survey was completed in: 1 = English; 2 = Spanish; 3 = Cantonese; 4 = Korean; 5 = Mandarin; 6 = Russian; 7= Vietnamese; 8 = Not applicable RECEIVED Date survey was received or completed: YYYYMMDD FOCALTYPE Provider type: 1= Primary care, 2 = Specialist PRTITLE Type of provider (physician, physician assistant, nurse practitioner, certified nurse midwife, certified clinical nurse specialist) 18 8 yyyymmdd Date survey was received: YYYYMMDD, = Not applicable 26 1 Numeric From sample file Text From sample file PRFNAME Provider first name Text From sample file PRLNAME Provider last name Text From sample file 69

70 File Specifications (cont'd) Each field in the Survey Status Section requires an entry for a valid submission Use code 8 Not Applicable if appropriate Survey Language for a blank mail survey Survey Completion Mode for a mail survey that was not returned AND no phone number was obtained 70

71 File Specifications (cont'd) Beneficiary Survey Data Section Contains survey responses from every beneficiary who has a final disposition of Completed (10), Partially completed (31) or Blank or Incomplete survey returned (34) o Leave survey data section blank for all other dispositions 71

72 File Specifications (cont'd) Beneficiary Survey Data Section (cont d) For survey records included, all response fields must have a valid value Valid values can include: o 88 - Not Applicable o 98 - Don t Know o 99 - Refused o M - Missing 72

73 Decision Rules and Coding Guidelines Mail Surveys Except for questions where a respondent can select more than one response option, such as the race or proxy question (if applicable), when more than one response option is marked, code as M Missing 73

74 Decision Rules and Coding Guidelines Mail Surveys (cont'd) Decision rules for screener and dependent questions Some items can and should be skipped by certain beneficiaries Dependent questions that are appropriately skipped should be coded as 88 Not Applicable Screener questions that are left blank should be coded as M Missing 74

75 Decision Rules and Coding Guidelines Mail Surveys (cont'd) Decision rules for screener and dependent questions If respondent made an error in the skip pattern, survey vendors must not clean or correct skip pattern errors o Enter the value provided by beneficiary Do not impute a response based on beneficiary s answers to dependent questions An error in the skip pattern will occur if a respondent left a screener question missing and then skipped subsequent dependent questions 75

76 Decision Rules and Coding Guidelines Mail Surveys (cont'd) Dependent questions answered in violation of skip patterns are not counted toward the number of applicable to all (ATA) or SSM items to determine a complete or partially complete survey A screener question left blank does not trigger a skip, so subsequent responses to dependent questions should be included in count of answered survey items 76

77 Decision Rules and Coding Guidelines Mail Surveys (cont'd) Decision rules for beneficiary survey data section Enter all survey responses provided by the beneficiary for each survey item For Completed (10), Partially completed (31) or Blank or Incomplete survey returned (34) surveys, code missing answers as M Missing or 88 Not Applicable o All survey questions must have a valid code 77

78 Survey Disposition Codes Survey disposition codes are used to track and report whether a beneficiary has completed a questionnaire or requires follow-up Survey vendors are required to assign and maintain up-to-date survey disposition codes for each beneficiary in the sample 78

79 Survey Disposition Codes (cont'd) Vendor s interim disposition codes are for internal purposes only and should not be reported Only final disposition codes are reported Submitted data files must contain a final disposition code for each beneficiary in the file 79

80 Survey Disposition Codes (cont'd) Final Disposition Code Description Criteria Completed survey 10 A completed survey includes a response to at least one question in the 12 SSMs and >50% of the ATA items Partially completed survey 31 A partially completed survey includes a response to at least one question from the 12 SSMs and <50% of the ATA items A completed survey includes a response for at least one question from the 12 SSMs and greater than or equal to 50% of the ATA items. Appropriately skipped questions don t count against the required 50 percent. There must be no evidence that the beneficiary is ineligible. A partially completed survey includes a response to at least one question from the 12 SSMs and less than 50% of the ATA items. There must be no evidence that the beneficiary is ineligible. Institutionalized 11 Institutionalized Institutionalized or residing in a group home or institution (hospice, nursing home, etc.). Deceased 20 Deceased Deceased at the time of survey administration. Language barrier 22 Unable to complete the survey in English and any offered optional language Unable to complete the survey in English and any offered optional language. 80

81 Survey Disposition Codes (cont'd) Final Disposition Code Description Criteria Mentally or physically unable to respond 24 Mentally or physically unable to respond to either mail or phone portion of the survey Mentally or physically unable to respond either to mail or phone portion of the survey. Refusal 32 Refused to complete the survey Refused to complete the survey. Non-response 33 No response collected No response collected either by mail or by phone when there is no indication of bad address or bad phone number. Blank or Incomplete survey returned 34 Responded by mail or initiated CATI interview, no answers to any question from the 12 SSMs Responded by mail or CATI, with no answers to any question from the 12 SSMs. There must be no evidence that the beneficiary is ineligible. Bad address and/or Bad phone number 35 Unable to obtain a viable address or phone number for the beneficiary Excluded from survey 40 Was excluded from all survey processes Unable to obtain a viable address and/or phone number. Beneficiary was determined to be ineligible after sample selection but before data collection was initiated (see sampling section of manual). 81

82 Survey Data Section Required Following codes require submission of survey data section: Completed Survey (Code 10) Partially Completed Survey (Code 31) Blank or Incomplete Survey returned (Code 34) 82

83 Survey Data Section Required (cont'd) Completed survey (Code 10): A response to at least one question from the 12 SSMs A response to at least 50% of the ATA items Questions answered in violation of a skip pattern do not count as a response 83

84 Survey Data Section Required (cont'd) Partially completed survey (Code 31): A response for at least one question from the 12 SSMs A response to fewer than 50% of the ATA items Questions answered in violation of a skip pattern do not count as a response 84

85 Survey Data Section Required (cont'd) Blank or Incomplete survey returned (Code 34): No responses to any question from the 12 SSMs Questions answered in violation of a skip pattern do not count as a response 85

86 Survey Data Section Not Required Final survey disposition codes that do not require submission DISPOSITION of survey status section CODE Non-response: No response collected 33 Bad address and/or Bad phone number: Unable to obtain a viable address and/or phone number 35 Excluded from survey: Beneficiary excluded from survey processes 40 Institutionalized 11 Deceased 20 Language Barrier: Unable to complete survey in English and any offered translation Mentally or physically unable to respond 24 Refusal: Refused to complete survey

87 Break

88 Data Preparation and Submission

89 Objectives Data Warehouse Access Data Submission Processes Data Submission Deadlines Data File Submission File Encryption Overview of Data Warehouse Guide to Data Submission Process Data Auditing and Validation Checks Data Submission Notification 89

90 Data Warehouse Access Survey vendors must: Submit a Vendor Access to PQRS Data Warehouse Form by 9/8/2015 Designate a Data Administrator, Back-up Data Administrator and a Project Manager 90

91 Data Submission Processes CAHPS for PQRS Survey Project Team has developed a secure PQRS Data Warehouse using a web-based application hosted by the RAND Corporation Will operate as a secure file transfer system that Survey Vendors will use to pick up sample files and submit survey data Does not require special software or licensing fees for survey vendors with the exception of PGP 91

92 Data Submission Deadlines Interim survey data file must be submitted by survey vendors no later than 12/11/2015 at 11:59 p.m. Eastern Time Survey vendors must submit files early to allow them enough time to resubmit if they have to and still meet the deadline If vendors submit more than once, files must include all records in the re-submission Final survey data file must be submitted by survey vendors by 2/12/2016 at 11:59 p.m. Eastern Time It is the responsibility of the survey vendor to ensure that data are submitted on time 92

93 Data File Submission Survey vendors must use the following file naming convention: Vendorname.submissionN.mmddyy.txt.pgp N = number of the submission sent in that day mm = number of month of submission (justify leading zero) dd = day of the month of submission (justify leading zero) yy = 2 digit year of submission Example: XYZResearch.submission txt.pgp 93

94 File Encryption Data files from survey vendors must be encrypted using PGP software prior to submitting them to PQRS Data Warehouse All prior versions of PGP acceptable If necessary, install latest version Symantec File Share Encryption 94

95 File Encryption (cont'd) Survey vendors must create a Public Key that the CAHPS for PQRS Survey Data Coordination Team will use to encrypt vendor sample files CAHPS for PQRS Survey Data Coordination Team will provide the survey vendors with a Public Key to receive survey data files Public Keys will be exchanged using the vendor s folder in the Data Warehouse 95

96 Overview of Data Warehouse Available via the Internet Hosted on RAND Corporation s Website Survey vendor s folder will contain controls for submitting survey data files as well as for downloading sample file and/or other project documentation 96

97 Overview of Data Warehouse (cont'd) Submitted data files that are not encrypted (don t have.pgp extension) or don t otherwise comply with the established naming standards are deleted After each data submission, survey vendor staff receive letting them know that the file was/was not successfully submitted 97

98 Guide to Data Submission Process Once you have completed the Vendor Access to PQRS Data Warehouse Form, you will receive an from a member of The RAND Corporation Data Team with an invitation to the PQRS Data Warehouse Step-1 Click on the CAHPS for PQRS Secure File Sharing link. You will be directed to the Data Warehouse login page. 98

99 Guide to Data Submission Process (cont'd) Step-2: Enter your address and temporary password from your invitation mail Step-3: Click the Login button 99

100 Guide to Data Submission Process (cont'd) Step-4: The first time you login, you will be prompted to choose a new password 100

101 Guide to Data Submission Process (cont'd) Step-5: Re-enter your temporary password in the Verify Current Password box Enter your new password in both the Change Password To and Re-type New Password boxes Click Update Password. You will see the confirmation screen. 101

102 Guide to Data Submission Process (cont'd) Step-6: Click OK, you will be transferred to the Warehouse (File Manager tab) from where you can access your secure folder 102

103 Guide to Data Submission Process (cont'd) Step-7: Click your folder name to open the folder and enable action buttons 103

104 Guide to Data Submission Process (cont'd) Step-8: To send a file to your workspace within the PQRS Data Warehouse, click the Add File button to start the Add Files dialog 104

105 Guide to Data Submission Process (cont'd) Step-9: Click Choose File, navigate to the folder on your local system where your file is located Step-10: Select the file, then click Open 105

106 Guide to Data Submission Process (cont'd) Step-11: The file name will appear in the Add Files window Step-12: Click Add to submit the file to the Secure Workspace of the PQRS Data Warehouse. To remove the file without submitting, click the red X. 106

107 Guide to Data Submission Process (cont'd) Step-13: During the secure transfer you will see a progress bar 107

108 Guide to Data Submission Process (cont'd) Step-14: When the upload is complete, the file will be listed in your secure workspace folder 108

109 Data Auditing and Validation Checks CAHPS for PQRS Survey Data Coordination Team will audit data files as they are submitted for compliance with file layout specifications File audit includes: Checking for.pgp file extension Logical record lengths, appropriate character set, naming conventions Presence of required data fields Range checks Verification of coding of Survey Disposition Code 109

110 Data Submission Notification Survey vendor (Data Administrator, Back-up Data Administrator and Project Manager) will receive two notifications for each data submission First (automated) , from contains notification that file was received Second , from the CAHPS for PQRS Survey Data Coordination Team, is sent after audit checks Indicates if file successfully passed checks Will go out no later than 8:00 p.m. Eastern Time one business day after submission 110

111 Data Submission Notification (cont'd) If file fails any audit checks, will: Instruct survey vendors that they must submit data files again Contain full detail of the audit check report including a list of involved records If file passes checks, will say that no further action is necessary, and provide a summary of file contents for verification by the vendor 111

112 Technical Support Contact the CAHPS for PQRS Survey Data Coordination Team for technical support and/or assistance related to data submission at: 112

113 Data Analysis and Reporting

114 Objectives Value-Based Payment Modifier How Data are Reported to Group Practices Public Reporting of PQRS CAHPS Data Data Analyses Conducted by CMS Data Analyses Conducted by Survey Vendors 114

115 What is the Value-Based Payment Modifier (VM)? The VM assesses both the quality of care furnished and the cost of that care during a performance period The VM is an adjustment made on a per-claim basis to Medicare payments for items and services furnished under the Medicare Physician Fee Schedule (PFS) High quality and/or low cost groups and solo practitioners can qualify for upward adjustments Low quality and/or high cost groups and solo practitioners, including those that fail to satisfactorily report under the Physician Quality Reporting System (PQRS) are subject to downward adjustments The VM is applied at the Taxpayer Identification Number (TIN) level and applies to all physicians and non-physician EPs billing under the TIN who are subject to the VM during the payment year 115

116 What is the Value-Based Payment Modifier (VM)? (cont'd) The following table outlines how and when the VM will be applied: Performance Period VM Payment Adjustment Period VM Applied To Physicians in groups with 100 or more EPs Physicians in groups with 10 or more EPs Physician solo practitioners and physicians in groups with 2 or more EPs; VM for physicians participating in the Medicare Shared Savings Program will be based on the ACO s quality data and average cost; Proposed to waive the VM for physicians participating in the Pioneer ACO Model, Comprehensive Primary Care Initiative, or other similar Innovation Center models during the performance period Physicians, PAs, NPs, CNSs, and CRNAs who are solo practitioners or in groups with 2 or more EPs 116

117 CAHPS Contribution to the Value Modifier The quality of care composite score determines one-half of the VM calculation The CAHPS for PQRS Survey falls into one of six domains that contribute to the quality of care composite score 11 of the 12 SSMs contribute to the Person and Caregiver- Centered Experience and Outcomes domain in the quality of care composite Health Status and Functional Status is not included Groups with 2+ Eligible Professionals (EPs) may elect to have CAHPS for PQRS included in their VM quality composite calculation. 117

118 CAHPS Contribution to the Value Modifier (cont d) 1. Person and Caregiver-Centered Experience and outcomes 2. Effective Clinical Care 3. Community/Population Health 4. Patient Safety Quality of Care Composite Score 5. Communication & Care Coordination 6. Efficiency & Cost Reduction Total per capita costs (plus MSPB) Total per capita costs for beneficiaries with specific conditions Cost Composite Score Value Modifier Amount 118

119 Calculating CAHPS Scores for the Value Modifier The contribution of each SSM to the VM is based on the difference between the performance rate and the prior-year benchmark The benchmark for each SSM is the weighted average from the calendar year prior to the measurement year on a metric across practices To compare each practice to the benchmark, the weighted SSM scores are converted to a standardized score (performance rate benchmark) / weighted standard deviation The Person and Caregiver-Centered Experience and Outcomes Domain score for each practice is the unweighted average of the standardized scores for all measure in the domain, including SSMs with 20 or more cases Performance is evaluated based on statistical significance and the magnitude of the difference between a practice s score and the benchmark 119

120 How Data are Reported to Group Practices Every group practice receives a detailed report from CMS SSMs o Reported as mean scores Multi-question measures Individual questions Response rate Shows data for each group practice as well as: Means of SSMs for all group practices 120

121 Public Reporting of CAHPS for PQRS Data The 11 of the 12 CAHPS for PQRS SSMs will be reported on Physician Compare for groups of 100 or more EPs who participate in PQRS GPRO and for groups of 25 to 99 EPs reporting via a certified CAHPS vendor 2014 data will be publicly reported in late 2015 Reports will present top box scores, which are the percentage of patients with the most positive experiences For the 0-10 Provider Rating, the top box is a 9 or 10 rating For all other question types, the top box is the most positive response (for example, Always for reports of experience, Excellent for the rating of physical health) Scores with low and very low reliability will not be publicly reported 121

122 Data Analyses Conducted by CMS In addition to scoring and benchmarking, CMS conducts the following analyses Reliability Reliability is assessed for each question and SSM Case-Mix Adjustment and Weighting Ensures a level playing field and that data represent group practices fairly 122

123 Data Analyses Conducted by Survey Vendors Survey Vendors may conduct own analyses of data for quality improvement purposes, however, per your CMS DUA: Cell sizes must not be less than 11 o No information based on fewer than 11 sampled members can be released, meaning no cell sizes under 11 can be displayed in any cross tabulations, frequency distributions, tables, Excel files, or other reporting mechanisms o No number smaller than 11 should appear in any material provided to your client For example, if a certain response option is chosen fewer than 11 times, data for that response option must not be displayed, even if 11 or more responses were received for the corresponding question as a whole Intervention or follow-up with any sample member is not permitted Survey vendors cannot provide individual-level data to group practices 123

124 Data Analyses Conducted by Survey Vendors (cont'd) CMS-calculated results are official results Vendors will not have sufficient information to replicate CMS analyses All reports provided to the group practices must include a statement that vendor results are unofficial and are for practices internal quality improvement purposes only CMS-calculated results include data from completed and partially completed surveys 124

125 Vendor Oversight

126 Objectives Oversight Activities Non-compliance and Sanctions Discrepancy Reports 126

127 Oversight Activities Ensure: Compliance with CAHPS for PQRS Survey protocols Survey data collected and submitted are complete, valid and timely Standardization and transparency of CAHPS for PQRS Survey results Data Security 127

128 Oversight Activities (cont'd) Oversight activities include: Review of Quality Assurance Plan (QAP) and survey materials Conduct site visits and conference calls Analysis of submitted data 128

129 Oversight Activities (cont'd) QAP Documents understanding, application and compliance with survey protocols Follows the Model QAP specifications Provides a guide for the site visit Vendor submits each year Submit via the CAHPS for PQRS Survey Technical Assistance pqrscahps@hcqis.org Submit by 08/12/

130 Oversight Activities (cont'd) Review of survey materials Review for compliance with CAHPS for PQRS Survey protocols and guidelines Submitted each year of survey administration Only survey vendors with a contracted CAHPS for PQRS client(s) need to submit survey materials Submit via the CAHPS for PQRS Survey Technical Assistance pqrscahps@hcqis.org English Mail materials due 9/18/2015 English CATI screenshots due 10/9/

131 Oversight Activities (cont'd) Site visits and conference calls Review and observe systems, procedures, facilities, and resources Discussions with project staff o Including subcontractors, if applicable All materials related to survey administration are subject to review Feedback report includes action items for follow-up Conference calls as needed 131

132 Oversight Activities (cont'd) Analysis of submitted data Intended to detect errors in data submission Includes review of outliers, anomalies, unusual patterns, etc. Follow-up as appropriate 132

133 Non-compliance and Sanctions If survey vendors fail to adhere to CAHPS for PQRS Survey protocols, including missing deadlines/due dates, they will be required to develop and implement corrective actions If survey vendors do not fix persistent problems, they may lose approved status for conducting the CAHPS for PQRS Survey Other sanctions may also be applied 133

134 Discrepancy Reports Report any variations from CAHPS for PQRS Survey protocols during survey administration Complete and submit online report within 1 business day after discovery of issue at A second updated report may be submitted once root cause, scope of issue and/or corrective action has been identified Vendors must not wait until the discrepancy has been resolved to submit an initial Discrepancy Report 134

135 Discrepancy Reports (cont d) Required Discrepancy Report detail includes Description of discrepancy, how and when it was discovered All affected group practice names and ID numbers impacted by the discrepancy For each group practice listed: o Affected timeframe o Group practice ID number o Count of sample members affected by the discrepancy Description of corrective action to be taken along with proposed timeline Provide as much information as possible in initial report File updated Discrepancy Report with any additional information 135

136 Discrepancy Reports (cont d) CMS Review Process Acknowledgment of receipt Assessment of actual or potential impact on data Additional information may be requested Will notify vendor of review outcome 136

137 Questions?

138 Wrap Up and Next Steps Important Dates 8/12/2015 QAP due to CAHPS for PQRS Survey Project Team via Technical Assistance 9/8/2015 Vendor Access to the PQRS Data Warehouse Form due o Survey vendors must complete and the Vendor Access to PQRS Data Warehouse Form to pqrs-datasupport@rand.org 9/18/2015 English Mail survey materials due 9/22/2015 Survey Vendor Authorization Forms due to the CAHPS for PQRS Survey Data Coordination Team: o May be submitted via US Mail, Federal Express, UPS, or other expedited mail service with tracking information o See Appendix B for address information 9/30/2015 Submit DUA to CMS 10/9/2015 English CATI screenshots due 138

139 Wrap Up and Next Steps (cont'd) See 2015 Data Collection Schedule for key survey administration dates Post Training Survey Vendor Quiz Immediately upon conclusion of training Accessible via webinar for 20 minutes Vendor Notification CMS follow-up regarding quiz by 7/24/2015 Feedback on Training Follows quiz Accessible via webinar for 20 minutes 139

140 Contact Us CAHPS for PQRS Survey Information and Technical Assistance Website: Phone: Toll free

141 Post Training Activities Quiz 20 minutes Post Training Evaluation Form 20 minutes 141