Provider Satisfaction Survey: Research and Best Practices

Transcription

1 Provider Satisfaction Survey: Research and Best Practices Provider Satisfaction Survey Revamp Provider Satisfaction with the health plan has been one of the most popular proprietary survey tools that TMG clients have sought over the past 10+ years. Given the changes in the structural and organizational landscape within the delivery of care market and the challenges broadly experienced when surveying the provider population, we launched a research initiative to review our existing tool, content, question order, length, and response scale. We also addressed willingness to complete the survey, barriers to completing the survey, and opportunity to enhance the value of findings for our clients. Findings We first conducted a focus group with physicians to gather feedback on TMG s original Provider Satisfaction survey. 1 Here s what they said: Physicians want to see the survey results and understand what their peers are saying. They want to be compensated for the time that it takes to complete the survey. Knowing what actions the plan would take as a result of the survey findings was a common theme. Physicians want to know that if they take the time to give feedback, the health plan is listening and taking action. E mail methodology allows physicians more flexibility in completing the survey. Having it conveniently located in their inboxes allows them to complete it at home in their spare time. With regard to the survey questions on the original survey instrument: It is difficult to rate all other plans as a group on specific key elements. Most plans performance might be very good, but then there might be an outlier with poor performance, and it is difficult for a respondent to mentally average that out and provide one response. Physicians reviewed each question and offered suggestions for improvement in the wording and the response scale. Physicians also suggested new topics to focus on or commented that certain topics were not of importance to them. 1 For background information on the original survey, please see page 4 of this document.

2 Provider Satisfaction Survey Research & Best Practices We also contacted several office managers in different practice settings (primary care and specialty, large and small groups) to seek their feedback on the survey and methodology. Office managers preferred to take the survey via e mail or over the phone. Similar to providers, office managers recommended the use of incentives to compensate for their time and for providing their opinions. In addition to monetary compensation, some office managers cited on site visits or assistance from the health plan as a good incentive to take the survey. While office managers did not mention seeing the survey results as a main focus, they did want to know what actions would be taken based on their feedback to the plan. Office managers reviewed each question and provided feedback on the relevance of the question content, wording, and response options. The office managers shared the same difficulty in rating all other plans that the physicians expressed during the focus group. added new items based on topics that office managers deemed important to them that were not covered on the original survey. As a result of this feedback, TMG has revised the Provider Satisfaction survey template, completed a pilot test with several health plans, and developed the following best practice recommendations. Best Practices Methodology and Response Rates recommends a mixed methodology using e mail, mail, and phone. Our research found that physicians prefer to complete the survey via e mail. Below are a few other best practices we have put together related to response rates: Use an incentive. Physicians expect to be compensated for their time. Incorporate information from the prior year s survey and any improvement initiatives the plan has put in place into the cover letter. Physicians want to know that you are doing something with their feedback. Include providers who see a high volume of your members in your sample. Our research found that providers with more of a stake in your company will be more likely to respond. 2

3 Provider Satisfaction Survey Research & Best Practices Plans may conduct the Provider Satisfaction survey annually or on a more frequent basis to monitor and track quality improvement. If you do not have e mail addresses for your providers, offers a Provider Verification program that can help collect that information, along with other data fields that you need to ensure your provider directories are up to date. This may also help you with NCQA standard RR 4, Element C (Physician Information Validation). Sampling will work with you to determine a sampling strategy that best suits your needs. However, the following are some of the things that we recommend you consider: Start with a sample of at least 1,500 providers. Stratify the sample to ensure an appropriate amount of PCPs, Specialists, and Behavioral Health Clinicians (for example, 60% PCPs, 30% Specialists, and 10% Behavioral Health Clinicians). Consider patient volume for each physician to ensure you re targeting high volume physicians first. Cap the number of physicians at each office who will receive a survey. Sending too many surveys to one location sometimes has an adverse effect on response rates. Choose your specialty types wisely. We see that certain specialty types (for example, Dentists) do not always respond at rates equal to other specialty types. Also, ensure that the specialty types you are surveying are likely to see patients on a regular basis. For more details on our Provider Satisfaction research initiative, please see the following page. 3

4 Provider Satisfaction Survey Research & Best Practices Overview An In Depth Review of Our Research had several objectives in conducting research on successfully measuring provider satisfaction, many of which were focused around the survey instrument and methodology. Our objectives included the following: Confirm the relevance of the survey topics Identify opportunities to improve response rates Determine the preferred method of completing the survey Provide more insightful information to our clients Shorten the length of the survey instrument Improve the flow and ease of completing the survey s Original Provider Satisfaction Survey and Methodology The following information describes the original survey instrument and recommended methodology to put our research into perspective. Survey Instrument s original survey instrument was developed several years ago, and addressed the following topics: Call Center/Member Services Staff Provider Relations Utilization and Quality Management Finance Issues Continuity/Coordination of Care Network Pharmacy Overall Satisfaction and Loyalty Demographics The survey contained six demographic questions, 64 total scaled questions (addressing 35 distinct aspects of satisfaction), and two open ended questions where respondents could leave comments. For benchmarking purposes to provide a comparison to All other plans, 29 of the 35 distinct aspects of satisfaction were measured for both the sponsor plan and all other plans with which the provider worked, which significantly lengthened the survey. 4

5 Provider Satisfaction Survey Research & Best Practices Question scales from the original survey are described below: For most items, providers were asked to rate the plan and all other plans for several specific aspects of satisfaction on a scale of Excellent, Very good, Good, Fair, and Poor. Two questions asked providers if they would recommend the sponsor plan to other physicians and to other patients on a scale of Definitely yes, Probably yes, Probably not, and Definitely not. Two overall satisfaction questions asked about satisfaction with the sponsor plan and all other plans on a scale of Very satisfied, Somewhat satisfied, Neither satisfied nor dissatisfied, Somewhat dissatisfied, and Very dissatisfied. Survey Methodology The standard survey methodology involved sending a paper survey and cover letter to a sample of physicians through U.S. mail, followed by a replacement survey to physicians who did not respond to the first wave. A third wave included phone follow up to the office managers of physicians who did not respond. Typically, each office is included only once in the phone sample based on having a unique phone number so that respondents are not overburdened with phone calls for each non responding physician at the office. Some plans also opted to use a web survey through one of two methods: 1) Providing the web address for the survey in the mailed cover letter or 2) E mailing the survey to the physician using the physician s e mail address from the database. We typically find that e mail is a much more successful methodology than providing a URL in the mailed cover letter since the physician does not have to type in the URL if it has been e mailed directly to his/her e mail account. Through this mixed methodology, we typically obtain responses from physicians, office managers, nurses, and other office staff, and the type of respondent is identified through one of the survey questions. Cover Letter Our original cover letter was fairly generic, stating the purpose of the survey and describing the logistics for completing and returning the survey. There was some language in the original cover letter indicating that the survey results would help the plan provide quality service to the physician and his/her staff, but no specific initiatives or areas of focus would typically be included. Physician Focus Group and Office Manager Interviews We recruited physicians and office managers to review the original survey and discuss our objectives using a discussion guide that included the following exploratory areas: The idea of providing feedback to health plans (e.g., whether it s worthwhile, awareness of NCQA requirements, how providing feedback helps them, and thoughts on what health plans do with the feedback) Frequency of receiving surveys and length of time required to complete the surveys Best way for health plans to seek feedback from providers Nature and frequency of provider interactions with health plans (e.g., what topics are discussed, who they speak with, how often the physicians contact the plan vs. how often the office managers contact the plan and 5

6 Provider Satisfaction Survey Research & Best Practices for what reasons) Survey deep dive to explore which survey topics are most important and why, the relevance of all topics and survey questions, any topics that should be added to the survey, and any questions or response options where the wording should be changed Specific exploration of all other plans questions to find out what providers think of when they are asked to rate all other plans with which they contract Best way to complete the survey (mail, phone, Internet) and why, ideal survey length, and willingness to provide plans with e mail addresses Barriers to completing the survey and reasons why providers might be more willing to complete a survey from a specific health plan Competitive probing to find out what other types of surveys providers receive and what those surveys look like These topics are discussed below in more detail, along with the survey upgrade details to make the connection between the research and the results. Providing Feedback to Health Plans When we asked physicians their thoughts on providing feedback to health insurance plans, there was considerable skepticism as to whether health plans actually care about what the physicians have to say. One physician noted that maybe the health plans are worried that the physicians won t want to contract with them anymore due to reimbursement rates or other issues, and that s why they conduct physician satisfaction surveys. Several of the respondents indicated that most communication from the health insurance companies has been in the form of a nudge to complete a preventive screening to avoid a red mark on your profile. One physician said that he has some mistrust of the health plans and feels like the health plans scrutinize what the physicians are doing, even though certain issues should not be considered his fault or within his realm of responsibility. Another respondent was more optimistic, saying that seeking their feedback is a good thing and that she hopes they are seeking feedback to increase efficiency and make any needed changes. A subject that came up several times throughout our research was that physicians and office managers both want to know that if they are taking the time to complete the survey, something will be done with their feedback. Physicians mentioned wanting to see the actual results of the survey and to know what improvements would be made based on the results. Because of this emphasis, has new recommended verbiage for the cover letter that accompanies the survey instrument in the mail or via e mail. We are recommending the following: Cite scores from the prior year or areas where you could improve Provide examples of improvement initiatives that are under way as a result of the prior year s survey Tell the physicians that you will publish some of the key findings from the current survey and let them know where to look for them (e.g., the April issue of the Provider Newsletter ) Ensure them that you are listening and that what they say counts 6

7 Provider Satisfaction Survey Research & Best Practices Typical Physician and Office Manager Interactions with the Health Plan Some of the survey topics that we reviewed with the physicians were topics that the physicians don t deal with on a day to day basis, but they do get involved when an issue is escalated to them from their staff members (for example, writing letters to the health plan to prove a patient needs a service or treatment). Office managers had similar feelings that they did not deal with all of the topics on the survey. However, the physicians still thought that it was very important to get feedback from both the physicians and the office managers because each has valuable feedback to provide. We asked physicians and office managers to rank the topics on the survey from most important to least important. Based on that feedback, we reordered the survey topics to put the most important ones first and engage the respondents. There were some differences between what was important to physicians vs. office managers, and in those cases, we chose the physician topics to put before the office manager topics. In addition to physician and office manager respondents, we typically see that nurses and other staff respond to the survey as well. This information is gathered through one of the survey questions. As we typically find that different types of respondents answer the survey differently, we provide segmentation in the final reporting to show responses separately by respondent type. Survey Length and Incentives Physicians said that a 5 or 10 minute survey is reasonable for them to complete. A 15 minute survey would be a stretch, but one respondent said she would do it if it is going to impact her patients or if there was an incentive. One interesting suggestion from the focus group was that renumbering the survey to make it appear shorter could prompt more people to fill out the survey. As a result of this comment, renumbered the survey so that it appears to have 9 short series of questions rather than upwards of 40 total questions. For longer surveys (approximately 30 minutes or longer), physicians tend to receive incentives upwards of $300. As another comparison, one physician mentioned that she receives a $100 reimbursement for spending 15 minutes with a patient. Another interesting finding was that physicians will take surveys via the web if the incentive is receiving information on an interesting topic (no financial incentive). Office managers mentioned financial incentives as well as an incentive in the form of help with tasks around the office. Incentives have been shown to increase response rates. Aside from what we learned from the physicians and office managers, we have had clients use incentives for Provider Satisfaction studies in the past. We have noted that the best success with incentives occurs when each respondent receives an incentive rather than the promise of a drawing where each respondent might or might not receive the reward. Methodology E mail was the most popular methodology mentioned from both physicians and office managers that we talked to. The e mail methodology seemed to provide the most convenience for respondents. Physicians also indicated that they would be willing to provide their e mail addresses to plans so that the plans can conduct the survey via e mail. 7

8 Provider Satisfaction Survey Research & Best Practices Rating All Other Plans One of the most significant changes to the survey was the removal of the separate items that asked the providers to rate All other plans for each service area. Physicians that we talked to indicated that it would just be easier to complete the survey about the sponsor plan and not have to think about all other plans. The all other plans questions seemed to be a big burden for both physicians and office managers because it was very difficult for them to group all of the other plans that they work with into one single rating for each service area. The performance between separate plans could be so disparate that it was difficult for respondents to average out all of those ratings in their heads to be able to answer the questions. To add value to the survey, we allow plans to ask about providers overall satisfaction with the health plan s top four competitors within the overall satisfaction section of the survey. This puts less of a burden on the respondents and still allows the sponsor plan to measure satisfaction with the competition. Factors Affecting Survey Completion With all of the paperwork that goes into the physicians offices from the health plans, some of it gets overlooked or set aside. Physicians indicated that they were more likely to fill out the survey if the sponsor plan represents a large portion of their patients. When asked about barriers to completing the survey, time was a big one mentioned by the physicians and by the office managers when talking about the physicians. Additional barriers include a desire for a financial incentive and skepticism about the health plan making changes based on the feedback. Questions and Response Scales The physicians and office managers had considerable feedback on the questions and the response scales. As a result of the feedback related to specific questions, we removed several questions, added some, and changed the spirit of a few questions to be more in line with what was relevant to the respondents. One physician focused in on the response scale that we originally had, and said the following: I was looking at this excellent, very good, good, fair, poor. Anything excellent, very good, good, even fair those are all in favor. To me, I m looking at one, two, three, four decent [responses] and one bad [response]. The physicians discussed this further and went back and forth about different scale options, but finally settled on recommending one with Average in the middle. As a result, changed the response options throughout the survey to have a true midpoint of Average. Another benefit to this new scale is that it provides a comparison to other plans without having to ask the questions separately about all other plans with which the provider contracts. 8

9 Provider Satisfaction Survey Research & Best Practices s Upgraded Provider Satisfaction Survey and Methodology The following information describes the upgraded survey instrument and recommended methodology based on the research that we have conducted. Survey Instrument s upgraded survey instrument now addressed the following topics: Demographics Comparative Rating Finance Issues Utilization and Quality Management Network/Coordination of Care Pharmacy Health Plan Call Center Services Staff Provider Relations Overall Satisfaction The survey contains seven demographic questions, 33 scaled questions, and one open ended question where respondents can leave comments. Based on our research, we were able to cut the survey length nearly in half. Question scales from the original survey are described below: For most items, providers are asked to rate the plan as compared to all other plans for several specific aspects of satisfaction on a scale of Well below average, Somewhat below average, Average, Somewhat above average, and Well above average. This scale now has a true midpoint and allows for automatic comparisons to other plans without adding more questions to the survey. One loyalty question asks providers if they would recommend the plan to other physicians on a scale of Yes or No. This scale was altered due to physician feedback that you either would or you would not recommend the plan. Additionally, the second loyalty question was removed because physicians said that they don t typically recommend health plans to their patients. Five overall satisfaction questions ask about satisfaction with the sponsor plan and the plan s top four competitors separately on a scale of Completely dissatisfied, Somewhat dissatisfied, Neither dissatisfied nor satisfied, Somewhat satisfied, and Completely dissatisfied. Survey Methodology Though we will still use a mixed mail and phone methodology for this survey to reach as many physicians and office managers as possible, we are now putting additional emphasis on encouraging and using an e mail methodology since that is the physician s preferred methodology. If plans do not have physician e mail addresses, offers a Provider Verification Program to reach out to physicians and get updated contact information. 9

10 Provider Satisfaction Survey Research & Best Practices Cover Letter Our cover letter now emphasizes the data and findings from the prior year s survey, any quality improvement initiatives that are occurring or have already been completed to demonstrate how the health plan uses the feedback physicians provide, along with information on how the physicians can access the current survey s results once data collection has ended. Pilot Testing and Validation conducted a pilot test of the new survey with eight health plans to ensure reliability. The pilot test resulted in a dataset that included 1,524 respondents representing Primary Care Physicians, Specialists, and Behavioral Health Clinicians. Through extensive analysis performed on the pilot study dataset, we are confident that the new survey will provide a reliable measurement of provider satisfaction. Our Analytics team conducted the following statistical analyses on the pilot survey data, and the results of each analysis follow: Reliability Analysis Factor Analysis Regression Analysis Reliability Analysis Cronbach s alpha is the most commonly used estimate of reliability of questions in a survey. The alpha coefficient ranges in value from 0 to 1 and measures the extent to which responses acquired at the same time correlate with each other. The higher the score, the more reliable the generated scale. The customary benchmark value is an alpha of or higher. We found the benchmark composites to provide a reliable measure of provider satisfaction with the health plan. The Cronbach s alpha values are shown in the following table for each composite on the upgraded survey. Composite Cronbach s Alpha Finance Issues Utilization and Quality Management Network/Coordination of Care Pharmacy Health Plan Call Center Services Staff Provider Relations

11 Provider Satisfaction Survey Research & Best Practices Factor Analysis Factor analysis helps us determine the fewest number of underlying factors driving the relationship between each attribute (survey question) and overall satisfaction with the health plan. Because many attributes are often correlated with each other, it is not always a straightforward process to look at correlations to determine the key relationships. Factor analysis helps to look for the lowest common denominator, where the lowest common denominator is the core set of relationships from which all others spring. We started with exploratory factor analysis to determine whether any factors could be clearly defined that relate to overall satisfaction with the health plan. The result of a factor analysis is a table that displays the number of factors found to relate to overall satisfaction with the health plan. The attributes related to each factor are also provided. We then label each factor based on its associated attributes to create new composites, which are then used in the regression analysis. The factor analysis suggested four underlying factors that matched up well with the composites of the new survey tool. This means that each of the composites on the new survey tool is likely measuring a separate and unique aspect of satisfaction with the health plan. The four factors mentioned earlier are the following: Provider Relations and Staff Quality and Accessibility Pharmacy Finance Issues Regression Analysis Regression analysis is often used for prediction. For our purposes, the goal of the regression analysis was to gain an understanding of how ratings of overall satisfaction with the health plan may change when any of the composite scores change. As part of this analysis, significance testing is conducted to determine which composites can reliably be used to predict overall satisfaction ratings with the health plan. We talk about regression in terms of the percentage of the variation in ratings of satisfaction that can be accounted for by the composites we created. In other words, the regression analysis gives us a percentage of the variation in overall satisfaction with the health plan that is accounted for by our composites and how much is left unaccounted for (i.e., influenced by other factors not measured by our survey tool). Using each of the composites that we created as a result of the factor analysis, the regression analysis accounted for approximately 39% of the variation in ratings of overall satisfaction with the health plan. This is similar to what we find for other satisfaction surveys. Analysis Conclusion To sum up, the factor analysis and regression indicate four factors that match up closely with the composites of the survey tool and are significant predictors of overall satisfaction with the health plan. 11

12 Provider Satisfaction Survey Research & Best Practices Conclusion Based on findings from in depth focus groups, additional interviews with physicians and office managers, pilot study results, and comprehensive analysis and testing, has redesigned our Provider Satisfaction survey and refined our Provider Satisfaction survey processes. After lengthy research and analysis, we are pleased to introduce our redesigned Provider Satisfaction survey to provide health plans with more insightful physician feedback, enhancements for improved response rates, and increased overall reliability in measuring physician satisfaction with the health plan. Contact Information For additional information about s Provider Satisfaction Survey or best practices for survey administration, please contact: Your Executive Or x13 he Myers Group is a full service healthcare improvement organization providing call center support, member engagement/outreach, surveys and research. The company is an NCQA certified survey provider for the CAHPS, CAHPS PCMH, and Medicare HOS survey, and a CMS approved Medicare CAHPS survey provider. has been a leader in healthcare services since 1993, employing expertise and innovation in survey design, call center support, data collection, analysis and reporting. For additional information contact Clarissa Payne at x Phone: x13 12