California s Franchise Tax Board: Understanding Taxpayer Satisfaction with the Customer Service Call Center

Transcription

1 California s Franchise Tax Board: Understanding Taxpayer Satisfaction with the Customer Service Call Center Report Prepared for: California Franchise Tax Board September 2015 Allison Musvosvi Alton Williams David C. Barker Institute for Social Research, California State University-Sacramento Institute for Social Research California State University, Sacramento 6000 J Street Sacramento, CA (916)

2 ACKNOWLEDGEMENTS As with any complex and multifaceted research effort, this study s success is due to the combined efforts of several individuals across organizations. The ISR research team would like to thank all of those who made data collection and interpretation possible, and recognize the Franchise Tax Board staff for their assistance and collaboration throughout this project. We are also very grateful to the individuals who responded to our telephone survey. Special thanks especially to: Susan Marcale Jozel Brunett Anne Miller Norman Scott Allen Wilson Sandee Lee Josh Fernandez Lindsay MacFarlane Yadira Morales AJ Johal Victoria Tellesen Annette Webster Rita Stapp Joseph Jaco Amelia Russell Sheila Christanson Donia Ataya THE INSTITUTE FOR SOCIAL RESEARCH Founded in 1989, the Institute for Social Research (ISR) at Sacramento State University conducts public opinion surveys, performs program/policy evaluations, and provides a comprehensive list of data analytic services (both quantitative and qualitative) for government agencies, non-profits, and the academic community. A multidisciplinary social science organization, the ISR has extensive experience in designing research projects, data collection, data analysis, consulting, and data management. Our projects have enhanced decision-making, improved the use of resources, ensured program fidelity, and advanced the overall quality of programs/policies designed to address various social problems. The Institute for Social Research California State University, Sacramento 6000 J Street Sacramento, CA i

3 TABLE OF CONTENTS ACKNOWLEDGEMENTS... i EXECUTIVE SUMMARY... iii KEY FINDINGS... iii KEY FINDINGS continued... iv INTRODUCTION... 2 I. THE SAMPLE... 3 A. Demographic Characteristics... 3 B. Tax Preparation and Filing... 5 C. Characteristics of Latest Call to the FTB... 7 II. LIVE CALL SATISFACTION A. How Satisfied are Customers with Live Calls? B. Why are Some People Less Satisfied with Live Calls? III. AUTOMATED SYSTEM SATISFACTION A. How Satisfied are Callers with The Automated System? B. Why are Some People Less Satisfied with the Automated System? IV. MyFTB ACCOUNT SATISFACTION A. How Does Satisfaction Vary? B. Why are Some People Less Satisfied with MyFTB? V. LIVE-CHAT SATISFACTION A. How Satisfied are Customers with Live Chat? B. Why are Some People Less Satisfied with LiveChat? CONCLUSION Salient Findings Limitations Recommendations for Future Data Collection APPENDIX A: SURVEY DEVELOPMENT Pilot Testing Survey software CATI Methodology on Number of Call Backs Spanish Translation, Back Translation Process Response/Participation Rate APPENDIX B: STUDY METHODOLOGY APPENDIX C: SURVEY INSTRUMENT APPENDIX D: 2014 SURVEY FIGURES ii

4 EXECUTIVE SUMMARY As part of the Franchise Tax Board (FTB) s effort to meet one of its four main strategic goals, Taxpayer Centric Service, outlined in its Strategic Plan, the FTB contracted with the Institute for Social Research (ISR) in 2014 to conduct an initial call center and website customer satisfaction survey, and again in 2015 to conduct a second iteration of the survey. The FTB and ISR developed the customer satisfaction survey collaboratively. The telephone survey data collection period occurred from May 5th through June 7th, 2015, resulting in 1,030 completed interviews. KEY FINDINGS 1. The sample (n=1,030) was diverse but primarily Caucasian and English speaking; just under half were female, most were middle aged, middle income and college educated. 2. Most prefer using the phone to do their tax business and prefer to speak to a live person. 3. The most common reasons for calling were in response to receiving a notice, to get information and to request a payment plan (respectively). 4. Around 90% report being highly satisfied or satisfied, overall, with the live customer service. 5. More than half were satisfied with the amount of time spent on hold and waiting for a callback. 6. More than half of callers were able to complete their task without having their call transferred from one agent to another. 7. The agent characteristics that received the highest satisfaction ratings were 1) agent s voice was easy to understand; 2) agent was professional; 3) agent was friendly; 4) agent was knowledgeable; 5) agent was efficient. These characteristics all received positive satisfaction ratings from more than 90% of respondents. 8. The agent interaction characteristics that received the lowest satisfaction ratings were 1) hold time was reasonable amount of time; 2) agent was responsive to needs; and 3) agent was clear and thorough. Notably, we saw a 29% increase in satisfaction ratings on hold time from 2014 to The key agent characteristics that influenced satisfaction were responsiveness, efficiency, and knowledge. Those who strongly agreed that the agent who provided their service had demonstrated these characteristics were substantially more satisfied than those who felt the opposite. 10. While controlling for all demographic factors, respondents in the highest income group tended to be less overall satisfied with the agent than those in the lowest income group. Customers with the highest levels of educational attainment tended to express higher levels of satisfaction than those with the lowest educational attainment. 11. About 67% report being highly satisfied or satisfied, overall, with the automated service system. 12. The features of the Automated System that received the highest satisfaction were 1) system was easy to use; 2) system had a reasonable number of self-service options; and 3) system routed customers to the right place. 13. The features of the Automated System that had the lowest satisfaction ratings were 1) the system gave information needed; and 2) system took a reasonable amount of time. Close to iii 70% felt that the Automated System took a reasonable amount of time to use.

5 KEY FINDINGS continued 14. All of the system features influenced satisfaction with the Automated System. The number of selfservice options, the ease of system navigation, and the amount of time the system takes to use contributed to satisfaction ratings. Customers who were routed to the correct place and given the information they needed tended to be more satisfied with the system. 15. Age, race/ethnicity, and educational attainment were useful in predicting satisfaction with the Automated System. The eldest respondents, white respondents, and those with the highest levels of educational attainment tended to express more overall satisfaction with the system % of callers chose the option of receiving a scheduled call-back prior to speaking to an agent. 17. More than 90% of respondents received their call back within the timeframe they were given. 18. Roughly 39% of the sample visited the website in the 2014 tax season and did so before calling in. 19. Respondents primarily visited the website to check on their refund and do something payment related. Qualitative data indicate finding phone numbers and verifying information were two other common reasons. 20. Only 18% used the MyFTB account option prior to calling; very few used Live Chat (3%). 21. Nearly three quarters (72%) were satisfied or highly satisfied with the MyFTB account option of the FTB website in general. 22. The aspects of the MyFTB account that got the highest satisfaction ratings were 1) logging on was easy; 2) using MyFTB took a reasonable amount of time; and 3) navigating MyFTB was easy. These factors all had positive satisfaction ratings from more than 80% of respondents. 23. The feature of MyFTB that got the lowest satisfaction rating was 1) information was easy to find. More than 23% of respondents expressed disagreement with this statement. 24. Respondents who strongly agreed that they could find the information they needed and within a reasonable time tended to be more satisfied with the account feature. The log-in process also affected satisfaction ratings for the MyFTB account feature. 25. Women, and those in the highest income group tended to express less satisfaction with the MyFTB account. 26. Most respondents who had used the LiveChat feature were satisfied with the LiveChat agent s performance. 27. The LiveChat characteristics that got the highest satisfaction ratings were 1) agent responded in a timely fashion and 2) LiveChat was easy to use. These two characteristics were tied, receiving positive ratings from more than 94% of respondents. The next highest ranked characteristics were 3) there were reasonable number of pauses between messages; 4) agent was responsive to needs; and 5) agent was knowledgeable. Agent responsiveness and knowledgeability were also tied with equally positive ratings. 28. The LiveChat characteristic that got the lowest satisfaction ratings was1) got issue resolved and did not need to speak to an agent. 29. The key agent characteristic that influenced satisfaction was clarity. Those who strongly agreed that the agent provided clear instruction tended to be more satisfied than customers who felt the opposite. Whether a customer was able to resolve their issue via LiveChat was also an influential factor on overall satisfaction with the LiveChat experience. 30. Demographic characteristics of the customer were not useful for predicting customer satisfaction with the LiveChat agent. iv

6 Highlights: Live Agent: Strongly Agree/Agree Overall satisfaction with Live Agent On hold a reasonable amount of time Agent transferred me to right person Agent voice was easy to understand N=537 N=1,018 83% 90% (Up 7%) 48% 77% (Up 29%) 71% 86% (Up 15%) 93% 95% Agent was clear and thorough 84% 90% Agent was responsive 85% 89% Agent was friendly 88% 92% Agent was knowledgeable 83% 91% (Up 8%) Agent was professional 92% 95% Automated system: Strongly Agree/Agree Overall satisfaction with Automated system N=69 N=1,100 63% 67% Was easy to use 65% 79% (Up 14%) Took a reasonable amount of time Had a reasonable number of options 64% 70% 67% 76% (Up 9%) Routed me to the right place 63% 74% (Up 11%) Gave me the information I needed 56% 54% (Down 2%) LiveChat: Strongly Agree/Agree N=11 N=37 Overall satisfaction with LiveChat N/A 1 78% Easy to use 72% 94% (Up 22%) Response time reasonable 70% 95% (Up 25%) Reasonable number of pauses 63% 86% (Up 23%) Agent was knowledgeable 67% 81% (Up 14%) 1 Overall satisfaction with the LiveChat feature was not asked directly in the 2014 survey. The closest comparison we can make is that in 2014, 75% of respondents said they were satisfied or very satisfied with the website. 1

7 INTRODUCTION The Institute for Social Research (ISR) at California State University, Sacramento (CSUS) partnered with the California Franchise Tax Board (FTB) to execute the second study of customer satisfaction with FTB Call Centers, the website, and its features. Using Computer Assisted Telephone Interviewing (CATI) techniques, we surveyed California residents who had called the FTB during the 2014 filing season. As a follow up to the 2014 study, the ISR collaborated with the stakeholders at FTB to modify the 2015 survey questionnaire, based on lessons learned from the previous year s study. In addition, some changes reflected a shift in objective as it pertained to the website questions. On the whole, we worked together to alter the 2015 instrument so as to better suit FTB goals while maintaining enough structure to be comparable to 2014 results. The most notable change in 2015 is an increased emphasis on probing for greater detail and gaining open-ended responses from respondents. Specifically, in several instances, survey interviewers asked follow-up questions to respondents, urging them to elaborate on why they were dissatisfied with aspects of their interaction with the FTB Call Center or features of the website. These drill-down questions provided a goldmine of responses, which enabled a more fine-grained analysis of the system limitations and weaknesses from the customer s perspective. Other improvements to the 2015 data collection strategy included FTB providing a larger sampling frame and extending their participant collection period. The FTB recruited participants who called into the call centers over a longer time period than in 2014, thereby increasing the number of customer contacts ISR had to pool from. Moreover, survey interviewers received more in depth training on methods to reduce behaviors that could lead to unintended biases that affect the quality of data. In this report, we review the conclusions of the study. Using simple descriptive analysis, qualitative methods, and multiple regression modeling, we were able to describe customer satisfaction regarding the FTB Call Center experience, the website, and its various features. 2

8 I. THE SAMPLE The Franchise Tax Board provided ISR with the names and phone numbers of 4,271 taxpayers who had called into the FTB Call Centers during the 2015 tax season (March 16-March 30; April 1-3) and agreed to take a customer satisfaction survey. A total of 1,030 participants completed the survey fully; an additional 83 respondents answered a portion of the survey questions, bringing our total survey sample size to 1,113. Data collection occurred between May and June of A. Demographic Characteristics In an effort to determine the general composition of our sample and to make comparisons to the broader California State composition, we collected demographic data from our participants. Comparing the racial composition, gender, age, education, income, and language preferences of our sample, we determined that the 2015 sample of respondents was representative of the broader state of California population, but that the population of FTB callers does differ from the broader population in some important respects. Race and Ethnicity As Table 1 illustrates on the next page, and in-keeping with what we had observed in 2014, the sample revealed significant racial and ethnic diversity: 46% of sample respondents identify as White and not Hispanic, 21% as Hispanic, 15% as Black/African American, 7% as Asian/Asian-American, 4% as mixed race, 1% as Native American, and 1% as Hawaiian/Pacific Islander. Five percent (5%) identify as Other. As compared to the most recent California Census, callers to the Franchise Tax Board are slightly more likely to be White (46% to 39%), somewhat less likely to be Asian/Asian American (7% to 14%), somewhat more likely to be Black/African American (15% to 7%), and somewhat less likely to be of Hispanic/Latino ethnicity, regardless of race (21% to 38%). Gender and Age As had been the case in 2014, men comprised a slightly larger percentage of the sample (56%) than women, and respondents between 31 and 54 years old comprised the largest proportion of sample respondents (53%). This age group was followed by those between 55 and 64 (22%), those between 19 and 30 (13%), and those 65 and over (12%). Education and Income Again, matching what we had observed in 2014, in comparison to the general public, the FTB sample included higher percentages of residents who had attained baccalaureate degrees (34%) and graduate/professional degrees (20%), and a lower percentage of residents that had not obtained a high school degree (<1%). As for income, the sample revealed a bimodal distribution, which is to say that the largest percentages of respondents fell into the highest and lowest income categories. To be precise, 22% of respondents reported earning more than $100,000 annually, whereas 21% reported under $30,000 annually). 3

9 TABLE 1: DEMOGRAPHICS* Age Group 2 N=1,039 N=569 Highest Education N=1,040 N=555 Under 19 years <1% Junior High School <1% 1% 19 to 30 13% 8% High School 21% 21% 31 to 54 53% 59% Community College 25% 26% 55 to 65 22% 22% Four-year College 34% 31% 66 or older 12% 11% Professional/Grad School 20% 20% Income Group N=1,033 N=481 Race N=951 N=550 Under 30 White thousand dollars 21% 23% 46% 45% 30 to 40 13% 13% Latino 21% 22% 40 to 50 12% 8% African American 15% 16% 50 to 60 11% 9% Asian 7% 6% 60 to 70 8% 8% Native American 1% 1% 70 to 80 5% 6% Hawaiian/Pacific Islander 1% 3% 80 to 90 4% 4% Mixed 4% 4% 90 to 100 5% 3% Other 5% 3% % 23% Primary language N=1,023 N=574 Gender N=1,025 N=578 English 91% 92% Male 56% 57% Spanish 5% 4% Female 44% 43% Mandarin/Chinese <1% 1% Cantonese <1% <1% Farsi <1% <1% Korean <1% <1% Other 3% 3% *The margin of error for all demographic categories, in 2015, is less than or equal to plus or minus 3%. 2 In 2015, the age categories were divided slightly differently from the 2014 age categories. Specifically, younger age groups were parsed out in order to provide greater detail. 4

10 B. Tax Preparation and Filing In this section, we discuss the filing status and business preference of California tax payers. In general, there appears to be a trend towards self-filing, and an increase in the percentage of people who use tax software to file online. Tax Category of Respondents? As Figure 1 illustrates, the largest percentage of callers are single adults (41%), followed by those who are married and filing jointly (37%). Participants whose 2014 tax filing status differed from those mentioned comprised a smaller portion of the sample. Figure 1: Tax Category (N= 940)* 41% 37% 16% 4% 2% Single Married filing jointly Head of Household Married filing separately Widow (er) *Margin of error is less than or equal to 3%. How Do Respondents File Their Taxes? As Table 2 demonstrates below, and as was the case in 2014, a majority (60%) of the respondents reported that they had worked with a tax professional to file their 2014 tax returns, and only a small percentage of them were referring to an online tax service rather than another person. Of the respondents who filed their taxes on their own, 66% used tax preparation software, and 27% used the CalFile option. Finally, while some respondents filed their taxes by mail (39%), most who filed did so online (61%). 5

11 TABLE 2: TAX PREPARATION Did you file your taxes yourself this year or did you use a tax professional? Total Percent Total Percent Was the tax professional an online service, or did you actually N=1,042* take your taxes N=615*** to another person to prepare them? SELF % TAX PROFESSIONAL % NOT YET FILED 56 5% YES (AN ACTUAL PERSON) NO (AN ONLINE SERVICE) % 27 4% Did you use tax preparation software to file your taxes this year? N=420** Did you file your taxes online or by mail? N=985* YES ONLINE % % (INCLUDES E-FILE) NO % BY MAIL % *Margin of error is less than or equal to 3%. ** Margin of error is less than or equal to 5%. ***Margin of error is less than or equal to 4%. How Do Respondents Prefer to Conduct Their Tax Business? As we see in Figure 2, roughly twice as many respondents reported preferring to conduct tax business over the phone (49%), as compared to doing so online (25%) or through the mail (20%). Only 6% prefer conducting such business in person. Consistent with these results, most respondents did not visit the FTB website before making a call in to the call center. As compared to 2014, we observed an increase in respondents who preferred to conduct their tax business over the phone and a decline in those who preferred to do so online (see Appendix C, Figure 2). 6

12 Figure 2: How People Prefer to Conduct Tax Business with FTB (N=985)* 49% 25% 20% Over the phone Online Through the mail In person *Margin of error is less than or equal to 3%. 6% C. Characteristics of Latest Call to the FTB In this section, we examine the general features of the caller s experience. We consider the main reason customers call in to the FTB, the number of times they are transferred from one agent to another, whether they make a request to speak with a Supervisor, and whether they visited the FTB website before calling. Finally, for those who did visit the website, we inquired as to why they visited the website. Why Do People Call the FTB? Almost two-thirds of the sample (63%) called the FTB Call Center in response to a notice they had received. As Figure 3 demonstrates, some 32% of respondents reported that they called to get information, and 22% called to request a payment plan like an Installment Agreement. The rest of respondents called to make a payment or in order to file a tax return. These reasons seem relatively consistent to what we had observed in 2014 (see Appendix C, Figure 3). 3 Through open-ended survey questions, we explored the responses of the 33% of respondents who had some other reason for calling the FTB Call Center. 3 In 2015, we changed the wording of the question about respondents reason for calling the FTB from what it had been in 2014, in order to get more detailed responses. Specifically, in 2015, we separated this item into two distinct questions. The first of these simply inquired about whether they had called in response to a notice they had received. This was in the form of a yes or no question. The second question required respondents to report the reason for their most recent call to the FTB, by choosing which from a list of various reasons for calling the FTB applied to themselves beyond having received a notice. 7

13 Open-Ended Responses: Why Do People Call the FTB? Of those who selected the other response as to why they called the FTB, responses tended to center around payment related inquiries, refund related inquiries, requests for extensions, and requests for clarification. Some respondents called for multiple reasons and could not select one as the primary reason for their call. A number of callers were requesting information from their filings in previous tax years, while others were verifying the authenticity of the notices they had received. One respondent said there were typos on his notice letter, so he thought it was fraudulent. A few callers had complaints about fraudulent activity related to their taxes or about lost tax or refund checks. There were complaints about the amount of money being withheld from checks, or about the garnishment amounts. In particular, one respondent said, Someone is using my husband s Social Security number and they are clearing it up. A number of customers called to correct a mistake that they had made, or to report an error that the FTB had made. One respondent said I filed taxes in 2011 and they claimed I didn't. Figure 3: Primary reason for call (N=1,102)* 33% 32% 22% 9% 4% Some other reason Get information Request a payment plan Make a payment File a tax return *Margin of error is less than or equal to 3%. How Do Customers Route Themselves to the Agent? Following their interaction with the FTB Automated System, 65% of callers chose the option of receiving a scheduled call-back. Roughly 91% of these respondents received their call-back within the expected timeframe. How Many Times is the Typical Caller Transferred? In 2015, a majority of respondents were able to accomplish what they needed with the FTB agent without being transferred at all (54%). As we see in Figure 4, 30% were transferred once, 11% were 8

14 transferred twice, and only 5% were transferred three or more times. These numbers suggest significant improvement over 2014, when approximately 33% of respondents reported being transferred three or more times, and only 45% reported accomplishing their goals without being transferred at all (see Appendix C, Figure 6). Figure 4: Number of Times Transferred Between Agents (N=1,000)* 54% 30% 11% 5% Never Once Twice Three or more times *Margin of error is less than or equal to 3%. How Often Do Callers Request a Supervisor? As Figure 5 shows, most respondents (96%) did not request a Supervisor Call-Back. We observed a general decline in requests to speak with a Supervisor from 2014 to 2015 (see Appendix C, Figure 7). 9

15 Figure 5: Supervisor Call-Back Requests (N=1,067)* 4% 96% Yes No *Margin of error is less than or equal to 3%. Did Callers Visit the FTB Website Before Calling FTB? As Figure 6 illustrates on the next page, most respondents (61%) did not visit the FTB website before calling the call center. Figure 6: Visited website before contacting FTB (N=1,060)* 39% 61% Yes No *Margin of error is less than or equal to 3%. 10

16 What was the Primary Reason for Visiting the FTB Website? As Figure 8 shows below, the primary reason for visiting the FTB website varied substantially. Reasons tended to center around payment inquiries -- either money owed or expected refunds. Some 21% of respondents had visited to check on their refund status while 20% visited for something payment related. The rest visited in hopes of getting information on a tax question, or to get clarification or confirmation that they had filed their taxes correctly. The remaining 26% who said other reported a mixed bag of reasons for visiting the website. Figure 7: Primary Reason for Visiting the FTB Website (N=413)* 26% 21% 20% 12% 9% 7% 5% Other Check refund status Make a payment Research a tax question Set up a payment plan Make sure my payment or tax return was received Get filing assistance *Margin of error is less than or equal to 5%. Open-Ended Responses: Why Did Customers Choose to Call After Visiting the Website? A surprisingly high percentage of respondents reported contacting FTB, after visiting the website, to get information not available online. Many of the respondents gave an account of the online information being too broad, forcing them to call in order to get specific information regarding their particular situation. As one respondent put it, The website gave generic answers but I wanted a more specific explanation. Another simply stated, Because the information I was seeking was not available. Having said that, a few respondents indicated that the website may have had too much information in certain respects, making it hard to navigate. The open-response data suggest that many of those who responded to this survey question had unique concerns/situations that fell outside of the scope of the website s function. A fair number of respondents described reading website instructions that suggested they call FTB in order to resolve their issue. In the next section of this report, we consider survey respondents satisfaction with live calls. 11

17 II. LIVE CALL SATISFACTION In this section, we discuss the customer s experience with the Customer Service Agent, and their overall level of satisfaction with the call. A. How Satisfied are Customers with Live Calls? First, we consider FTB callers overall levels of satisfaction. Overall Satisfaction Respondents rated their experience with the agent on a five-point scale, with responses that ranged from strongly agree to strongly disagree. Overall, as shown in Figure 9, most respondents were satisfied or very satisfied with their experience with the Customer Service Agent, though the percentage of those who reported being very satisfied is down by about ten percentage points from Figure 8: Satisfaction with Customer Service Agent (N=1,080)* Overall satisfied with agent 41% 49% On hold a reasonable amout of time 23% 54% Agent transferred me to right person 29% 57% Agent's voice was easy to understand Agent was efficient Agent was clear and thorough Agent was responsive Agent was friendly Agent was knowledgeable Agent was professional 41% 39% 38% 41% 42% 42% 44% 54% 52% 52% 48% 50% 49% 51% Strongly Agree Agree Neither Disagree Strongly Disagree *Margin of error is less than or equal to 3%. Specific Agent Characteristics When it comes to specific agent characteristics, 95% of respondents agreed that the agent was professional, almost half of whom (44% overall) strongly agreed with this statement. Only a few respondents disagreed that the agent was professional. Similarly, the general consensus was that the agents were knowledgeable. A strong 49% of respondents agreed that the agent they spoke to was knowledgeable, and an additional 42% strongly agreed to the same. A good majority of respondents agreed or strongly agreed that the 12

18 agent they spoke to was friendly, 50% and 42% respectively. Agent responsiveness was also rated by respondents; of the 89% who agreed that the agent they spoke to was responsive to their needs, 41% strongly agreed. Agent clarity/thoroughness and efficiency were other characteristics that respondents evaluated. More than half (52%) agreed that the agent was clear/thorough, and another 38% strongly agreed. Essentially the same breakdown applies to perceptions of agent efficiency, which represents an improvement over When it comes to the agent s voice clarity, specifically, an even larger percentage of respondents agreed that the voice was easy to understand (54%), with another 41% strongly agreeing. Other Call Characteristics Moving away from agent characteristics, most respondents reported being satisfied with other aspects of the call as well. Specifically, most respondents (86%) agreed that they were transferred to the right place during their call, and 77% of respondents agreed that the agent put them on hold for a reasonable amount of time, with 23% strongly agreeing. B. Why are Some People Less Satisfied with Live Calls? In this section, we examined the extent to which demographic characteristics and the Customer Service Agents characteristics help to account for differences in caller satisfaction, overall. How Do Demographic Differences Predict Differences in Overall Satisfaction? As Figure 10 reveals, income and education were predictive of overall satisfaction, but the effect sizes are relatively small. Specifically, all else being equal, those with the highest incomes tended to express levels of satisfaction that were 6% lower than those reported by respondents with the lowest incomes, whereas (perhaps paradoxically) respondents with the highest levels of educational attainment tended to express levels of satisfaction that were 7% higher than those with the lowest educational attainment. Gender, age, race, and preferred language did not help explain overall satisfaction with the agent s service. Figure 9: Demographic Predictors of Overall Satisfaction with Customer Service Agent (N=947) Income Age -6% 0% Language Gender Race 0% 1% 2% Not Statistically Significant Statistically Significant Education 7% 13

19 In comparison to 2014 results, we found that race/ethnicity mattered less in 2015, but that education and income mattered more as correlates of overall satisfaction (see Appendix C, Figure 20). How Do Differences in Agent Characteristics Predict Differences in Overall Satisfaction? As Figure 11 highlights below, the most important factor to callers appears to be the agent s perceived responsiveness; those who strongly agreed that the agent was responsive to their needs were 34% more satisfied with their interaction than were those who strongly disagreed. Next, respondents who strongly agreed that the agent was efficient were 25% more satisfied overall, and those who strongly agreed that the agent was knowledgeable were 22% more satisfied overall than those who strongly disagreed. Lastly, those who expressed that their call was transferred a reasonable number of times tended to report satisfaction levels that were 17% higher than those who expressed disagreement with this statement. Compared to what we had observed in 2014, hold time was less important in 2015, and agent characteristics were more important (see Appendix C, Figure 21). Figure 10: Predictors of Overall Satisfaction with Customer Service Agent (N=1,066) Agent was thorough -4% Agent voice was clear -2% Number of calls to FTB 0% On hold reasonable amountof time Agent was friendly Agent was professional 2% 4% 11% Not Statistically Significant Statistically Significant Transferred reasonable number of times Agent was knowledgeable Agent was efficient Agent was responsive to my needs 17% 22% 25% 34% Open-Ended Responses: Overall, why were you dissatisfied with the Customer Service Agent? Of those who chose to provide open-ended responses, the leading explanation for a respondent s dissatisfaction with the customer service they received from the agent was that they received conflicting or incorrect information from the agent(s). Some respondents who spoke to multiple agents cited receiving conflicting information from the different agents. One respondent mentioned that his/her issue is not yet resolved and [it appears] the paperwork I sent in had not been processed and previous calls [were] not noted. Furthermore, some respondents indicated that the agents were brief and seemed, at times, unsympathetic particularly with callers who were requesting payment plans or inquiring about collections. In the words of one respondent: I felt as though I was being treated like a criminal due to my inability to pay. Finally, some respondents felt that the agents were setting payment amounts that were unrealistic for them to attain. As one respondent said, Because I was trying to arrange a 14

20 payment plan an agent chose an amount of money to pay for me which was much larger than an amount I could afford. In the next section, we move away from live calls to explore caller satisfaction with the Automated System. III. AUTOMATED SYSTEM SATISFACTION In this section, we discuss the customer s experience with the Automated System, which greets the customer when they call into the FTB Call Center. A. How Satisfied are Callers with The Automated System? First, we explore the degree to which customers reported being satisfied with the Automated System. We measured such satisfaction generally, as well as with regard to five specific aspects of the phone call experience. Respondents rated their experience with the Automated System on five-point scales, with responses that ranged from strongly agree to strongly disagree. Respondents overall rating and evaluation of those five component statements were as follows. How Satisfied are Taxpayers Using the Automated System? Similar to the 2014 results, and as Figure 12 illustrates, more than half of the respondents (67%) were satisfied with the Automated System -- including 12% who reported feeling very satisfied. By contrast, about 26% of respondents expressed dissatisfaction with the Automated System. Again, though, these strong levels of satisfaction are lower than what was reported in 2014 (see Appendix C, Figure 28). Figure 11: Satisfaction with Automated System (N=1,100)* Overall, satisfied with system 12% 55% Was easy to use 13% 66% Took a reasonable amount of time 9% 61% Had a reasonable number of options 10% 66% Routed me to the right place 15% 59% Gave me the information I needed 10% 44% Strongly Agree Agree Neither Disagree Strongly Disagree *Margin of error is less than or equal to 3%. When it comes to specific aspects of the system, respondents largely agreed that the Automated System (a) was easy to use (79%), (b) had a reasonable number of self-service options (76%), (c) routed them to the right place (74%), and took a reasonable amount of time (70%). However, 15

21 comparatively fewer respondents (54%) agreed/strongly agreed that the Automated System gave the information they needed. In general, most respondents agreed, at varying degrees, that they were satisfied with the Automated System. However, compared to the other methods of interacting with FTB and its various customer service tools that we examined, the Automated System elicits the weakest satisfaction ratings from customers. 4 B. Why are Some People Less Satisfied with the Automated System? In this section, we examine the extent to which demographic characteristics and the features of the Automated System predict overall customer satisfaction with the FTB automated call system. Demographic Differences As Figure 13 shows, differences in age, educational attainment, and race/ethnicity seemed to affect overall levels of automated call satisfaction. The eldest respondents in our sample tended to express levels of satisfaction that were 8% greater than did the youngest respondents, and those with the highest educational attainment tended to express satisfaction levels that were 11% greater than did those with the lowest educational attainment. Finally, White respondents tended to express satisfaction levels that were 4% greater than did respondents of color. By contrast, gender, income, and preferred language did not have a predictive influence on overall satisfaction with the Automated System. Although preferred language spoken in the home was a relatively important predictor of satisfaction in 2014 (with non-english speakers tending to report higher levels of satisfaction), we did not observer a comparable pattern in 2015 (see Appendix C, Figure 29). 4 One caveat to note is that customers who were able to complete their task using only the Automated System were not represented in this sample, only customers who spoke to a live agent were recruited for the sample. 16

22 Figure 12: Demographic Predictors of Overall Satisfaction with Automated System (N=947) Income -5% Gender -3% Language Race 0% 4% Not Statistically Significant Statistically Significant Age 8% Education 11% How Do Differences in Call Characteristics Predict Satisfaction? Again, using multiple regression analysis, we modeled how each feature of the system contributed to overall satisfaction. Figure 14 summarizes the results. The most important factor appears to be whether one was routed to the right place; those who strongly agreed with that statement tended to report levels of satisfaction that were, on average, 28% higher than did those who strongly disagreed. Other important factors include the degree to which the Automated System (a) gave the respondent the information s/he needed, (b) was easy to use, (c) had a reasonable number of self-service options, and (d) took a reasonable amount of time. Figure 13: Predictors of Overall Satisfaction with Automated System (N=1,089) Number of calls to FTB 1% Took a reasonable amount of time Reasonable number of selfservice options Was easy to use Gave me the information I needed 16% 18% 20% 20% Not Statistically Significant Statistically Significant Routed me to the right place 28% 17

23 These results largely mirror what we observed in 2014 with regard to being routed to the right place and getting the information that they needed. However, in 2014, the other factors were not statistically related to overall satisfaction (see Appendix c, Figure 30). Open-Ended Responses: Why were you dissatisfied with the Automated System? Of those who reported dissatisfaction and chose to provide open-ended explanations, the most prominent reason was that it simply took too long to navigate the system. Callers also complained about having to listen to service options multiple times or having to figure out which option came closest to their situation in order to get routed to the right place. Relatedly, respondents commented on the lack of a service option that would directly connect a caller to a Customer Service Agent. A significant portion of respondents were also critical of the Automated System s lack of self-service options or ability to answer non-standard questions. One respondent said, It didn t offer me anything I needed to address what I was calling about. Similarly, another respondent said, None of the categories on the prompt pertained to my question. Some comments pertained to a generalized dislike of Automated Systems; for example, one respondent said, Don t like listening to generic comments that might work in a cookie-cutter kind of way. I like the idea of service and being able to reach somebody. Another respondent indicated, [I] Despise Automated Systems; burden is on the people calling. Finally, one respondent reported spending months trying to determine the security and authenticity of the phone line. S/he reported being hesitant to punch their Social Security Number into the phone for fears of identity theft. In the next section, we analyze levels of customer satisfaction with the MyFTB account. 18

24 IV. MyFTB ACCOUNT SATISFACTION Beyond the call experience, we inquired about customers experiences with the website. In this section, we discuss the customer s experience with the MyFTB account. As was the case in the previous section, we begin by discussing the levels of satisfaction, and then we attempt to explain the variance in that satisfaction. A. How Does Satisfaction Vary? Only 18% of our sample had used the MyFTB option. Of those, as Figure 15 illustrates, a majority (72%) report feeling some level of overall satisfaction, with 16% strongly agreeing. Figure 14: MyFTB Account Satisfaction (N=202)* Overall, I was satisfied 16% 56% Took a reasonable amount of time 17% 67% Information was easy to find 13% 59% Navigating was easy 17% 66% Logging on was easy 24% 62% Strongly Agree Agree Neither Disagree Strongly Disagree *Margin of error is less than or equal to 7%. More specifically, 86% agreed that logging on was easy, and 83% reported that navigating was easy. Seventy-two percent (72%) agreed that information was easy to find, and 84% agreed that the process took a reasonable amount of time. To compare these results to those we observed in 2014, see Appendix C, Figure 31. B. Why are Some People Less Satisfied with MyFTB? In this section, we look at the extent to which demographic characteristics and the MyFTB web features could predict use and overall customer satisfaction with the MyFTB account. Demographic Differences First, we examined the degree to which using the MyFTB account option could be predicted by demographic characteristics. As shown in Figure 16, we found that gender, age and income made a difference. Specifically, women were 5% less likely to use the MyFTB feature than were men, and 19

25 those with the highest income were 13% less likely to use the MyFTB feature than were those with the lowest income. On the other hand, race, preferred language, and education did not have predictive influence on likelihood to use the MyFTB account option. Finally, in something of a surprise, the eldest adults tended to be 10% more likely to use the MyFTB account than were the youngest adults. Figure 15: Demographic Characteristics as Predictors of Using MyFTB Account (N=929) Income -13% Education -4% Gender Language -5% -1% Not Statistically Significant Statistically Significant Race 0% Age 10% How Do Different Perceptions about the MyFTB Account Predict Satisfaction? As we can observe in Figure 17, the factors that influenced satisfaction with this online feature were ease of login process, ease of finding information, and the amount of time it took to use MyFTB. Specifically, those who strongly agreed that it was easy to find information on MyFTB were 44% more satisfied with the MyFTB account than were those who strongly disagreed. Similarly, respondents who strongly agreed that MyFTB took a reasonable amount of time were 39% more satisfied with the account than were those who strongly disagreed, whereas those who strongly agreed that logging into MyFTB was easy were 22% more satisfied with the MyFTB account than were those who strongly disagreed. 20

26 Figure 16: Predictors of Overall Satisfaction with MyFTB (N=201) Navigation was easy-4% Visited website before calling FTB 0% Number of calls to FTB Logging on was easy 1% 22% Not Statistically Significant Statistically Significant Took a reasonable amount of time 39% Information was easy to find 44% Open-Ended Responses According to those who provided open-ended responses, dissatisfaction with the MyFTB account was due largely to difficulty navigating the website, inability to gain account access or trouble logging in, and lack of information or difficulty understanding the information posted online. Respondents seemed to have a hard time understanding the information posted on the website. One respondent notably said, Because I couldn t understand the information there, the reasoning behind it. Other respondents had a hard time navigating the website saying, I couldn't find the information I was looking for. A different respondent simply suggested that it should perform more functions. Respondents seeking account information were frustrated about the limited amount of information available. A few reported that they could not see a full record of their payment history, or that past due payments could not be accessed online. Some reported errors in the balances shown, or that payments were not correctly reflected on their account histories. One respondent complained that, it did not provide sufficient information for the status of the payment and when it was completed. Difficulties logging in were another commonly cited reason for dissatisfaction with the MyFTB account. Log in problems ranged from the system reporting that their account did not exist, or the system not accepting their social security number for log in purposes. In addition, some respondents could not resolve their issue on the MyFTB account and had to make a call in to the call center. This was a point of irritation for them. In the next and final section of this report, we discuss our findings as they pertain to the degree to which customers were satisfied with the LiveChat function, as well as the factors that explain why some people were more satisfied than others. 21

27 V. LIVE-CHAT SATISFACTION In this final section, we discuss the customer s experience with the LiveChat feature on the FTB website. This feature allows users to engage with an agent through a secure online chat system. Given the low number of respondents who had used the LiveChat feature, these results should be interpreted with caution. A. How Satisfied are Customers with Live Chat? As Figure 18 illustrates, of the small number of respondents who used the LiveChat feature (only 3% of the sample), 78% were satisfied and 22% were dissatisfied, overall. Figure 17: LiveChat Satisfaction (N=37)* Overall, I was satisfied 28% 50% Got my issue resolved 8% 54% Had a reasonable number of pauses between messages 16% 70% Was easy to use LiveChat agent responded in a timely Provided clear and thorough information LiveChat agent was responsive to my needs LiveChat agent was knowledgeable 32% 30% 27% 24% 24% 62% 65% 57% 57% 57% *Margin of error is less than or equal to 16%. Strongly agree Agree Neither Disagree Strongly disagree With regard to specific features of LiveChat, of those who used this feature, 81% agreed that the agent they communicated with was knowledgeable. Of these respondents, 17% disagreed with the statement. Another 81% of respondents who used the LiveChat option agreed that the agent was responsive to their needs. About a fifth (19%) disagreed with the statement that the agent was responsive. Most respondents agreed or strongly agreed that the LiveChat agent provided clear and thorough information, 57% and 27% respectively. Only 16% disagreed with this statement. In terms of efficiency in the LiveChat communication experience, most respondents agreed that the LiveChat agent responded in a timely fashion. Over half (65%) agreed while 30% strongly agreed. Only 5% of respondents disagreed with the statement about the agent s timeliness in responding. Most respondents found the LiveChat feature easy to use. The overwhelming majority of respondents who used LiveChat agreed that the system was easy to use (94%). Respondents reported that there were a reasonable number of pauses between messages. Most agreed or strongly agreed with this statement, 70% and 16% respectively. Only 13% disagreed with this statement. 22

28 About 54% of LiveChat users were able get their issue resolved and did not need to speak to an agent. Eight percent (8%) strongly agreed with this statement. Some 35% expressed dissatisfaction in getting their task resolved. There were only a few items related to the LiveChat feature that elicited dissatisfaction from respondents. While a majority were generally satisfied with the website feature, a number expressed dissatisfaction. However, few chose the neutral option about the LiveChat features, indicating that customers held strong feelings, both positive and negative, towards the LiveChat. B. Why are Some People Less Satisfied with LiveChat? In this section, we explore the extent to which demographic characteristics and the characteristics of the LiveChat agent predict overall customer satisfaction with the LiveChat experience. Demographic Differences As we see in Figure 19, demographic characteristics do not help to account for differences in LiveChat use, but they do help explain differences in satisfaction among those who used the feature. Specifically, the eldest LiveChat users tended to express levels of satisfaction that were 76% lower, on average, than were those of the youngest users. Furthermore, those with the highest level of educational attainment tended to report levels of satisfaction that were 39% greater, on average, than were those attributable to those with the lowest levels of educational attainment. Figure 18: Demographic Characteristics as Predictors of Overall Satisfaction with LiveChat (N=31) Education 39% Language -16% Race Income -7% 18% Not Statistically Significant Statistically Significant Age -76% Gender -3% The Relative Impact of Different LiveChat Features As we observe in Figure 20, those who strongly agreed that they were able to resolve their issue online tended to report levels of satisfaction that were 39% higher, on average, than were those attributable to respondents who strongly disagreed. Similarly, respondents who felt strongly that the LiveChat 23