Net Promoter Scores August 29th, :00-5:00pm (EDT)

Transcription

1 Net Promoter Scores August 29th, :00-5:00pm (EDT) Overview In response to some recent CCSF member discussion, this brand new topic on Net Promoter Scores (NPS) was discussed in its first ever conference call format. For those not yet familiar with this new score, it is a metric that seeks to learn if an organization s customer is a promoter or a detractor. It is a methodology touted by Baine & Company and made popular by Fred Reichheld s book The Ultimate Question. The ultimate question is: Would you recommend us to a friend? The score is, quite simply, the total number of promoters minus detractors. Some enthusiasts for this new metric believe that it is a better way to measure customer loyalty and organizational health, more so than the typical customer satisfaction scores. However, the simplicity of the metric sparked some debate and raised a number of valid questions. Therefore, to help sort through the information and put it in context was Dr. Jodie Monger of Customer Relationship Metrics. As a short summary to the outcome of this call, it is still a very new metric and those in the contact center industry do have some valid questions that arise from using it as a stand alone metric. As such, some are using it in conjunction with customer satisfaction scores or at the enterprise-level as a leading indicator type of metric. On this call there did appear to be more cons than pros, but that doesn t mean it should be discounted as there were quite a few organizations on the call going down this path. It is an excellent metric that can further spark member discussion. Participants using this metric shared how it is being used within their organizations:

2 One member shared that they started recording it a contact center level for the last 6 months. At a corporate level, they are considering whether or not to move from customer satisfaction ratings in the next financial year to a Net Promoter Score or to use both. The calculation method is to ask the question in their post call evaluation surveys and corporate surveys: Would you promote or recommend us? They work on a zero to 10 scale. They take the number of customers that have given a rating of 9 or 10 out of 10 (the advocates) and subtract any customer that has rated a 6 or below (those customers that are indifferent or could possible speak poorly of the company); For another participant, she is in the middle of a project plan for introducing this score to her organization s Canadian division on October 11 with a pilot. They are looking at a very small customer sampling of 50, which is their quota. This is because the US division has been piloting this for a couple of quarters using small sample sizes and yielding great results. So, they have 10 of their executives tagged to 5 survey respondents each. They are charged with getting an in-depth analysis of why the customer is either a detractor or a promoter or passive in the scale. By Q1 or Q2 of 08 it will likely be used as the primary customer satisfaction metric; As expressed by another participant, the likelihood to recommend question has been asked since January of 2005 in what is termed their perceptual survey (a corporatewide survey). It consists of a random sampling of their entire customer base, which is about ½ million. They collect about 1800 completed telephone-based interviews per year. They have been calculating the NPS on request of their CEO for about the past year. Another method described was by an organization that has been using NPS since April of this year. They survey almost 1000 customers per month, both clients and prospects who call the center. NPS is also used by another organization in conjunction with their customer satisfaction scores. They sample approximately 100 to 200 individuals or merchants per month using a zero to 10 scale. A sixth organization is also using NPS and has been since January They measure it very much the same way as has been outlined by others. However, they use it as an indicator only. Where they put more emphasis is with the key customer satisfaction drivers and the customer verbatims. A slightly different method described was the absence of the 10-point scale. Instead, this organization takes the percentage of customers that are extremely likely to promote the organization (the promoters) and subtract from that those that are somewhat, not very

3 or not at all likely to promote (the detractors). They also have a very likely rating, which is treated as passive. They survey about a 1000 customers per month. Connecting customer satisfaction scores with individual agent performance has been a focus within the CCSF membership for quite some time now. So, it begs the question of whether or not this practice can still be achieved using NPS. One organization is about to launch an initiative that does link NPS to the agent. They will be using a product called Echo with their IVR system. It will include the likelihood to recommend question in their survey. They conduct about 360 completed interviews per agent on an annual basis, totaling approximately 190,000 per year. They will ask the customer satisfaction questions in the same survey, as well as FCR related questions. There will be 3 agentbased questions plus a marketing question and the likelihood to recommend question. As a follow up to the likelihood to recommend question, they will be asking customers why they would recommend or why not. This will be included in the voice capture part of the system. The plans for implementation are the 4 th quarter of this year. The actual performance based portion will roll out in the first quarter of 08. The quality and coaching team will be assessing the percentage allotment for this portion on the total performance scorecard. They want to view the results and ensure that they understand the information in terms of the behaviors or activity that the agent could have performed differently to impact that score. Another organization shared that they are going to be trying this as well and will also be doing so in Q1 of The conference call participants included: Frank Haute, Amazon Bettina Stix, Amazon Bryan Anderson, Amcore Bank Sam Makki, Bell Canada Kenny Hunter, Best Buy Jason Flanagan, BT Financial Jason, BT Financial Kim Zuk, Ceridian Canada Nancy Fung, Citifinancial Retail Services Jim Rembach, Customer Relationship Metrics Bill Daniels, ING Direct Mark Deep, ING Direct

4 Neal Dlin, ING Direct Kiersten Dempster, Pizza Pizza Mark Pellow, Reuters Lisa, Reuters Rob Stanbrook, St. George Bank Don Hale, TD Canada Trust Sandy Wildenborg, TDS Metrocom Bob Moore, TDS Telecom Scott Rennie of Rennie Research Group moderated the call. Key Learnings This section explores the key objectives of the call, namely to identify the problem(s), suggest potential solutions and explore the lessons learned. 1. What is the Problem? What are the pros and cons of using the Net Promoter Score? 2. What are the potential solutions? Nine key elements were identified: i) NPS is still a new term without enough history to determine the underlying drivers; As shared by one member who has been tracking the NPS for the past 6 months, they are trying to determine if it is truly a better measure of customer advocacy. It is a tougher hurdle than just measuring customer satisfaction. Everyone is familiar with customer satisfaction, but NPS is a new term and they don t understand the underlying drivers. Without knowing those drivers, how can an organization improve that score? Adding to this is trying to understand what this score really means to their business;

5 At the moment, this organization is considering measuring both NPS and customer satisfaction. It is a question of which measure to use as the lead indicator in a scoreboard and throughout the business. ii) The NPS is more volatile than the customer satisfaction metric; As noticed by one organization, the NPS fluctuates much more dramatically than their overall: How would you rate the company? score. The results to that question do not fluctuate as much as the NPS (no more than a point and a half per year). In the past year, they ve gone anywhere from a minus 7 to a positive 21 with NPS on a monthly basis. As a result, they are a little apprehensive to role this out as a corporate number. They do appreciate the philosophy behind the score, but are just not as keen on the calculation itself; The question was asked of this same participant to comment on the relationship between the NPS and the overall satisfaction, i.e. if the NPS goes up, does satisfaction also increase? The reply was absolutely ; Another organization who has been using NPS since April, 2007 stated that they have not experienced any variance in their scores. They survey almost 1000 customers per month, both clients and prospects who call the center. However, it has only been a short time that this measurement has been in place; Yet one more organization came forward to offer that they, too, have not noticed a fluctuation in the scores. This organization has been using NPS company-wide for about 3 months now in conjunction with their customer satisfaction scores. They are not noticing any movement in their numbers from month-to-month. Now, they are only sampling about 100 or 200 individuals or merchants each month. What they do notice is that there is a correlation between their top box scores. When the customer satisfaction top box score is high, the net promoter scores are in the 50 s and 60 s. When the top box is lower, the numbers also drop to the teens; Another participant who uses NPS (but without the 10-point scale) stated that they do see the volatility on a monthly basis. Their scores have ranged from about a 10-point spread over the last year, but they do not see that in their top box CSI scores at all. On a year-todate average, they are very similar and very closely tied with what is happening with the CSI top box scores. As stated, they conduct about 1000 surveys per month and when

6 you start drilling down to smaller numbers, obviously there will be more fluctuation within some of the centers on a month-to-month basis. iii) The Net Promoter score can be deceiving without a standard sampling of customers; Dr. Monger of Customer Relationship Metrics wanted to bring to the attention of the participants that it is imperative that one understands how the scores technically compare with each other. Also, what made up one score compared to another could be particularly troublesome. For example, if you have 50% of your customer base in the promoter category and 40% in the detractor category, that s a 10% NPS. However, having 40% of customers that are detractors is much more significant an issue than another organization that also has a 10% Net Promoter Score, but derived that calculation from 20% promoters and 10% detractors. Also, take into consideration that there is a large group of people in the middle that are basically not accounted for in this score. So, when you are looking at fluctuation and scores, certainly look at that as an issue, but really look at what is making up that NPS. In the end, you have one number, but how that number was calculated is just as significant and cannot be ignored; The true scale that is recommended by Reichheld is a scale of zero to 10. So, for that middle group on a scale of zero to 10, there will be a group that gave 7s and 8s that are not part of the calculation. The detractors have a huge influence on an organization s core business. Dr. Monger again emphasized that the fact that you can have the same NPS score with a very large detractor group compared to a smaller group is an incongruity for which every organization needs to be aware; Further information on this is available through a recent article in The Journal of Marketing, July, There are some very interesting discussions around this NPS phenomenon. There s not as much data in academia to support the NPS as a score and this article highlights some of the issues for which organizations should be made cognizant. iv) Using the NPS score can provide the organization with more granular customer data and discern brand momentum;

7 For the participant that has been using NPS since April of this year, one of the reasons that they went to this type of score was because of the granularity of the feedback received using a 10-point scale vs. what was being used in the past. They found that when customers were given fewer choices, 90 plus percent of their clients would recommend them. So, they decided to go to the 10-point scale because those that rate 9 s and 10 s are promoters of the company brand, which is good for word of mouth and other marketing type needs. The lower scores are more likely to be detractors, potentially bad mouthing the brand. So, this company is using this information as a good leading indicator for brand momentum, as well as a single metric that the organization can measure and rally around. v) It can be argued that the Net Promoter Score provides more of a brand satisfaction measurement than an actual customer satisfaction measurement; Dr. Monger stated that this is a discussion that they ve had on the table in terms of how this measure should be introduced. It is undecided whether the measure should be used at more of a corporate level because the brand heavily influences the perception for recommendation of the organization on NPS. At a contact center level, however, it doesn t really measure how well the company has actually provided a service to the customer; With his net promoter concept, Fred Reichheld is basically saying that this is the single most reliable indicator of a company s ability to grow. Those are his words. He is looking at this as the only metric needed to run a business. However, those in the contact center have questions. At a corporate strategic level it might be fine, but in order to explain why things are happening in the contact center there needs to be more than one metric. Dr. Monger wanted to make it clear that she is not bashing the net promoter concept. However, she has been asking the likelihood to recommend question for the past 20 years. It is not a new question. What is new is the concept that Reichheld has put forward that this is the single most reliable indicator, which translates to this being the only question needed. That has then been spun into this quick fix method for organizations to obtain their measurements so that they can expediently move on to the next management issue. Dr. Monger wanted to make it clear that there is not a single, simple solution that fits all companies and cautions organizations against that belief. As a matter

8 of fact, Reichheld has made an addendum to this statement by adding that it is necessary to follow up the likelihood to recommend question with further questions that ask the customer why they have chosen the answer that they did. Research requires qualitative information that can be coded and analyzed. vi) There is some customer emotion tied to the Net Promoter Score, which is fickle. As a result, it can be hard to defend the numbers when challenged; Interestingly, one member was able to bring forward an anomaly in their customer feedback data. They have the ability to ask customers their likelihood to re-purchase and likelihood to recommend and then follow-up to see if the customer s behavior corresponds to what they stated in the survey. They track every single purchase that a customer makes, using the contact center interaction as the trigger. They review the purchase behavior of the customer 6 months post-contact and 6 months pre-contact and compare that to a control group. What they have seen in their data is that there is an extremely high correlation between the answers provided from both of those 2 questions. They have noticed that those customers who say that they would not re-purchase do, in fact, come back and re-purchase. For this participant, this makes him question the methodology of promoters and distracters; Dr. Monger offered that it is all about the emotion. When any contact center executive stands up to defend his/her numbers, they want to speak with confidence in the numbers and back it up with solid data. If you have to keep explaining the numbers, to her it undermines the importance of what this metric is really intended to do, i.e. to really be a strategic indicator of the health of an organization. Again, she emphasized that she is not telling organizations not to use this score, but to exercise caution and use additional research so that the fluctuations can be explained with legitimacy. vii) Media coverage, good or bad, can impact NPS; Participants discussed this factor and its impact on the NPS. As shared by one participant, he does see a little bit of a correlation. NPS is a little bit more brand-centric.

9 An example was with recent media events that highlighted bank fees and government focus on those fees. With this media coverage on the industry as a whole, they did see a dip in their net promoter score. However, the CSI score didn t change at all; Another participant added that from an organizational point of view, they use NPS. However, when they are drilling down to more of an operations or contact center level, that is one of their biggest issues because they do experience volatility with their centers across the globe. They have been using NPS since early 1995, but do wonder about the score s effect when there is something in the media. They don t want the media to promote or detract from their service. From a contact center point of view, they are struggling to bring it down to that level. viii) Additional questions should be added if there is an intermediary involved in selling the organization s product; An intermediary example is a financial planner, an advisor or a broker. If an organization would like to know what their customers are thinking, one participant shared that an idea they have tossed around is to add a question regarding the likelihood to recommend that particular agent/broker/planner/advisor. This is the organization that is implementing the Echo system. They did discuss the difference between asking the question companywide vs. likelihood to recommend the agent. That is also something that Echo suggests. If the customer would recommend the intermediary, then by proxy they are, in turn, recommending the company. ix) NPS does not have a benchmarking comparison available as yet, whereas the customer satisfaction measurement is widely used and available as a benchmark; Another caution with this relatively new NPS metric is that there are not yet any benchmark comparisons that can be used. Furthermore, there is not a standard formula for its calculation. As seen on this call, there are many different ways that organizations are calculating their individual scores, making comparisons impossible. Customer satisfaction benchmarking, however, is commonplace and can be achieved with

10 statistical validity. Dr. Monger stated that she is relieved that there is not a database out there with NPS scores for that very reason; there is no consistency. Reporting these types of scores when they have been calculated differently is very troublesome, particularly if someone is going to try and make a comparison of themselves to the industry; As elucidated by one participant, they are very clear about only using it as an internal measurement. 3. What were the lessons learned? For the organization that has been calculating the NPS since January of 2005, they have discovered that it has a similar impact as their customer satisfaction scores. They measure a range of things including the ACSI type calculation on the customer satisfaction questions, as well as a set of loyalty questions and an overall satisfaction rating question, which has been the standard. Overall, NPS ties pretty closely to the overall rating. In the discussion regarding the middle group in the overall calculation, the question was asked as to why this group is considered ambivalent. One reply was that for the actual score, an organization might be trying to distill it down to a single number. It is not to say that an organization will ignore that middle group, but the questions asked of them for follow up or the strategy used for this group will be different than the strategy used for the promoters or detractors. Dr. Monger cautioned organizations not to fall into the trap of believing that this is a simple, magic solution that is an easy way to fulfill the organization s measurement needs. Dr. Monger also warned that the score alone does not give a true picture. One company could have 40% detractors and another company have 20% detractors, but still arrive at a 10% NPS. The score is the same, but the organization with 40% detractors should be much more concerned than the other organization. Furthermore, there is not any consistency in how the scores are calculated. One participant shared that in comparing their NPS and customer satisfaction scores there is a correlation between their top box scores. When the customer satisfaction top box score is high, the net promoter scores are in the 50 s and 60 s. When the top box is lower, the numbers also drop to the teens.

11 Another philosophy promoted by Fred Reichheld is the good profits, bad profits philosophy. For example, if an organization charges fees for a service, but so do all their competitors, the customer is stuck because they do not have a choice. That customer might give the organization a bad net promoter score, but will still use the company because they don t have a choice. However, if another organization enters the marketplace and they do not charge service fees, the customer may jump ship quickly in favor of the other company. There would have to be some other service or feature that would keep the customer tied to the original company. If there is a high quality of service that comes with the fee, customers may not complain about paying it. Where customers become disgruntled is paying a fee for poor service.