The NPS and the ACSI: a critique and an alternative metric



Similar documents
Measurement Deficiencies in the Net Promoter Score Robert East, Kingston Business School, London. Abstract

Customer Satisfaction and the Success of Your Organization

WHITE PAPER: Quantifying the Impact of Customer Satisfaction on Business Performance

Facebook fans: A fan for life? Karen Nelson-Field and Jennifer Taylor Admap May 2012

The dangers of common sense Les Binet Market Leader Quarter 3, June 2009

Linking Customer Loyalty to Growth By By Timothy L. Keiningham, Lerzan Aksoy, Bruce Cooil and Tor Wallin Andreassen

An Empirical Investigation of Customer Defection & Acquisition Rates for Declining and Growing Pharmaceutical Brands

The Simple Way to Measure Customer Experience. First Briefing Paper

The 2013 Financial Institution Consumer Recommendation & Loyalty Study Advanced

on share price performance

The 10 Week Business Success Challenge

Brand Loyalty in Insurance Companies

Managing Customer Retention

Admap Research - The Case Study of BAV Brand Growth

In 50 Words Or Less Accelerating business growth depends on effectively measuring the three facets of customer loyalty related to retention,

The Case for Improving the B2B Customer Experience

The Case for Improving the B2B Customer Experience

What You Should Know About the Net Promoter Score NPS

How Australia s utilities can boost customer loyalty

Maximizing Your Customer Experience Management Metrics

Grounded Benchmarks for Item Level Service Quality Metrics. Michael Vogelpoel, Anne Sharp, University of South Australia

CUSTOMER SATISFACTION SURVEY POINT Research and Marketing Consultancy

Assessing the Economic Value of Making the Right Customer Satisfaction Decisions and the Impact of Dissatisfaction on Churn

provide a summary of the previous meetings discussions on this issue;

Market Seasonality Historical Data, Trends & Market Timing

PARETO SHARE IN CUSTOMER KNOWLEDGE BASED BRAND KNOWLEDGE. Jenni Romaniuk and Byron Sharp University of South Australia.

Study into the Sales of Add-on General Insurance Products

The 2013 Superannuation Consumer Recommendation & Loyalty Study

Software User Experience and Likelihood to Recommend: Linking UX and NPS

Overlapping ETF: Pair trading between two gold stocks

Navigating the Alphabet Soup of Survey Methodologies. David Jackson CEO, Clicktools

A-LEVEL BUSINESS Paper 3 Specimen Assessment Material. Mark scheme

A Study on Customer Satisfaction in Mobile Telecommunication Market by Using SEM and System Dynamic Method

GE Capital The Net Promoter Score: A low-cost, high-impact way to analyze customer voices

Trends in Corporate Climate Change Governance

A B2B PERSPECTIVE: GETTING MORE FROM YOUR NET PROMOTER SCORE*

THE IMPORTANCE OF BRAND AWARENESS IN CONSUMERS BUYING DECISION AND PERCEIVED RISK ASSESSMENT

The Heart of the Matter: The Challenge of Customer Lifetime Value

CEM+ Customer experience survey. What drives customer loyalty and advocacy for your brand? Introduction

An Evaluation of Cinema Advertising Effectiveness

Assessing Employee Satisfaction at the Zimbabwe Open University

How Hedging Can Substantially Reduce Foreign Stock Currency Risk

BRAND EQUITY AND BRAND SURVIVAL: EVIDENCE FROM AN EMERGING WINE REGION

Measuring the success of a managed volatility investment strategy

The Effect of Switching Barriers on Customer Retention in Korean Mobile Telecommunication Services

IJMT Volume 2, Issue 9 ISSN:

CEM+ Calculator HOW DO YOU PREDICT THE ROI OF CUSTOMER EXPERIENCE? Introduction

Principles of Measuring Advertising Effectiveness

Improving Customer Satisfaction to Accelerate Your Business Results

Usefulness of expected values in liability valuation: the role of portfolio size

Assessing energy supply profitability: does a margins approach make sense?


Morningstar Core Equities Portfolio

Integrated channel planning: Effective integration Kate Cox Admap June 2011

Volatility: Implications for Value and Glamour Stocks

Chapter component of the convertible can be estimated as =

Asking the "tough questions" in choosing a partner to conduct Customer Experience Measurement and Management (CEM) programs for Your Company

Exceptional Customer Experience AND Credit Risk Management: How to Achieve Both

Private Equity Performance Measurement BVCA Perspectives Series

Index trackers and cluster munitions

for Analysing Listed Private Equity Companies

Customer relationship management MB-104. By Mayank Kumar Pandey Assistant Professor at Noida Institute of Engineering and Technology

Theories of Exchange rate determination

Schroders Investment Trust ISA

Exercise 7.1 What are advertising objectives?

2016 Interim Results Presentation 24 November 2015

Mental-accounting portfolio

DELIGHTFUL OR DEPENDABLE? VARIABILITY OF CUSTOMER EXPERIENCES AS A PREDICTOR OF CUSTOMER VALUE

Why Use Dashboard Metrics? Because the right metrics matter. RKM Research and Communications, Inc., Portsmouth, NH. All Rights Reserved.

Lead Scoring. Five steps to getting started. wowanalytics. 730 Yale Avenue Swarthmore, PA

Resetting digital strategy in Australia: Delivering what customers really want

How Integrated Marketing Communications (IMC) Can Build Strong Brand Equity?

Fundamentals Level Skills Module, Paper F9

How To Understand The Growth In Private Health Insurance

Valuation of Your Early Drug Candidate. By Linda Pullan, Ph.D. Toll-free USA Worldwide

ON THE RISK ADJUSTED DISCOUNT RATE FOR DETERMINING LIFE OFFICE APPRAISAL VALUES BY M. SHERRIS B.A., M.B.A., F.I.A., F.I.A.A. 1.

Do loyalty Programs Build a Long-term Relationship with customers? A Banking Perspective. Zeynep Roberts and Leanne Carter Macquarie University

Investing on hope? Small Cap and Growth Investing!

Moving the NPS Needle - How to Use Customer Feedback to Drive Improvement

COURSE SYLLABUS. Return on Marketing Investment

consumerlab Keeping Smartphone users loyal Assessing the impact of network performance on consumer loyalty to operators

Brand metrics: Gauging and linking brands with business performance

Financial Planning Newsletter

Does an Independent Board Matter for Leveraged Firm?

Are you listening to your customer feedback? A Handy Guide to Net Promoter Score

Stock Market Dashboard Back-Test October 29, 1998 March 29, 2010 Revised 2010 Leslie N. Masonson

A Study Of Two Customer Retention Measures: The American Customer Satisfaction Index And The Conversion Model

Transcription:

Title: The NPS and the ACSI: a critique and an alternative metric Author(s): Robert East, Jenni Romaniuk and Wendy Lomax Source: International Journal of Market Research Issue: Vol. 53, No. 3, 2011 The NPS and the ACSI: a critique and an alternative metric Robert East Kingston Business School and Ehrenberg-Bass Institute, University of South Australia Jenni Romaniuk Ehrenberg-Bass Institute, University of South Australia Wendy Lomax Kingston Business School Introduction Customers who are satisfied may show greater retention and express more positive word of mouth (PWOM) about the brand, leading to customer acquisition and further sales. Thus, measures of satisfaction and word of mouth (WOM) may predict brand performance. The most established satisfaction metric is the American Customer Satisfaction Index (ACSI), developed by Fornell and his associates (Fornell 1992; Fornell et al. 1996). This measure has been shown to predict sales and profit but, in 2003, Reichheld claimed that a single measure of WOM, the Net Promoter Score (NPS), could provide a better prediction of brand performance than the ACSI (Reichheld 2003). Subsequent studies, which we review below, have not supported Reichheld s claim. Our own research finds weaknesses in the NPS; in particular, that the NPS is poor at measuring negative word of mouth (NWOM). We suggest that the ACSI is similarly poor at detecting dissatisfaction. Both the NPS and the ACSI are in worldwide use for company/brand appraisal, and Fornell et al. (2006) use the ACSI for investment decisions. Thus, any improvements in the measurement of WOM or satisfaction could be of value to both managers and investors. In this paper, we first review evidence on the predictive power of the NPS and ACSI, since this provides much of the motivation for research in this field. Then we discuss questions raised by the design of the NPS and ACSI. We show that ex-users and never-users, who are not questioned in the NPS procedure, give some PWOM and most of the recorded NWOM, and we show that those customers who Reichheld (2003) claims are responsible for NWOM actually express very little. We then describe a new WOM metric that could overcome the problems we have identified. Thus, the contribution of this paper is primarily to show that both the NPS and ACSI have weaknesses, which might be avoided by using a new alternative measure. We present evidence that our alternative metric has little correlation with the NPS and ACSI in two categories, showing that it is indeed a different measure. 2

The metrics and brand performance The measures The Net Promoter Score Reichheld (2003) designed the NPS to measure the effect of PWOM and NWOM on sales. To establish the NPS, customers report their likelihood of recommending their brand/firm on a 0 10 scale. Those scoring 9 or 10 are promoters of the brand, while those scoring 0 to 6 are detractors; those scoring 7 or 8 are passives. The NPS for the brand is the promoter percentage minus the detractor percentage. The American Customer Satisfaction Index Fornell et al. (1996) describe how the ACSI is measured using three questions about recently purchased brands that relate to overall satisfaction, expectancy disconfirmation and perceived performance compared with ideal performance. For practical purposes, the ACSI is the average of responses to the three items, though Fornell and his associates use different weights for specific predictions. When a firm markets several brands, a composite ACSI score for the firm can be derived from the scores for each brand. The ACSI items are part of a larger set of questions that cover complaints, loyalty, expectations, value and quality. Sales and profit The NPS and ACSI are important because they are used to predict brand performance, which may be sales or profit measured as return on assets, total price-to-book ratio and Tobin s Q (Tobin 1969). Reichheld (2003) showed an association between the NPS and sales in a correlational study in the US; Marsden, Samson and Upton (2005/6) reported a similar finding in the UK. Keiningham et al. (2007), using the three industries selected by Reichheld, found that the ACSI performed about as well as the NPS in predicting sales. Morgan and Rego (2006) constructed a measure in the NPS form, based on past recommendation, and found that this was not as effective as other measures, particularly the ACSI, at predicting later company performance. Gruca and Rego (2005) showed that a 1% increase in satisfaction was followed by a significant increase in the net operating cash flow. Pingitore et al. (2007) tested several measures as predictors of brand performance. They concluded that there were no grounds for preferring the NPS and that, on balance, satisfaction and loyalty measures performed better. Therefore, the claim by Reichheld (2003) that the NPS gives better brand performance predictions than satisfaction measures has not been supported. Share price What has excited researchers is the possibility that increases in the ACSI score may be used to pick stocks that are more likely than other stocks to rise in value. Two studies (Anderson et al. 1994; Anderson et al. 2004) have shown that increases in shareholder value are predicted by increases in customer satisfaction. In a more comprehensive study, Fornell et al. (2006) found that, by investing in a portfolio of companies scoring in the top two deciles of the ACSI in their sector, and above average across all sectors, it was possible to outperform stock market indexes by a substantial margin. This study was conducted over 1997 2004 but, subsequently, Fornell et al. (2009) reported that, up to October 2008, the investment portfolio had generated an average return of 15.1% compound in the previous eight years compared with 0.5% for the S&P 500. Aksoy et al. (2008) also find that changes in the satisfaction score predict stock value changes, but only when the economy is 3

expanding. The success of Fornell s portfolio suggests that share prices do not fully factor in the profit implications of consumer satisfaction. Two further studies assess whether the market really does fail to reflect customer satisfaction in the share price. These papers suggest that the circumstances underlying customer satisfaction are in the public domain at the time of measurement and should therefore already be reflected in the share price the efficient market assumption. Jacobson and Mizik (2009) argue that Fornell et al. s results are largely carried by two sectors in their portfolio. O Sullivan, Hutchinson and O Connell (2009) adjust for risk and transaction costs, and conclude that there is no compelling evidence that the market misprices customer satisfaction. Fornell et al. (2009) reply to these papers; they reject Jacobson and Mizik s point as a hindsight criticism, raise doubts about the significance testing employed by O Sullivan et al., and cite further work that supports their position. [1] Deficiencies in the measures and a comparison of given and received WOM To measure the effect of word of mouth on the sales of a brand, we need the volumes of PWOM and NWOM on the brand and the mean impact on purchase of instances of PWOM and NWOM. Then a metric can be constructed using the products of volume and mean impact for each type of WOM. We return to this measure later. Here, we consider whether the NPS and the ACSI do measure volume and impact. The volume of WOM A measure of WOM should tap the total volume of PWOM and NWOM circulating on each brand. The NPS provides a measure of the propensity to produce PWOM but contains no measure of NWOM. Instead, it is suggested that those who are unlikely to give PWOM will give NWOM. Reichheld (2006) states that detractors are responsible for 80% to 90% of a company s negative word of mouth, but he provides no evidence to support this claim. Those who are unlikely to give PWOM on a brand may be disinclined or have little opportunity to give any form of WOM. Furthermore, much NWOM may come from ex-customers. According to a study of five categories by East, Hammond and Wright (2007), half of all NWOM is given by exusers and 30% by never-users of the brand. Such never-users may have formed opinions about the brand through observation or through hearing the comments of others. The NWOM of ex-customers and never-customers will reduce customer acquisition and, as a result, have a negative effect on brand performance. Thus, an effective measure of WOM about Brand A should question consumers of the category rather than just current customers of Brand A. In ACSI research, respondents are asked about the brands that they have bought over a period ranging from a month (for frequently purchased goods) to three years (for durables). This provides data on brands in current use, but excludes brands purchased prior to the period. Some previously purchased brands are likely to have been abandoned before the review period because they caused dissatisfaction. Thus, the ACSI procedure fails to measure the dissatisfaction of these ex-customers as well as the sentiments of never-customers who may reflect the feelings of disaffected customers. This analysis indicates that both the NPS and the ACSI miss much of the negative sentiment expressed about a brand. The extent of this failure will depend upon the proportion of negative sentiment that is given by those who are not current customers. To clarify this matter, we extend the work of East et al. (2007) by investigating how PWOM and NWOM are shared between current, ex- and never-customers in a further ten categories. We also measure how much of the total PWOM and NWOM on a brand is given by the promoters and detractors of that brand. The impact of WOM 4

Impact is not assessed in the NPS. Indeed, it cannot be assessed because, to do this, a respondent must report on the effect of WOM that he/she has received and Reichheld s measure is about the inclination to give PWOM rather than about what has been received. This is a serious deficiency. A similar problem applies to satisfaction measures: satisfaction and dissatisfaction should be weighted by their effect on sales and no relative weighting can be used in the ACSI since dissatisfaction is not separately measured. We return to evidence on the relative impact of PWOM and NWOM in the Discussion section below. Given and received WOM Volume may be measured on both given and received WOM. There are arguments supporting the measurement of both forms. In favour of received WOM, a person may give advice to more than one other person, and this exchange will be recorded as one instance of given WOM but several instances of received WOM. The latter is preferable when we are trying to establish the total effect of WOM. In addition, we are interested in sampling the interpersonal exchanges that have an impact on purchase probability, and the advice that a person can recall receiving is likely to be closer to the point of impact than the advice that a person can recall giving. In favour of given WOM, there may be reinforcement effects from giving WOM that alter the purchase propensity of the giver. There is no direct evidence on this matter but Chandon et al. (2005) show that the act of expressing an intention raises the likelihood of performing the intended action; WOM expressions could have a similar effect. In addition, Cowley (2008) shows that people tend to distort their memory towards the content of their advice and that this is particularly the case when the advice is designed to be informative, which applies to WOM. This process could make a person who gives PWOM/NWOM on a brand more convinced about the merits/demerits of the brand, with consequent effect on their behaviour. A measure based on received WOM would not pick up these reinforcement effects. Research questions This review leads us to investigate two main research questions. RQ1: What proportions of given PWOM and NWOM are about the current brand, previously used brands and brands that have never been used? This will indicate the limitations of measures that only sample current customers of the brand. RQ2: What proportions of the total PWOM and NWOM are expressed by promoters and detractors? This will show whether promoters mostly produce PWOM and detractors mostly produce NWOM; if these components of the NPS do not discriminate in this way, the measure is flawed. After evidence on these questions, we propose a measure for the combined effect of PWOM and NWOM, and report on some preliminary comparisons between this measure, the NPS and the ACSI. Method Surveys and questionnaires The categories, method, sample size and response rates of our surveys are shown in Table 1. We used convenience samples. In most cases, we delivered questionnaires to respondents homes and collected them later. 5

Deliveries were to a number of different areas to reduce any location effects. All studies were one-wave, and no incentives were used. In some cases, data on two categories were gathered in one questionnaire and these are reported as two studies. Coffee shops and banking services were each studied twice, but in different years and locations. This gave ten studies. In all cases, respondents were asked if they had received and given positive and negative advice in the last six months on any brand in the specified category (items 12, 15, 20 and 21 in the Appendix). Where advice had been given, we established whether the brand was the current main brand, a previously used brand or never used (items 14 and 22 in the Appendix). In three categories, additional questions were asked so that the NPS, ACSI and our alternative measure could be compared; a sample questionnaire in the Appendix shows the items that we used. Normally, the NPS scale is unlabelled but we use a labelled 0 10 Juster measure (Juster 1966), which has been shown to agree well with objective probabilities (Wright & MacRae 2007). This labelling may have a small effect on brand scores, but is unlikely to make much difference to comparisons between metrics. Findings In total, there were 2254 survey respondents. Of these, 1113 reported receiving and 1152 reported giving PWOM; for NWOM, 787 stated that they had received and 645 that they had given NWOM. The correlations by brand for received and given PWOM and NWOM volume are shown in Table 1. These correlations are quite high (means 0.78 for PWOM and 0.63 for NWOM), and indicate that volume measures based on received WOM will give similar results to those based on given WOM. What proportions of given PWOM and NWOM are about the current brand, previously used brands and brands that have never been used? Table 2 shows whether the brand cited in given WOM was the respondent s current main brand, a brand used previously, or 6

never used. In Table 2, we have inserted findings (shown by b ) from East et al. (2007), so that the table covers all known cases. The pattern for PWOM is quite consistent. In every case, the respondent s current main brand is given the most PWOM, though this is somewhat lower for categories where multi-brand usage is common (except coffee shops). The pattern for NWOM is also quite consistent; in 13 out of the 15 cases, the greatest percentage of NWOM is about a previous brand. The row at the base of Table 2 shows the means; 71% of PWOM is about the advice giver s current brand and 77% of NWOM is about brands other than the advice giver s current brand. Respondents reported that they were much more likely to give NWOM than PWOM on brands that they had never used (22% versus 7%). What proportions of the total PWOM and NWOM are expressed by promoters and detractors? We answer this question using the three studies on supermarkets, coffee shops and skin care products where additional data were gathered. The NPS question (item 2 in the Appendix) was used to classify respondents as promoters (9 or 10), passives (7 or 8) or detractors (6 or lower). We measured the total PWOM and NWOM that respondents claimed to have given in the last six months on both their main brand and other brands (items 12 and 20 in the Appendix). We then computed the proportion of this total that was expressed by the promoters and detractors. For example, promoters expressed 234 units of PWOM on their main supermarket out of the total 734 units of PWOM expressed on this supermarket, so promoters were responsible for 32% of the PWOM circulating on their main brand, while 68% was expressed by passives, detractors and the main users of other supermarkets. In six instances, respondents claimed to have given more than 10 instances of PWOM or NWOM; these outlier numbers were reduced to 10, though this had little effect on the results. Table 3 shows that promoters are responsible for less than half the total PWOM, but they give little NWOM so promoters numbers indicate the scale of PWOM. However, when we turn to detractors, we find that they are responsible for both NWOM and PWOM. In the case of the supermarkets, 31% of the total 7

NWOM and 13% of the total PWOM comes from detractors and, for the other two categories, detractors give more PWOM than NWOM, so detractor numbers cannot be regarded as a measure of NWOM. We conclude that the NPS provides some measure of PWOM but does not reflect NWOM. We also measured respondent satisfaction with the main brand using the ACSI items 4, 5 and 6 (in the Appendix). We used the ACSI scores to define quasi-promoters and quasi-detractors. The three 1 10 satisfaction measures were summed and, to select reasonably sized proportions, 24 30 were allocated to the quasi-promoters and 3 17 to quasi-detractors. Table 4 shows a similar pattern to Table 3. In the ACSI measurement, data are gathered on recently used brands; this would often be more than one supermarket brand per respondent and this could show more evidence on brands that are unsatisfactory. We gathered data only on the main brand used and, therefore, our findings will be more typical of categories where customers normally use only one brand. The net effect of word of mouth (NEW) Our evidence suggests that the NPS does not provide adequate measurement of NWOM and, because of the way it is restricted to customers of recently used brands, we conclude that the ACSI is similarly poor at capturing dissatisfaction. We also note that the NPS lacks a measurement of the impact of WOM. As we indicated earlier, the total effect of WOM should be established by the separate measurement of the volume and impact on purchase of PWOM and NWOM. We now set out a formula combining these measures. For each brand, we work out the net effect of WOM (NEW) as: [(volume impact) PWOM + (volume impact) NWOM ]/market share We use the sum because the impact of NWOM is negative, so (volume impact) NWOM will be a negative number and thus reduce purchase likelihood. The sum of products will relate to total sales effect so we divide by the market share indicated by respondents to give the effect on market share. In the exploratory test that follows, the volume is measured by respondent estimation of the number of instances of PWOM/NWOM received and given in a six-month period and the impact is measured as a shift in purchase intention in response to received WOM, using the scale developed by Juster (1966). In this type of measurement, there is a lack of data on small brands since, by definition, these brands are rarely bought. In 8

consequence, one or two respondents giving outlier responses on the volume of PWOM or NWOM could produce substantial sampling error. When it is impossible to gather large quantities of data, simplifications are necessary. First, we may reduce sampling error by reducing outlier values. Second, we may assume that the mean impact of PWOM and the mean impact of NWOM are each the same for all brands in the category. (Note: we do not assume that the impact of PWOM is the same as the impact of NWOM.) Third, we may remove small brands from the analysis. We have used these simplifications in our exploratory comparison of alternative metrics now reported. What are the correlations between alternative metrics? To justify further development of the NEW measure, we have to show that it differs from the NPS and ACSI measures. We used item 2 in the Appendix to establish the NPS, and the mean of items 4, 5 and 6 to establish the ACSI. The PWOM component was established with items 7 10, 12 and 13, and the NWOM component with items 15 18, 20 and 21; these questions gave volume of received WOM, impact and volume of given WOM, and the two volume measures were averaged. We applied the simplifications described in the previous section, added the PWOM and NWOM components to form the NEW metric, and produced the correlation in Table 5. With only four brands, significant correlations occur in Table 5 only when there is close correspondence but we are interested in a lack of correlation since this indicates that the measures are dissimilar. From Table 5 we see that the ACSI is quite closely related to the NPS in all three cases. The ACSI and NPS measures correlate highly with NEW in the case of supermarkets, but not in the case of coffee shops and skin care products. We regard this lack of correlation as promising, justifying further work. Discussion Overview In this paper, we show that most NWOM is given by those who are ex-customers or never-customers. By contrast, most PWOM is given by current customers. As a consequence, metrics based only on current customers, such as the NPS, do not measure NWOM effectively. We show that detractors give little of the total NWOM on the brand and that, in two out of the three categories that we studied, detractors were responsible for more PWOM than NWOM. Similar patterns are found in an analysis based on the ACSI measures, which suggests that the NPS and ACSI are closer than their respective proponents are willing to claim. [2] We propose the NEW word-of-mouth metric that measures both PWOM and NWOM among users of the category. In this respect, it has some affinity with a measure proposed by Samson (2006), which combined the NPS with a measure of NWOM, 9

though Samson s metric lacked the measurement of impact that we incorporate. To justify further development of the NEW measure, we must show that it differs from the NPS and ACSI measures. The associations in Table 5 show substantial differences in two of the three categories, thus warranting further research. Measurement concerns The NEW metric is based on a logical analysis of the measurement problem, but some refinement of the measures may be needed. When WOM volume is assessed over a long period (in our case six months), it is likely that respondents provide an estimate since people will not be able to remember all conversations containing WOM on a category. A shorter period could give recall that is more accurate. However, if the period is too short there will be few instances of WOM and large numbers of respondents will be needed. An alternative is to ask about the last recommendation of the category that was received and how long ago this was; the reciprocal of the mean time lapse would provide a weighting for the volume estimate. This type of measure could also show recent changes in the rate of production of WOM on a brand. We also suggest that alternative measures of impact are possible. The change in the probability of purchase seems appropriate, but it might be argued that it is the number of final decisions produced by WOM that is more relevant. However, we then have the problem of weighting a decision not to buy against the decision to buy; a negative decision may not settle the choice since a respondent may be considering three or more options. Another measurement problem occurs if there is differential recall of PWOM and NWOM so that one component of the measure is consistently over- or under-measured relative to the other component. When prediction studies can be conducted, the separate contribution of the positive and negative components of the measure can be tested and correcting weights can then be established. A further measurement concern is that PWOM about good value may impact more on sales than profit since margins tend to be reduced when a product is low-priced. A converse point can be made on NWOM about high price. It is possible to remove price-related WOM from the measure and we show relevant questions in the Appendix (items 11 and 19). In further work, comparisons should be made between this depleted measure and the full NEW measure to see whether price-related WOM has different brand and share performance implications. Applications This work stops short of predictive tests that compare the NEW measure with established measures. Predictive tests need large samples if small brands are to be measured adequately, and the measurement should extend over substantial periods and cover many categories. This sort of work may be conducted by market research organisations. We believe that our analysis of deficiencies in existing measures, together with our new evidence, indicates that an improved consumer-based (rather than customer-based) metric could be developed. One reason for developing consumer-based metrics of WOM is that they may serve as an investment tool. Fornell et al. (2006) have produced credible evidence that such metrics can provide a basis for investment. However, so far we have no effective prediction of downward movement of shares from a consumer-based metric. Such a metric would assist arbitrage investments where share purchases are balanced by short selling of shares that are expected to fall. Such shares are not necessarily those of companies that give most dissatisfaction. O Sullivan et al. s (2009) data indicate that investment in low-satisfaction companies may be very profitable, indicating that investors overreact to companies causing dissatisfaction. 10

PWOM versus NWOM We have not discussed the relative contribution of PWOM and NWOM to the performance of the brand. Lack of measurement of NWOM in the NPS will be less important if NWOM has little effect on performance. The effect of NWOM depends on the relative volumes and impacts compared with PWOM. Volume measures show that more PWOM than NWOM is produced though the ratio varies with the category. In our data, the mean volume ratios, PWOM/NWOM, were 1.4 for received and 1.8 for given. Romaniuk (2007) finds about 4 for television programmes, Chevalier and Mayzlin (2003) approximately 2 for online book reviews, and East et al. (2007) about 3 averaged across a range of categories. Arndt (1967) assessed recently expressed WOM about a new grocery product and found a ratio of 8, which was also found by the Keller Fay group, using recall of product related conversations in the last 24 hours (Rosen 2009). These are category ratios; at the brand level there is greater variability, and East et al. (2007) found that brands occasionally show an excess of NWOM over PWOM. Turning to impact, East et al. (2008) find that NWOM usually has somewhat less impact than PWOM, while Ahluwalia (2002) finds that positive and negative information have much the same impact in experimental research when the brands are familiar. Other studies suggest that negative measures are quite strongly predictive. Morgan and Rego (2006) find that sales growth and Tobin s Q are negatively related to the level of complaints; Luo (2007) finds that customer complaints are associated with reduced stock returns. Marsden et al. (2005/6) find that NWOM gives the best prediction of sales effect. This leaves the relative effect of PWOM and NWOM unclear. The greater volume of PWOM suggests that this is more powerful, but this is tempered by uncertainty about relative impact. Research with the NEW metric may clarify this matter. Conclusion We show that the NPS and the ACSI do not measure negative sentiments about brands effectively. We suggest the form of a WOM metric that could provide measurement of the effect of both PWOM and NWOM on brand performance, and show that this metric differs from the NPS and the ACSI. 11

12

13

References Ahluwalia, R. (2002) How prevalent is the negativity effect in consumer environments? Journal of Consumer Research, 29, 2, pp. 270 279. Aksoy, L., Cooil, B., Groening, C., Keiningham, T.L. & Yalcin, A. (2008) The long-term stock market valuation of customer satisfaction. Journal of Marketing, 72, 4, pp. 105 122. Anderson, E.W., Fornell, C. & Lehmann, D.R. (1994) Customer satisfaction, market share, and profitability. Journal of Marketing, 58, 3, pp. 53 66. Anderson, E.W., Fornell, C. & Mazvancheryl, S.K. (2004) Customer satisfaction and shareholder value. Journal of Marketing, 68, 4, pp. 172 185. Arndt, J. (1967) The role of product-related conversations in the diffusion of a new product. Journal of Marketing Research, 4, 3, pp. 291 295. Chandon, P., Morwitz, V.G. & Reinartz, W.J. (2005) Do intentions really predict behavior? Self-generated validity effects in survey research. Journal of Marketing, 69, 2, pp. 1 14. Chevalier, J.A. & Mayzlin, D. (2003) The effect of word of mouth on sales: online book reviews. Journal of Marketing Research, 44, 3, pp. 345 354. Cowley, E. (2008) The perils of hedonic editing. Journal of Consumer Research, 35, 1, pp. 71 84. Dimson, E., Marsh, P. & Staunton, M. (2004) Low-cap and low-rated companies. Journal of Portfolio Management, Summer, pp. 1 12. East, R., Hammond, K. & Lomax, W. (2008) Measuring the impact on brand purchase probability of positive and negative word of mouth. International Journal of Research in Marketing, 25, 3, pp. 215 224. East, R., Hammond, K. & Wright, M. (2007) The relative incidence of positive and negative word of mouth. International Journal of Research in Marketing, 24, 2, pp. 175 184. Fornell, C. (1992) A national customer satisfaction barometer: the Swedish experience. Journal of Marketing, 56, 1, pp. 6 21. Fornell, C., Johnson, M.D., Anderson, E.W., Cha, J. & Bryant, B.E. (1996) The American Customer Satisfaction Index: nature, purpose, and findings. Journal of Marketing, 60, 4, pp. 7 18. Fornell, C., Mithas, S. & Morgeson, F.V. (2009) The statistical significance of portfolio returns. International Journal of Research in Marketing, 26, 2, pp. 162 163. Fornell, C., Mithas, S., Morgeson, F.V. & Krishnan, M.S. (2006) Customer satisfaction and stock prices: high returns, low risks. Journal of Marketing, 70, 1, pp. 3 14. 14

Gruca, T.S. & Rego, L.L. (2005) Customer satisfaction, cash flow, and shareholder value. Journal of Marketing, 69, 3, pp. 115 130. Jacobson, R. & Mizik, N. (2009) Financial markets and customer satisfaction: re-examining possible financial market mispricing of customer satisfaction. Marketing Science, 28, 5, pp. 820 825. Juster, F.T. (1966) Consumer buying intentions and purchase probability: an experiment in survey design. Journal of the American Statistical Association, 61, 315, pp. 658 696. Kahneman, D. & Tversky, A. (2000) Choices, Values and Frames. New York: Russell Sage Foundation. Keiningham, T.L., Cooil, B., Andreasson, T.W. & Aksoy, L. (2007) A longitudinal examination of net promoter and firm revenue growth. Journal of Marketing, 71, 3, pp. 39 51. Luo, X. (2007) Consumer negative voice and firm-idiosyncratic stock returns. Journal of Marketing, 71, 3, pp. 75 88. Marsden, P., Samson, A. & Upton, N. (2005/6) Advocacy drives growth. Brand Strategy, 198, Dec/Jan, pp. 45 47. Morgan, N.A. & Rego, L.L. (2006) The value of different customer satisfaction and loyalty metrics in predicting business performance. Marketing Science, 25, 5, pp. 426 439. O Sullivan, D., Hutchinson, M.C. & O Connell, V. (2009) Empirical evidence of the stock market s (mis)pricing of customer satisfaction. International Journal of Research in Marketing, 26, 2, pp. 154 161. Pingitore, G., Morgan, N.A., Rego, L.L., Gigliotti, A. & Meyers, J. (2007) The single-question trap. Marketing Research, 19, 2, pp. 9 13. Reichheld, F.F. (2003) The one number you need to grow. Harvard Business Review, 12, pp. 46 54. Reichheld, F.F. (2006) The microeconomics of customer relationships. Sloan Management Review, 47, 2, pp. 73 78. Romaniuk, J. (2007) Word of mouth and the viewing of television programs. Journal of Advertising Research, 47, 4, pp. 462 471. Rosen, E. (2009) The Anatomy of Buzz Revisited. New York: Doubleday Publishing Group. Samson, A. (2006) Understanding the buzz that matters: negative vs positive word of mouth. International Journal of Market Research, 48, 6, pp. 647 657. Tobin, J. (1969) A general equilibrium approach to monetary theory. Journal of Money, Credit, and Banking, 1, 1, pp. 15 29. Wright, M. & MacRae, M. (2007) Bias and variability in purchase intention scales. Journal of the Academy of Marketing Science, 35, 4, pp. 617 624. About the authors Robert East is Professor of Consumer Behaviour in the marketing department of Kingston Business School, London. He trained as a social psychologist and is a postgraduate of London Business School. His research focuses on consumer loyalty, 15

service switching and word-of-mouth patterns. Robert East is the lead author of an evidence-based textbook, Consumer Behaviour: Applications in Marketing (2008) London: Sage. Jenni Romaniuk is an Associate Research Professor of Brand Equity and Associate Director (International) at the Ehrenberg- Bass Institute, based at the University of South Australia. Her current research interests are brand equity metrics, brand salience and advertising effectiveness. In addition to her work with corporations such as Unilever, Colgate Palmolive, ITV, Mars and The Edrington Group, she has been awarded funding from the Australian Research Council to research the influence of word of mouth and advertising on how viewers learn about new television programmes. Wendy Lomax moved to Kingston Business School after a period in the industry and later became Head of Marketing. Her doctorate at London Business School examined the shifts in market shares following new entrants to a market. Subsequently she has published extensively in the field of word of mouth. Address correspondence to: Robert East, Kingston Business School, Kingston, KT2 7LB, UK. Email: R.East@kingston.ac.uk [1] There is evidence that markets are inefficient, e.g. value stocks are consistently undervalued by the market compared with growth stocks (Dimson et al. 2004). There are also many anomalous findings documented by Kahneman and Tversky (2000) which suggest that markets will show biases based on mechanisms of human information processing. On this evidence, the search for investment strategies that capitalise on market inefficiencies seems worthwhile. [2] The similarity between the NPS and ACSI caused us to check the correlations between scales at the individual level. These were close. The three scales of the ACSI had an average intercorrelation of 0.58 across the three categories, while the NPS scale correlated 0.55 on average with these three scales. Copyright Warc 2011 Warc Ltd. 85 Newman Street, London, United Kingdom, W1T 3EX Tel: +44 (0)20 7467 8100, Fax: +(0)20 7467 8101 www.warc.com All rights reserved including database rights. This electronic file is for the personal use of authorised users based at the subscribing company's office location. It may not be reproduced, posted on intranets, extranets or the internet, e-mailed, archived or shared electronically either within the purchaser s organisation or externally without express written permission from Warc. 16