Developing inventories for satisfaction and Likert scales in a service environment

Transcription

1 Developing inventories for satisfaction and Likert scales in a service environment by Contact Author: Karin Braunsberger, Ph.D. Associate Professor of Marketing University of South Florida St. Petersburg College of Business 140 Seventh Avenue South Bayboro Station 306 St. Petersburg, FL Telephone: (727) Fax: (727) braunsbe@stpt.usf.edu Roger Gates DSS Research 6750 Locke Avenue Ft. Worth, TX Telephone: (817) Fax: (817) rgates@dssresearch.com 1

2 Abstract Purpose: To produce up-to-date inventories for satisfaction and Likert scales that contain commonly used scale point descriptors and their respective mean scale values and standard deviations. Methodology/Approach: All data were collected online using the SSI Survey Spot Panel. The panel is national (U.S.) in scope and was screened to include individuals years of age. A random sample was drawn. Thirty-nine satisfaction items and 19 agreement items were tested, and the mean value and the standard deviation were calculated for each of these descriptors. Findings: Even though only six of the items that had been tested by Jones and Thurstone (1955) were included in the list of satisfaction scale descriptors, the semantic meanings of those six have changed very little over the years. Research limitations/implications: One limitation of the current study might be the chosen service context, since scale point descriptor inventories developed within the context of health insurance might not be valid in other service contexts. Practical Implications: Since the present study focuses on two types of scales that are frequently used in service environments, namely Likert and satisfaction scales, the major contribution of this study is to provide researchers and managers in services marketing with quantitative measurement of the meanings of commonly used scale point descriptors, which as pointed out by Myers and Warner (1968) will make possible the development of equal interval scales and thus aid analyses of data sets. It will thus help service marketers to develop questionnaires that more accurately reflect actual consumer satisfaction and opinions. Keywords: Satisfaction scale inventory, Likert scale inventory. Paper Type: Research paper. 2

3 Developing inventories for satisfaction and Likert scales in a service environment 1. Introduction Researchers and managers in services marketing are often concerned with assessing customer satisfaction and opinions (Bearden, Malhotra and Uscátequi, 1998). When developing questions to assess satisfaction it has been strongly suggested that the end points of preference response scales should be words or phrases that denote bi-polar extremes, and that all anchoring points should be suitably spaced along the semantic continuum connecting the end points (Jones and Thurstone, 1955). Jones and Thurstone (1955) further express the need to investigate the semantic properties of commonly used scale point descriptors to make sure that they possess the above properties and also carry meaning that is as clear as possible to subjects that represent the researcher s population of interest. Further, knowing the exact scale value of each scale point descriptor is of importance when constructing successive-interval type of scales. Consequently, Jones and Thurstone (1955) examine the semantic meanings, to respondents, of 51 scale point descriptors using 9-point scales and subsequently present the research community with a listing of words and phrases that range from those expressing greatest like to those conveying the greatest dislike. That is, the authors succeed in constructing a continuum of meaning that ranges from the end points best of all to its bi-polar extreme despise (p.33), and further provide future researchers with both the scale value and standard deviation of each of the tested words and phrases. Similarly, Myers and Warner (1968) argue that the construction of accurate and meaningful scales requires that researchers comprehend the psychological meaning, to the respondent, of scale point descriptors. These authors further assert that quantitative measurement 3

4 of the meanings of commonly used scale point descriptors would allow researchers to develop equal interval scales that are desirable for subsequent statistical analyses of data sets. Accordingly, Myers and Warner (1968) modify the technique introduced by Jones and Thurstone (1955), investigate the psychological meaning of 50 commonly used scale point descriptors to four different groups of respondents, and present the respective mean scale values and standard deviations for all four groups of respondents. Even though the four subject groups are very different from each other (i.e., housewives, business executives, undergraduate and graduate business students), their mean scale values and standard deviations are very similar. Similar studies have been conducted by Bartram and Yelding (1973), Vidali (1975), Wildt and Mazis (1978), and the findings indicate that inventory scale values such as provided by Jones and Thurstone (1955) and Myers and Warner (1968) are surprisingly consistent among very diverse groups of people, can be used with a high degree of confidence, and are likely to provide psychological scales that are virtually equi-distant (Vidali, 1975, p.25). Considering, however, that languages change over time (Graddol, 2004; Yang, 2000), and no recent inventories are available, the purpose of the present study is to produce a current inventory containing commonly used scale point descriptors and their respective mean scale values and standard deviations. Since the present study focuses on two types of scales that are frequently used in service environments, namely Likert and satisfaction scales, the major contribution of this study is to provide researchers with quantitative measurement of the meanings of commonly used scale point descriptors, which as pointed out by Myers and Warner (1968) will make possible the development of equal interval scales and thus aid statistical analyses of data sets. 2. Methods 4

5 The goal of the present research was to develop inventories for two types of frequently used response scales, namely satisfaction and Likert scales. A review of the literature focused on locating commonly used scale point descriptors for both types of scales (see Tables 1 and 2). Given that that there is considerable overlap of scale point descriptors, a final number of 39 satisfaction items and 19 agreement items was chosen and tested. [INSERT TABLES 1 AND 2 HERE] The data collection followed the method first outlined by Jones and Thurstone (1955). Accordingly, all satisfaction scale point descriptors were treated as items on nine-point scales (from -4 to +4). Each scale was anchored to the left by greatest dislike, its midpoint by neither like nor dislike, and to the right by greatest like (see Table 3 for the instructions given to respondents). The procedure for the Likert scale point descriptors was similar, except that the left-hand anchor read greatest disagreement, the scale midpoint neither agree nor disagree, and the right-hand anchor greatest agreement. For each of the scale point descriptors, respondents were asked to place a check mark in the space on the nine-point scale that best described the meaning of the respective scale point descriptor. [INSERT TABLE 3 HERE] All data were collected online, in the United States. For that purpose, the SSI Survey Spot Panel was used. The panel is national in scope and was screened to include individuals years of age. A random sample was drawn, and of those invited to participate by panel, 65% qualified to participate in the survey. That is, because the present study focuses on creating an inventory of satisfaction and agreement measures in the health insurance industry, we recruited only subjects who actually had experience with such insurance, i.e., had group health insurance through an employer [self or spouse]. Considering that 65% of the U.S. population has health 5

6 insurance, our samples are therefore representative of the population of interest. Further, only the household decision-maker or co-decision maker was qualified to participate. The response rate of those who qualified was 62%. All subjects were asked to rate each of the 39 satisfaction and 19 agreement items. The satisfaction scale point descriptors were rated first, followed by the agreement scale point descriptors. The order of the items within each of the categories (i.e., satisfaction and agreement descriptors) was random. Following the procedure outlined by Jones and Thurstone (1955) and defended by Myers and Warner (1968), all subjects (N = 272) were shown all scale-point descriptors within each category at once. 3. Data analysis and results The mean value and the standard deviation were calculated for each of the scale point descriptors (Tables 4 and 5). Interestingly, even though only six of the items that had been tested by Jones and Thurstone (1955) were included in the list of satisfaction scale descriptors, the semantic meanings of those six have changed very little over the years (see Table 4). [INSERT TABLES 4 AND 5 HERE] 4. Discussion and conclusion The current study examines the semantic properties of commonly used scale point descriptors for both satisfaction and agreement scales, and subsequently provides inventories of mean values and standard deviations for these scale point descriptors to be used by researchers. Knowing a scale point descriptor s mean value makes it possible to construct successive interval and/or equal interval scales that support meaningful statistical analyses and interpretation. Although the current study manages to overcome some of the limitations pointed out by Myers and Warner (1968) namely the use of relatively small samples that are not national in scope and are not random in kind one limitation of the current study that future research should 6

7 investigate is the limitation that might arise due to the chosen product context. It is conceivable that scale point descriptor inventories developed within the context of health insurance might not be valid in other product contexts. However, even as we point to this limitation, Mittelstaedt (1971, p. 236), who compares three different studies that focused on building scale point descriptor inventories, helps us argue that the product context used to develop an inventory is not very likely to impact the usefulness of that inventory in other product contexts: In spite of differences in time, place, subjects, instruments, instructions, referents and the contextual differences which may arise from using widely different arrays of stimuli, the correspondence among the scale values of the three studies seems remarkable. 7

8 Crosby and Stephens, 1987, Journal of Marketing Research (cited by Wirtz and Lee, 2003, Journal of Service Research) Kolodinsky, 1999, Journal of Consumer Affairs Peterson and Wilson, 1992, Journal of the Academy of Marketing Science Peterson and Wilson, 1992, Journal of Marketing Research Peterson and Wilson, 1992, Journal of the Academy of Marketing Science Preisser, 2002, Health Services and Outcomes Research Methodology SIP Servizio Opinioni, 1989, as cited in Peterson and Wilson, 1992, Journal of the Academy of Marketing Science Weinstein, 1989, American Banker Consumer Survey Westbrook, 1980, Journal of Marketing (T- D Scale) For reasons of completion and exploratory purposes, the following scale point descriptors were added TABLE 1 Satisfaction Scales Displeased Pleased Very dissatisfied Dissatisfied Neutral Satisfied Very satisfied Very satisfied Somewhat satisfied Somewhat dissatisfied Very dissatisfied Uncertain Very satisfied Somewhat satisfied Unsatisfied Very unsatisfied Completely satisfied Very satisfied Fairly satisfied Somewhat dissatisfied Very dissatisfied Excellent Very good Good Fair Poor Very satisfied Quite satisfied Not very satisfied Not at all satisfied Very satisfied Somewhat satisfied Completely unsatisfied Delighted Pleased Mostly satisfied Mixed (about equally satisfied and dissatisfied) Mostly dissatisfied Unhappy Terrible Extremely satisfied, acceptable, slightly satisfied, OK, neither satisfied nor dissatisfied, slightly dissatisfied, fairly dissatisfied, completely dissatisfied, extremely dissatisfied 8

9 TABLE 2 Likert Scales Albaum, 1997, Market Research Society Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree Hair, Bush and Ortinau, 2003, Marketing Definitely agree Research Generally agree Slightly agree Slightly disagree Generally disagree Jacoby and Matell, 1971, Journal of Marketing Research McDaniel and Gates, 2002, Marketing Research Menezes and Elbert, 1979, Journal of Marketing Research For reasons of completion and exploratory purposes, the following two scale point descriptors were added Definitely disagree Agree Uncertain Disagree Strongly agree Somewhat agree Neutral Somewhat disagree Strongly disagree Strongly agree Generally agree Moderately agree Moderately disagree Generally disagree Strongly disagree Completely agree Completely disagree 9

10 TABLE 3 Instructions to Respondents WORD MEANING TEST In this test are words and phrases that people might use to show like or dislike for health insurance plans. For each word or phrase make a check mark to show what the word or phrase means to you. Look at the examples. Example I Suppose you heard a person say that he/she barely liked his/her health insurance plan. You would probably decide that he/she likes it only a little. To show the meaning of the phrase barely like, you would probably check under +1 on the scale below. Greatest Dislike Neither Like Nor Dislike Greatest Like Barely like Example II If you heard someone say he had the greatest possible dislike for a certain health insurance plan, you would probably check under -4, as shown on the scale below. Greatest Dislike Neither Like Nor Dislike Greatest Like Greatest possible dislike For each phrase on the following pages, check along the scale to show how much like or dislike the phrase means. 10

11 TABLE 4 Satisfaction Items Item Valid N Means Std Dev Excellent Completely satisfied Extremely satisfied Very satisfied Delighted Very good Quite satisfied Mostly satisfied Pleased Satisfied Good Fairly satisfied Somewhat satisfied Acceptable Slightly satisfied OK Fair Neutral Mixed (about equally satisfied and dissatisfied) Neither satisfied nor dissatisfied Uncertain Slightly dissatisfied Somewhat dissatisfied Fairly dissatisfied Not very satisfied Displeased Unhappy Dissatisfied Poor Unsatisfied Mostly dissatisfied Quite dissatisfied Very unsatisfied Not at all satisfied Very dissatisfied Terrible Completely dissatisfied Completely unsatisfied Extremely dissatisfied * Statistically significant at the.05 level Jones and Thurstone (1955) inventoried the scale point descriptors excellent (mean = 3.71, std dev = 1.01); very good (mean = 2.56, std dev =.87); good (mean = 1.91, std dev =.76); fair (mean =.78, std dev =.47); neutral (mean =.02, std dev =.18); poor (mean = -1.55, std dev =.87) 11

12 TABLE 5 Likert Items Item Valid N Means Std Dev Completely agree Definitely agree Strongly agree Agree Generally agree Moderately agree Somewhat agree Slightly agree Neutral Neither agree nor disagree Uncertain Slightly disagree Somewhat disagree Moderately disagree Disagree Generally disagree Strongly disagree Definitely disagree Completely disagree * Statistically significant at the.05 level 12

13 REFERENCES Albaum, G. (1997) The Likert scale revisited: an alternate version, Market Research Society, Vol 39 No 2, pp Bearden, W. O., Malhotra, M. K., Uscátequi, K. H. (1998) Customer contact and the evaluation of service experiences: propositions and implications for the design of services, Psychology and Marketing, Vol 15 No 8, pp Bertram, P., Yelding, D. (1973) The development of an empirical method of selecting phrases used in verbal rating scales, Journal of the Market Research Society, Vol 15, pp Crosby, L. A., Stephens, N. (1987) Effects of relationship marketing on satisfaction, retention, and prices in the life insurance industry, Journal of Marketing Research, Vol 24 (November), pp Graddol, D. (2004) The future of language, Science, Vol. 303 (February 27), pp Hair, J. F., Bush, R. P., Ortinau, D. J. (2003) Marketing Research: Within a Changing Information Environment, 2 nd edition. McGraw-Hill, New York. Jacoby, J., Matell, M. S. (1971) Three-point Likert scales are good enough, Journal of Marketing Research, Vol 8 (November), pp Jones, L. V., Thurstone, L. L. (1955) The psychophysics of semantics: an experimental investigation, The Journal of Applied Psychology, Vol 39 No 1, pp Kolodinsky, J. (1999), Consumer satisfaction with a managed health care plan, The Journal of Consumer Affairs, Vol 33 No 2, pp McDaniel, C., Gates, R. (2005) Marketing Research, 6 th edition. Wiley, Hoboken, NJ. Menezes, D., Elbert, N. F. (1979) Alternative semantic scaling formats for measuring store image: an evaluation, Journal of Marketing Research, Vol 16 (February), pp Mittelstaedt, R. A. (1971) Semantic properties of selected evaluative adjectives: other evidence, Journal of Marketing Research, Vol 8 (May), pp Myers, J. H., Gregory Warner, W.G. (1968) Semantic properties of selected evaluation adjectives, Journal of Marketing Research, Vol 5 (November), pp Peterson, R. A., Wilson, W. R. (1992) Measuring customer satisfaction: fact and artifact, Journal of the Academy of Marketing Science, Vol 20 No 1, pp Preisser, J. S. (2002) Quasi-likelihood analysis of patient satisfaction with medical care, Health Services & Outcomes Research Methodology, Vol 3 No 4, pp

14 Vidali, J. J.(1975) Context effects on scaled evaluatory adjective meaning, Journal of the Market Research Society, Vol 17 No 1, pp Weinstein, M. (1989) Consumers still like service, but their enthusiasm erodes, American Banker Consumer Survey, Vol 6, pp. 18, 20. Westbrook, R. A. (1980) A rating scale for measuring product/service satisfaction, Journal of Marketing, Vol 44 (Fall), pp Wildt, A. R., Mazis, M. B. (1978) Determinants of scale response: label versus position, Journal of Marketing Research, Vol 15 (May), pp Wirtz, J., Lee, M. C. (2003) An examination of the quality and context-specific applicability of commonly used customer satisfaction measures, Journal of Service Research, Vol 5 No 4, pp Yang, C. D. (2000) Internal and external forces in language change, Language Variation and Change, Vol 12 No 3, pp