Louis I. Nzegwu. Department of Business and Accounting. Submitted to: Kathryn Lomax Office of Sponsored Programs University of Wisconsin- Platteville

Transcription

1 Study to Examine Nonresponse Assessment in Marketing Research: Current Practices and Suggestions for Improvement Louis I. Nzegwu Department of Business and Accounting Submitted to: Kathryn Lomax Office of Sponsored Programs University of Wisconsin- Platteville September 2005

2 Abstract Nonresponse bias and means of improving its assessment continue to remain salient to marketing researchers and survey methodologists. Nonresponse is often considered the biggest threat to valid research. Decision-makers from Madison Avenue to Washington often rely on data from survey research. The ability to extrapolate results from a sample depends on how accurately responses from the sample s participants reflect those of the parent population. One dimension that is critical to this extrapolation is the extent of nonresponse. Given that almost no surveys have one hundred percent cooperation and that participation rates continue to decline, an ancillary question is: to what extent do the nonrespondents differ form the respondents? Nonresponse error can impose systematic errors on survey findings if the subjects responding to the survey differ from nonrespondents in survey variables. Purpose The purpose of this study is to investigate the extent of nonresponse rate and how response is addressed in survey research articles from three top marketing journals: The Journal of Marketing Research (JMR), Journal of Marketing (JM), and Journal of Consumer Research (JCR). Nonresponse is often considered the biggest threat to valid survey research and a survey s usefulness as it hinders the ability to generalize one s sample results to the population (Curtin, Presser, and Singer 2000).

3 For example, examination of response rate variation from the 1990 census revealed that nonrespondents to the mail questionnaire differed from respondents in terms of race-ethnicity, home ownership, family composition, and house structure. Word (1997) found that nonresponse rates were higher for: 1) minorities compared to White non- Hispanics; 2) individuals who rent rather than own; 3) nonspousal households; and 4) individuals who lived in a mobile home rather than a building. Looking at the four demographics cumulatively, White, non-hispanic homeowners who live with a spouse in a building had the lowest nonresponse rate with 13.2%. In contrast, Hispanics who rented a mobile home and lived in a non-spousal household had the highest nonresponse rate of 64.3%. Many researchers address the nonresponse issue by attempting to reduce its likelihood of occurrence. Indeed, much research has been devoted to understanding what techniques are most likely to enhance response rate (for example, see Kanuk and Berenson 1975; Yu and Cooper 1983; Fox, Crask and Kim 1989; Church 1993; Roth and BeVier 1998.) However, this literature omits the other component of nonresponse bias: examination of how respondents and nonrespondents differ for the survey variables (Groves and Couper 1998). Researchers may deal with survey nonresponse bias in a variety of ways including substitution; postsurvey assessment of nonresponse; postsurvey adjustment of results to correct for nonresponse; postsurvey assumption that nonresponse bias is not a concern; or survey a sample of nonrespondents.

4 Research Questions This study examined nonresponse assessment in three top marketing journals. Specifically, this study addressed the following research questions: 1. How extensive is nonresponse in the survey research reported in these journals? 2. What percentage of these articles address potential nonresponse bias? 3. For those articles that do address nonresponse, how is it assessed? 4. How does overall sampling method, probability versus non-probability, relate to nonresponse assessment? 5. For which variables do authors examine differences between respondents and nonrespondents? 6. Are there significant differences between respondents and nonrespondents? Methodology Examination of how several marketing research textbooks (e.g. Churchhill 1999; Dillon, Madden and Firtle 1993; Malhotra 1999; McDaniel and Gates 1998; Sudman and Blair 1998; Hair, Bush and Ortinau 2000; Zikmund 2000) addressed nonresponse issues, outcome of this examination formed the foundation for developing the coding instrument used for this study. After several iterations of the coding instrument, I attempted to conduct three rounds of pretests to help separate coded selected survey-based on article from 1993 issues of three marketing journals.

5 Sample Selection This study originally attempted to review 98 articles that reported survey research results in the 1994 to 1999 issues of JMR, JM, and JCR. (See Table I) However due to limitation of incomplete articles from the journals only 95 were finally used as a research study instrument. Table I JMR JM JCR TOTAL Total Articles ( )* No. of Articles Reporting Number of Sampling Frames * Excluding Book Reviews and Marketing Literature Reviews (JM) From a preliminary review of the articles included in this study, nearly half of the articles already reviewed did not report any nonresponse assessment. Among those that did assess nonresponse, comparison of early versus late respondents was the most common, but how early versus late was operationalized was rarely reported. Examples of coding multiple frames for a single article in previous related study included: a survey administered to respondents in multiple countries (e.g. Hofstede, Stennkamp, and Wedel 1999); respondents diverse business functions (e.g. Moorman and Rust 1999); the analysis involved matched dyads such as suppliers and distributors (e.g. Fein and Anderson 1997).

6 Results This study provided sufficient contribution to the study of nonresponse rate in Marketing Research. Outcome of the study reveals that: The final coding instrument assessed a variety of measures including, for the purposes of this study: the survey method; sample type; sampling frame; overall sampling method; the number of individuals contacted; the final analysis sample size and non-response assessment efforts. Nonresponse bias and means of improving its assessment continue to remain salient to marketing researchers and survey methodologists. Continued reliance upon comparing responders to the initial request for cooperation to those who respond to a follow-up as a surrogate estimator for nonresponse bias has numerous limitations. Benefits of Further Study This study provided significant benefits. In order to conduct a valid research. Nonresponse assessment plays a critical role in survey research as the understanding of if and how nonspondents differ from respondents permeates all analyses and inferences regarding the population of subsequent marketing decisions.

7 The validity of the survey research methods is critical to advancing marketing knowledge. In addition, assessments of methods of addressing potential nonresponse error are essential to marketing research.

8 References American Marketing Association. Journal of Marketing Research and Journal of Marketing Manuscript Guidelines. Available: and Armstrong, J. Scott and Terry S. Overton (1977), Estimating Nonresponse Bias in Mail Surveys, Journal of Marketing Research, 14 (August), Bickart, Barbara and David Schmittlein (1999), The Distribution of Survey Contact and Participation in the United States: Constructing a Survey-Based Estimate, Journal of Marketing Research, 36 (May), Chen, Henry (1996), Direction, magnitude, and implications of non-response bias in mail surveys, Journal of the Market Research Society, 38 (July), Churchill, Gilbert A. (1999), Marketing Research: Methodological Foundations. The Dryden Press, Forth Worth, TX. Cook, Barry (1996), Panel Cooperation: Strategies for Success, Journal of Advertising Research, 36 (1) RC2-RC6. Dillon, William R., Thomas J. Madden and Neil H. Firtle (1993), Essentials of Marketing Research, Irwin: Homewood, IL. Groves, Robert M., Robert B. Cialdini and Mick P. Couper (1992), Understanding the Decision to Participate in a Survey, Public Opinion Quarterly, 56, Hair, Joseph F. Jr., Robert P. Bush and David J. Ortinau (2000), Marketing Research: A Practical Approach for the New Millennium, Irwin McGraw-Hill: Boston, MA. Keeter, Scott, Carolyn Miller, Andrew Kohut, Robert M. Groves and Stanley Presser (2000), Consequences of Reducing Non-response in a National Telephone Survey, Public Opinion Quarterly, 64, McDaniel, Carl and Roger Gates (1999), Contemporary Marketing Research, South- Western College Publishing: Cincinnati, OH. Nielsen Media Research (2001), Who We Are and What We Do, Available: Roth Philip L. and Craig A. BeVier (1998), Response Rates in HRM/OB Survey Research: Norms and Correlates, , Journal of Management, 24 (1):

9 Tomaskoviv-Devey, Donald, Jeffrey Leiter and Shealy Thompson (1994), Organizational Survey Non-response, Administrative Science Quarterly, 39 (3) Yu, Julie and Harris Cooper (1983), A Quantitative Review of Research Design Effects on Response Rates to Questionnaires, Journal of Marketing Research, 20 (February): Zikmund, William G. (2000), Exploring Marketing Research, 7 th Edition, Dryden Press: Fort Worth, TX.