Brian F. Blake Cleveland State University. Ryan Murcko Cleveland State University. Michael Allen University of Sydney

Transcription

1 Adjustment for Demographic Effects in Cross-Cultural Comparisons: Conceptual Considerations Brian F. Blake Cleveland State University Ryan Murcko Cleveland State University Michael Allen University of Sydney Kimberly Neuendorf Cleveland State University Aron Wilson Cleveland State University

2 Background Possibly reflecting the emerging attention to the global economy, the creation and diffusion of global consumer culture, and the ready availability of more efficient vehicles for long distance interaction and communication, among other factors, empirical analyses of crosscultural differences are increasingly prominent in the behavioral and social sciences. Studies of cross-cultural differences in values and norms (e.g., Hofstede, 2001), consumption of consumer goods (e.g., Solomon, 2004), lifestyles (e.g., Weiss, 1999), for example, are attracting the attention of theoreticians and practitioners. In these studies of cross-cultural differences, variables measured in a sample of persons drawn from culture A are compared (etically) with measures in a sample drawn from culture B to find how the two cultures differ on the dependent variables of interest. When doing so, investigators face the question: How should one handle differences in the demographic profiles of the two samples? Several procedures are regularly used to address such demographic differences: a) use of samples matched on those demographics anticipated to impact the dependent variables ( matching ), b) statistically removing the additive effects of individuals demographic characteristics from measures of dependent variables ( additive correction ), c) estimating the culture free impact of demographic characteristics on the dependent variables and then removing the culture free but not the culture specific impacts of the demographics from the dependent measures ( interactive correction ), d) weighting persons within each sample so as to represent the demographic profile of a culture s population ( representative weighting ), and e) assuming, at least implicitly, that sample differences in demographics are inconsequential ( non-adjustment ). A researcher s failure to use a procedure consistent with the study s theoretical orientation may well lead to the study s drawing questionable conclusions. The matching and additive correction effects have been popular for quite some time (e.g., Berry et al., 2002; Brislin, Lonner, & Thorndike, 1973; Karahanna, Evaristo, & Srite, 2002; Triandis & Brislin, 1980; van de Vijver & Leung, 1997). The interactive correction is new to the cross-cultural research domain and so is, as yet, relatively infrequently used. Representative weighting is often used in public opinion polling and marketing research (Dorofeev & Grant, 2006), but is less often used in cross-cultural analyses. Interestingly, many cross-cultural studies have tacitly used the non-adjustment approach by not attending to demographic differences among culture samples. Illustratively, Schaffer and Riordan (2003) found that fully 15% of the cross-cultural organizational studies simply ignored demographic differences. Research Objectives To date, treatments of these procedures have seen the investigator s selection of a procedure as a methodological issue, as a decision based on cost effectiveness and on the proportion of demographic related variance extracted. The first objective of this report is to argue that procedure selection is not simply a methodological choice, but rather is more fundamentally a theoretical issue hinging upon the definition of culture employed. This report is the first one known to the authors to propose that the procedures differ with the definitional perspective on culture implicitly assumed by a procedure. A researcher s failure to use a procedure consistent with the study s theoretical orientation may well lead to the study s drawing questionable conclusions. Hence, researchers need initially to specify the definitional perspective on culture used in a study and then incorporate that definition of culture into the interpretation of the

3 empirical results. A second objective is to use survey data to demonstrate these points using the two procedures (interactive correction and representative weighting) relatively infrequently used in cross-cultural behavioral science studies. Matching Alternative Procedures Samples are obtained that are similar with regard to pertinent demographics, restricting the culture samples to persons within a delimited demographic grouping (van de Vijver & Leung, 1997; Berry, Poortinga, Segall, & Dasen, 2002; Karahanna, Evaristo, and Srite, 2002). Illustratively, Feldman, Rosenthal, Mont-Rynaud, Leung, & Lau (1991), in their study of cultural differences in adolescent misconduct, limited samples in each culture to persons in high school, aged 15 to 18, having parents with some formal education. Additive Correction This may be used alone or in combination with matching (Berry et al., 1997). First, subjects (respondents) are pooled into a single data set without attention to any size differences among the culture samples. In one approach, demographic variables are then entered as an initial block in a hierarchical regression (OLS or logistic), and the cultures entered as a later block predicting the dependent variable of interest (e.g., attitude, preferences). A variation on this approach is a hierarchical discriminant function or logistic regression in which culture samples are the groups to be differentiated (i.e., the dependent variable), demographic characteristics are the initial block(s) of predictors, and the variable measures to be compared among the cultures (e.g., attitudes, preferences) are entered in later block(s) of predictors. A more popular alternative procedure is ANCOVA or MANCOVA in which the demographic characteristics serve as covariates, cultures are levels of the independent variable, and the variable measures to be compared among the cultures are the dependent variable(s). As commonly practiced, these statistical analyses assume that the effect of culture is separate from, operates in addition to, and does not interact with the demographics. While interaction terms can potentially be used as predictors or as covariates, this is rarely done. Interactive Correction Proposed by Blake and Neuendorf (2004) and by Blake, Shamatta, Neuendorf, and Hamilton (2009, in press), this assumes that any demographic characteristic can have two separate impacts on a dependent variable (e.g., attitudes, values), a culture free impact that is common across-cultural settings (e.g., an effect of age that is constant from one cultural setting to another) and a culture specific impact (e.g., an impact of age unique to a culture) above and beyond the culture free impact. The procedure involves, first, forming a combined data set in which each culture sample is weighted to provide equal representation. Second, multiple regression (or equivalent) using the combined data set estimates the effect of the demographics (entered simultaneously) on a given dependent variable (e.g., attitudes, preferences). Suppose the multiple regression shows that age impacts a dependent variable with a Beta of -.52, this would be the culture free estimate of age. In an analysis of data on only one particular culture within

4 that combined data set (United States), the effect has a Beta of In an analysis of data on only one other particular culture within the combined data set (Turkey), the Beta is The difference between -.52 and -.30 spring from the culture effect specific of age to the US culture and the difference between the -.52 and -.89 flows from the culture effect specific to Turkey. Third, the regression equations are used to calculate residuals, so that for each dependent variable each person has a residual score. Fourth, the residual scores are then used in an ANOVA/MANOVA/discriminant function/logistic regression to differentiate the cultural samples. Note that the residual scores are uncontaminated by the culture free impact of demographics, while within each single culture sample the residual scores can correlate with sample demographics as a reflection of culture specific effects of those demographics. Representative Weighting The goal is to make each culture sample a demographic microcosm of that culture s population. Separately for each culture sample, the data points for a given demographic group in the sample are weighted so that the percentage of the people in that demographic group in the sample is the percentage that that group forms of the population. For example, suppose that the college educated males over 65 compose 10% of a culture sample, but Census data reveal that this group forms 20% of that culture s population. Each person in that demographic group in the sample will be weighted 2.0 in the analysis, while other demographic groups that are over represented in the sample are weighted down (below 1.0). The representative weighted samples of the various cultures are then compared via the usual statistical analysis (e.g., MANOVA, discriminant function). Non-adjustment While an investigator may report the demographic profile of the culture samples and may even qualitatively take sample differences into consideration when interpreting the qualitative differences between samples, the researcher does not adjust the statistical analyses to compensate for sample differences in demographic profile. Strengths and Limitations Considered simply as a method in its own right, each procedure has particular strengths. For example, matching has face validity at least for those differences between cultures that occur within the selected demographic segments. Additive correction employs widely used and understood statistical techniques (e.g., Cardinal & Aitken, 2006; Cohen et al., 2003) available in popular statistical packages like SPSS. Interactive correction is consistent, certainly more so than is the additive procedure, with the frequent observation (e.g., Blake, Valdiserri, Neuendorf, & Valdiserri, 2007) that the relationship between a given demographic and a dependent variable can often vary greatly from one culture to another. Notably, representative weighting is the only one of the procedures that permits the researcher to answer the question, How does the typical person in Culture A differ from the typical person in Culture B? on a given dependent variable. As noted in the classic treatise of Calder, Phillips, and Tybout (1981), an effects application study (in which a numerical score is generalized from a sample to a specific population, e.g., via a confidence interval) requires a sample that is a microcosm of the population of interest, a

5 representative cross section of the culture s population. Finally, the advantage of the nonadjustment procedure seems to reside entirely in its convenience for the researcher. Conversely, viewed simply as a procedure per se, each procedure has its limitations. For example, matching samples on one variable may inadvertently lead to mis-matching on other variables (Berry et al., 2002) or result in skewed data distributions (e.g., Sansone, Morf, & Panter, 2004). Further, by limiting the various culture samples to a specific demographic segment, the study is blind to those differences between cultures that occur only in other demographic segments. This is what Sapsford means when he notes that matched samples are not representative of their parent populations (2007, p.78). Additive correction is suspect when the slope of the demographic regression line varies considerably from culture to culture, indicating an interaction between culture and a given demographic on the dependent variable (e.g., Cardinal & Aitken, 2006; Cohen et al., 2003; van de Vijver & Leung, 1997). This situation is especially troublesome when the culture samples vary greatly in size. In the interactive correction the stability of the estimated effect of the culture free impact of a demographic factor is dependent upon the breadth of the cultures selected for the pooled sample. Illustratively, the effects of particular demographic factors estimated from an equal weighted pooled sample of the US, Canada, Austria, Iran, and Taiwan in Blake and Neuendorf (2004) were somewhat different from the effects of those same demographics when estimated from an equal weighted pooled sample of the US, People s Republic of China, and Poland in Blake et al. (2009, in press). Representative weighting may be ineffective when the sample is quite distinct from the population as a whole (Dorofeev & Grant, 2006), for example, as when entirely lacking particular segments. A potentially fatal flaw is that the relevant population parameters to be used in the weighting may be unknown or unavailable. In the non-adjustment approach any systematic effects of the samples demographic profiles render interpretation of empirical results impossible. The fundamental limitation of all the procedures, though, is that their suitability is limited to only certain definitional perspectives on culture. The problem is compounded by the relative dearth of attention to this limitation in treatises on social science methodologies. Definitional Perspectives The theoretical definition of a population s culture is admittedly a rather fuzzy construct (Triandis, Bontempo, Villareal, Asai, & Lucca, 1988, p. 323). While numerous definitions have been offered, a key issue is the degree to which the demographic profile of the population is an inherent and inseparable part of that culture. For purposes of assessing the suitability of the various procedures to address sample differences in demographics, let us distinguish among four Definitional Perspectives. These are alternative views on whether a population s demographic profile is an inherent and inseparable part of a population s culture. These Perspectives are implicitly and fundamentally built into the above adjustment procedures. Hence, they serve as assumptions rarely acknowledged or considered in interpreting results of a study using one of the procedures described. Integral Perspective This view treats culture as inextricably bound to the demographic profile of a population. By ignoring the demographic makeup of a culture s population, the researcher is examining only

6 a part of the culture. Change in culture goes hand in glove with change in population demographic structure. Consider shifts in age and gender distribution within a national culture. In the US the baby boom generation is involved in various culture shifts over the last 50 years (Dychtwald, 1999; Knickman, Snell, & Knickman, 2002; Owram, 1996; Robertson, 1989). When Boomers reached college age, there was a youth oriented shift in the US toward distrust and rebellion towards authority. Later when they entered the workforce, the US experienced high unemployment levels. As they have aged, size of their cohort has spurred increased costs of medical care and social services. Turning next to Iran, long known for dissuading the participation of women in the workforce or educational systems, women held a surprisingly substantial portion of the workforce by 1990 (Moghadam, 2003). One explanation for a high level of women in the work force was the dearth of young adult males, who were decimated by the 1980 to 1988 war with Iraq. Iranian women had to fill the many open employment positions left vacant during and after the war. Later, in 1989, university quotas limiting the number of women to be enrolled in many disciplines were relaxed. A short time after that, the new presidential administration in Iran urged women to strive for equal representation in Iranian universities (Moghadam, 2003). Approaching culture from the Integral perspective, we cannot define Iranian culture without reference to its gender distribution or American culture without regard to its age distribution. Discussion of the evolution of these two cultures must include shifts in gender and age respectively. Independence Perspective In this view, demographic profile of the population is distinct and separable from culture per se. A demographic assumedly has effects above and beyond culture, so that one can speak of an age effect without consideration of the cultural context in which the age effect occurs. When comparing culture samples, demographic characteristics must be considered, not because they are an inherent part of culture, but rather because they may interfere with discovering true cultural differences (van de Vijver & Leung, 1997). Schaffer and Riordan (2003), illustratively state that [demographic] differences are important when they exist alongside cultural differences, because they can affect a study s results (p. 181). Interactive Perspective A demographic characteristic can have one effect in culture X but a different effect in culture Y. It is also possible that the demographic characteristic can have comparable effects across cultures. In this perspective demographics are separable from culture per se, but their effects may or may not be contingent upon the cultural milieu involved. Irrelevant Here sample demographics are assumed to be separable from culture, and further, to have no meaningful impact upon the processes studied. Investigators implicitly adopt this perspective whenever they employ convenience samples in different cultures and do not adjust analyses for demographic differences in sample demographics.

7 Definitional Perspectives and Adjustment Procedures Each procedure implicitly assumes a particular Definitional Perspective: Matching assumes the Independence perspective whenever the investigator draws conclusions about differences between cultures without delimiting the conclusions to the demographic sectors studied. Additive Correction clearly embodies the Independence Perspective, for it treats the demographic impacts as relevant, separable from culture per se, and does not consider potential differences among culture in the impact of the demographics. Interactive Correction operationalizes the Interactive Perspective. Note, that, when the impact of the demographics are found to be similar across culture samples, the Interactive Correction is still effective. Whenever demographics and culture interact, however, the additive correction is not viable. Representative Weighting attempts to provide a data set that is a microcosm of the culture and, so when it is the only adjustment used, assumes the Integral Perspective. Since a procedure assumes a particular Perspective, researchers should explicitly consider whether the Perspective underlying the adjustment procedure to be used fits the cultural constructs being assessed. Perusal of the literature, unfortunately, indicates instances of lack of fit. For example, in some cases conclusions were drawn as if the study were designed for effects application, but demographics were treated by additive correction. In other cases matched but unrepresentative samples were used and conclusions were drawn from an Integral perspective. Demonstration Let us now demonstrate the two less frequently employed procedures, interactive correction and representative weighting, and compare them to the non-adjustment scores. Methodology Sample. A United States sample of 486 (318 females and 168 males) was geodemographically heterogeneous with a mean age of 33.6 years and a mean annual household income of $55,058 US. An Australian sample of 136 (83 females and 53 males) was composed of students at a large university; it had a mean age of 21.6 years and a mean annual household income of $61,291 AU. Data collection. For the United States sample, a snowball procedure recruited respondents, principally in the Midwest, via multiple outlets including business, church, civic groups, as well as acquaintances and relatives of the research team members. Participants were directed to the University s website where they completed an online questionnaire. The participants from the Australian sample completed the questionnaire as a paper booklet distributed to students in classroom groups.

8 Measures. The participants rated how strongly a particular website attribute (see Table 1, col. 1) encouraged them to shop at a given website for each of the 55 attributes comprising VISA, the Variegated Inventory of Site Attributes (Blake et al., 2009). The items were rated on a 7 point scale anchored at (1) Does Not Encourage Me At All to (7) Strongly Encourages Me to shop for goods or services at a particular site. Eight items were selected for the present study based on their theoretical interpretability (Bart, Shankar, Sultan, & Urban, 2005; Rogers, 1995) and their having a statistically significant relationship with the demographic characteristics used in the study. Among other demographics questions were one asking for gender and one for annual household income. The latter question in the US provided eight response categories: $10,000 or less; $10,001 - $20,000; $20,001 - $30,000; $30,001 - $40,000; $40,001 - $50,000; $50,001 - $75,000; $75,001 - $100,000; and more than $100,000. The Australian question used five categories: under $25,000; $25,000 - $ 49,999; $50,000 - $74,999; $75,000 - $99,999; and $100,000 or more. Income categories were rescaled for analysis to the midpoint of the category. The highest category in each country was set to $120,000, while the lowest category was assigned $5,000 in the US and $12,500 in Australia. Also, for the regression analyses in the interactive correction Australian dollars were multiplied by.6 to better equate to the US dollars in value. Adjustment Procedures Interactive Correction. A combined data set was composed by weighting each person in the US sample by.640 and in the Australian samples by The weighting resulted in two samples accounting for an equal number of cases (311 of the 622). Second, eight multiple regressions were performed, entering gender (dummied) and income (continuous) as a single predictor block and an attribute s rating in the 622 case sample as the dependent variable. The R 2 s were significant but low (see Table 1, col. 2), indicating weak relationships between gender/income and attribute preference. Third, for each attribute respondents residual scores were computed to gauge each person s preference once the culture free impact of gender and income was removed. The mean of these preference scores are in Table 1, col. 3 and 4. Fourth, standard analyses (MANOVA, discriminant function, etc.) are used with the residual scores to identify culture difference after the culture free component of the demographics has been removed. Representative Weighting. The first step was to identify the percentage of the (here, national) population in each demographic sector, e.g., US females with an annual household income of $20,000 to $50,000 (combining individuals in the $20,001-$30,000, $30,001-$40,000, and $40,001-$50,000 response categories). Since good estimates of the percentage of a nation s Internet shoppers in each demographic sector were not available, for this demonstration weighting was based on the percentage of persons in each sector in the adult population based on 2007 US (US Census Bureau, 2008) and 2006 Australian (Australian Bureau of Statistics, 2008) income figures. Next, the percentage of the sample in that demographic sector is computed. In our illustrative sector, the population percentage is 17.9 and the sample percentage Third, the weight needed to have a sample sector contribute to the analysis in proportion to its contribution in the population is calculated. Here, the weight for our illustrative sector is.766. Fourth, separately in each culture sample, each person s raw dependent variable scores are multiplied by that person s representativeness weight. A US female with an income of $20,000

9 to $50,000 with a raw preference rating of 5 for a given attribute would have a weighted score of 3.83 (i.e., 5 times the weight of.766). Table 1, cols. 5 and 6, show the means of the eight weighted scores in each sample. Fifth, again the standard analyses are used with the weighted scores to identify culture differences when demographic effects are present and each demographic subgroup is contributing its proportional share. Non-adjustment. For comparison, the raw, unadjusted sample means are in Table 1, cols. 7 and 8. Comparison of Procedures. As confirmed by the lack of statistically significant differences obtained in MANOVA and ANOVA tests, inspection of the figures in Table 1 indicates minimal differences in scores as calculated by the three procedures. The absence of strong differences among the procedures is logical, given the very modest impact of age and income on preference ratings as seen in the R 2 s in Table 1. Implications A strong linkage was proposed to exist between the procedure used to address sample differences in demographic profile and the theoretical assumption about the role of population demographic profile in a population s culture. The small differences observed among the procedures in adjusted scores in this procedural demonstration undoubtedly reflected the modest impact of the two demographics on the preference ratings in this data set. Whenever demographics are highly correlated with the dependent variables of interest, however, substantial differences should appear among five procedures. The strong conceptual linkage suggests that researchers engaged in etic cross-cultural research: a) clearly report their assumption(s) about the role of population demographics in the theoretical definition of culture that is guiding the research, b) select a procedure consistent with these assumptions, and c) consider these linkages when attempting to generalize study results.

10 References Australia. Australian Bureau of Statistics. (2008, September 5). Gross individual income by age by sex. Retrieved 20 January 2009 from the Australian Bureau of Statistics Web Site: Bart, Y., Shankar, V., Sultan, F., & Urban, G. (2005). Are the drivers and role of online trust the same for all web sites and consumers? A large-scale exploratory empirical study. Journal of Marketing, 69(4), Berry, J., Poortinga, Y., Segall, M., & Dasen, P. (2002). Cross-cultural psychology, research and applications (2 nd ed.). New York, NY: Cambridge University Press. Blake, B. F., Hamilton, R., Neuendorf, K. A., & Murcko, R. (2009, April). Individuals preference orientations toward facets of Internet shopping sites: A conceptual and measurement model. Paper presented at the National Technology and Social Science Conference, Las Vegas, NV. Blake, B. F., & Neuendorf, K. (2004). Cross-national differences in website appeal: A framework for assessment. Journal of Computer-Mediated Communication, 9(4), Blake, B. F., Shamatta, C., Neuendorf, K. A., & Hamilton, R. (2009, in press). The crossnational comparison of website feature preferences: A practical approach. International Journal of Internet Marketing and Advertising. Blake, B. F., Valdiserri, C. M., Neuendorf, K. A., & Valdiserri, J. N. (2007). The online shopping profile in the cross national context: The role of innovativeness and perceived innovation newness. International Journal of Consumer Marketing, 19(3), Brislin, R. W., Lonner, W. J., & Thorndike, E. M. (1973). Cross-cultural research methods. New York, NY: Wiley. Calder, B., Phillips, L., & Tybout, A. (1981, September). Designing research for application. Journal of Consumer Research, 8(2), Cardinal, R. N., & Aitken, M. R. F. (2006). ANOVA for the behavioural sciences researcher. Mahwah, NJ: Laurence Erlbaum Associates. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analyses for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Dorofeev, S., & Grant, P. (2006). Statistics for real-life sample surveys: Non-simple random samples and weighted data. Cambridge: Cambridge University Press. Dychtwald, K. (1999). Age power: How the 21st century will be ruled by the new old. New York, NY: Putnam. Feldman, S. S., Rosenthal, D. A., Mont-Reynaud, R., Leung, K., & Lau, S. (1991). Ain't misbehavin': Adolescent values and family environments as correlates of misconduct in Australia, Hong Kong, and the United States. Journal of Research on Adolescence, 1, Hofstede, G. (2001). Culture s consequences: Comparing values, behaviors, institutions, and organizations across nations (2 nd ed.). Thousand Oaks, CA: Sage. Karahanna, E., Evaristo, R., & Srite, M. (2002). Methodological issues in MIS cross-cultural research. Journal of Global Information Management, 10, Knickman, J. R., Snell, E. K., & Knickman, J. R. (2002). Public policy impact: The 2030 problem: Caring for aging baby boomers. Health Services Research, 37,

11 Moghadam, V. (2003) Modernizing women: Gender and social change in the Middle East (2 nd ed.). Boulder, CO: Lynne Rienner Publishers. Owram, D. (1996). Born at the right time: A history of the baby boom generation. Toronto: University of Toronto Press. Robertson, I. (1989). Society: A brief introduction. New York, NY: Worth. Rogers, E. (1995). Diffusion of innovations (4 th ed.). New York, NY: Free Press. Sansone, C., Morf, C. C., & Panter, A. T. (Eds.). (2004). The SAGE handbook of methods in social psychology. Thousand Oaks, CA: Sage. Sapsford, R. (2007). Survey research (2nd ed.). London: Sage. Schaffer, B. S., & Riordan, M. (2003). A review of cross-cultural methodologies for organizational research: A best-practices approach. Organizational Research Methods, 6, Soloman, M. R. (2004). Consumer behavior: Buying, having, and being. Upper Saddle River, NJ: Pearson Prentice Hall: Triandis, H. C., Bontempo, R., Villareal, M. J., Asai, M., & Lucca, N. (1988). Individualism and collectivism. Journal of Personality and Social Psychology, 54, Triandis, H. C., & Brislin, R. W. (Eds.). (1980). Handbook of cross-cultural social psychology (Vol. 5). Boston, MA: Allyn & Bacon. United States. US Census Bureau. (2008). Selected characteristics of people 15 years old and over by total money income in 2007, work experience in 2007, race, Hispanic origin, and sex. Retrieved 20 January 2009 from the US Census Bureau Web site: and van de Vijver, F. J. R., & Leung, K. (1997). Methods and data analysis for cross-cultural research. London: Sage. Watts, R. (1992). Elements of psychology of human diversity. Journal of Community Psychology, 20, Weiss, M. (1999, October). Parallel universe. American Demographics, pp

12 Table 1. Percent of Variance Explained by Gender and Income and Means of Adjusted Scores for Each Attribute, US and Australia Samples Attribute R 2 Correction Weighting Adjustment Interactive Representative Non- US AU US AU US AU The order process is easy to use..039* The products I am looking for are easy to find..042* A return policy that is easy to understand and use..046* Price incentives (coupons, future sale items, frequent shopper program, etc.)..047* It has guarantee from the vendor that my personal information will not be used to invade my privacy..048* The company offering the product/service guarantees that my personal purchase information will not be shared with other people or organizations..036* The products are guaranteed to be in stock..056* The company offering the product/service guarantees that my credit card information would not be abused..036* * p <.001