Mobile Phone Usability and Cultural Dimensions: China, Germany & USA

Transcription

1 Mobile Phone Usability and Cultural Dimensions: China, Germany & USA Tobias Komischke, April McGee, Ning Wang, Klaus Wissmann Siemens AG 1 Beijing, China; Munich, Germany; Princeton, NJ USA [email protected]; [email protected]; [email protected]; [email protected] Abstract It is well known that cultural differences influence the way in which people think and behave in many areas of work and home. This paper describes an exploratory study comparing mobile phone usability in China, Germany and the USA. The test design includes a newly developed questionnaire that, for the first time, links perceived product attributes to cultural dimensions. Conclusions focus not only on comparing ease-of-use and user satisfaction between countries, but also on explaining these issues against the background of cultural factors. Keywords: mobile phone usability, cultural dimensions, internationalization 1. Background Since the beginning of the 1990s, it has become essential for companies who are globally active and want to market their products internationally to address intercultural considerations in their product development processes. Empirical research into the comparability of different countries with reference to special cultural dimensions (e.g. into intercultural usability engineering) originated in the field of international business. In order to support foreign negotiations or the posting of staff to foreign countries, broad, quantitative research was conducted through questionnaires (Hofstede 1991; Trompenaars, 1993). Because intercultural dimensions were not explored primarily for the purpose of supporting product development, one major step was to establish the connection to graphical user interface design. This was especially addressed by Marcus (2001) who linked the Hofstedian cultural dimensions with characteristic factors in user interfaces (e.g. navigation, interaction, presentation). This link, originally conceived for web design, was operationalized more explicitly by Röse (2002) and defined as cultural user interface factors. Yet since there are still relatively few publications available that describe what issues are important when products are to be adapted to different markets, much insight can be gained by applied research. In this project we focused on comparing the use of a product in different countries and on mapping product characteristics such as usability and perceived impression to cultural dimensions. As a result, we hoped to reach an understanding regarding: what differences and what similarities of usage can be found in the addressed countries the nature of links/relationships between products and cultures what conclusions can be drawn for the localization of a product 1 We wish to thank Phil Arko, Nuray Aykin, Stefan Mark, Alex Sanchez and Kathleen Stahlberg for their assistance.

2 2. Method 2.1. Participants The participants in this study were 15 working professionals, from China, Germany and the USA, five from each country. Participants were matched as closely as possible to the target audience defined for the mobile phone: People that have a high interest in design issues 55% female, 45% male Between 28 and 40 years old Early career success Willing to show their success Each country s group consisted of three females and two males for a total of nine females and six males. Their ages ranged from 27 to 43 years. Participants were screened to qualify as early adopters open to new technologies. Each participant received a monetary incentive as a thank you token Measures Participants were asked to complete eight tasks using the mobile phone. The language of the prompts and text feedback on the phone matched the country where the test took place. Tasks were representative of typical mobile phone operations: 1. Briefly explore the phone and tell me your first impressions 2. Place a call to the following number: <phone number specified> 3. Call your friend <name specified >, whose number is in the address book 4. You have a new business contact and want to save his name and office phone number in the address book: <name + phone number specified> 5. Write your friend <name + phone number specified> a short message <message specified > 6. Set a new calendar appointment. 7. Disable the vibration mode for the phone s ringer. 8. Set the alarm for 8 p.m. today. With the exception of the first task, the task order was randomized for each participant. The number of errors and help requests was measured. Test sessions lasted between 45 and 120 minutes. All sessions were videotaped. The 5 Usability Dimensions Attitude Scale ( 5UD ) (ISO 9241, 1993) was administered after each task and at the end of the entire session to track and measure the attributes of efficiency, likeability, self-descriptiveness, controllability and learnability. Each was rated on a scale from 1 to 9, where 1 represents the least favorable and 9 is the most favorable. To achieve our goals we adopted the Hofstedian cultural factors, which can be described as separate instances in the following dimensions: Power Distance: The extent to which the less powerful members of institutes or organizations of a country expect and accept that power is distributed unequally.

3 Individualism vs. Collectivism: Individualism reflects cultures in which the ties between individuals are loose. Collectivism implies that people are integrated into strong cohesive in-groups from birth onwards. Masculinity vs. Femininity: Masculinity defines societies where stark differences are drawn between male and female roles in society. In feminine cultures, there is greater ambiguity in the male and female roles. Uncertainty Avoidance: The extent to which people use formalities, punctuality and legal, religious and social requirements to avoid uncertainty and ambiguity. Long-term vs. short-term orientation: Long-term orientation is found in cultures that orient themselves towards future reward. Perseverance and thrift are valued. Cultures with short-term orientation promote the virtues of the past and the present. Hofstede reported values which normally range from 0 to 100 for each dimension. A higher number correlates with a stronger characteristic of the dimension within the respective country. Since Hofstede has no data available for China for the dimensions of Power Distance, Individualism, Masculinity and Uncertainty Avoidance, these scores were estimated using the average scores of the Asian cultures of Japan, Taiwan, South Korea and Hong Kong. To be able to link users perceived impression of the phone to cultural factors, a semantic differential scale was developed. For this, we used findings from Hassenzahl (2002) on perceived hedonic quality of products as well as from Marcus work mentioned above. The scale features 15 bipolar word pairs, three at a time can be attributed to one of Hofstede s 5 cultural dimensions. The understandability of the word pairs in all three languages (US- English, German and Mandarin Chinese) were validated by pre-tests. Since this scale only marks a first step towards attributing product impression to cultural factors the scale was not statistically analyzed yet for its internal validity. The cultural dimensions data derived from the semantic differential (median) were normalized to fit Hofstede s scores. Then the difference between the Hofstede scores and the current scores was calculated for each country and dimension. For the relationship between the findings from this semantic differential scale and Hofstede s findings, we formulated the following two hypotheses: (1) For Germany, we expected to find approximately the same scores than Hofstede reported. (2) For China and the US, we expected that the scores from the semantic differential scale would deviate from the Hofstedian scores, leaning towards the scores which Hofstede reported for Germany. The reason for both hypotheses is that we assume a phone developed in Germany would reflect a German interaction philosophy Procedure The test design was planned and generated in cooperation between all three countries. Some local adaptation had to be done, e.g. with the definition of the screening criteria to select test persons. Due to the different time zones, this had to be performed asynchronously. The resulting test design then was carried out similarly across the countries.

4 Participants were given a brief introduction to the session, then completed a demographic questionnaire. The researcher then handed the subject the phone and an index card with the first task. Participants were encouraged to think and talk out loud. After each task, participants completed a 5UD questionnaire for that task. If the participant became stuck or frustrated on a task or asked for help, the researcher provided assistance. If the participant gave up completely, the researcher proceeded to the next task. The researcher took notes using a standard paper protocol and videotaped each session. After the last task the 5 UD questionnaire for an overall usability rating was administered and a debriefing session obtained general impressions as well as likes and dislikes. The data from the tests were analyzed and prepared first locally in the respective country, and then were later merged in the US to a central results document. Further communication between the three countries was necessary to clarify some issues and fine-tune the results documentation. 3. Results 3.1. Usability findings Variations in Perception of Usability Task 4 indicates the first instance of variation in perception of usability. This task, which asked participants to save an entry to the phonebook, showed that Chinese participants found the phone to be unsympathetic to user errors and non-self descriptive when relying on the user interface for help. Task 5, writing a short message, further emphasises this departure. In this case, US participants gave negative ratings to factors of controllability ( 3 ) and learnability ( 4 ); Chinese participants gave a negative rating to controllability ( 4 ); while German participants still continued to display a favorable regard for all 5 usability dimensions: efficiency ( 7 ), likeability ( 7 ), self-descriptiveness ( 8 ), controllability ( 7 ) and learnability ( 7, ). In task 6, setting a calendar appointment, we begin to see another similar accord in that all 3 countries had difficulties setting up an appointment. In fact, all 3 countries rated the phone neutral or worse with two exceptions the Germans gave a likeability rating of 6 and the Chinese gave a learnability rating of 6. Chinese participants rated the phone s level of selfdescriptiveness with a 2, which is unique as there were no other ratings lower than 4 by any country for any of the other attitude measures. Task 7, disabling the vibration mode, yielded consistent voting behavior per country. For example, Chinese participants voted between a 7 and 8 across all 5 UD s, while German ones voted between a 6 and 7, and US participants a solid 5 throughout, thereby exhibiting consistent voting behavior within each country across all 5 UD s. Task 8, setting the alarm, in general, yielded positive results (a 5 or better), except for the UD of controllability, where the Chinese gave it a Request for Assistance in Task Completion On more complex tasks users did request assistance. During task 4 only 1 US participant needed help, and only once; 3 Chinese participants needed assistance at least once, but no

5 more than three times, in order to complete this task; while the Germans had 3 participants that needed help, but with no more than one question each. Task 5 gave 2 Chinese participants some difficulty, where a minimum of two and a maximum of 3 questions were asked. Several German participants needed help only once; and 1 US participant needed help once. Task 6 was the task where German participants requested assistance the most (3 participants with at least one question, but no more than two), while Chinese participants had fewer questions, but more usability problems. This is not an unusual finding given that in Chinese culture people are more reluctant to acknowledge a lack of competency than in Western cultures Distribution of errors Across all applied test tasks, the German users made fewest errors while the Chinese users made the most errors. Because it was a phone developed in Germany, the conclusion from an intercultural standpoint could be that the phone s interaction philosophy complies with the expectation of the German test users. One of the reasons that Chinese users made more errors than the German and US test persons on all but one task could have been that the Mandarin Chinese translation of several terms was not optimal. Across all three countries, the fewest errors were made for the tasks of calling someone and changing settings (tasks 2,7,8). Tasks that involved typing in alphanumerical code produced increased error rates. One part of the problem was that most American and Chinese test users were unfamiliar with the T9 technology, which automatically completes a word and therefore may display other characters than the user actually selected. Because the phone s default setting had T9 enabled, the American and Chinese users produced more errors. Since they could not use the phone long enough to learn about the advantages of T9, they felt a lack of controllability and therefore disapproved of the technology Results on impression The comparison of the results from the three countries show that for most word pairs from the semantic differential, they followed the same answer tendencies. Major deviations were found for the following questions. US and German test users perceived the phone as being geared towards experts, whereas the Chinese thought of it as being geared towards novices. The same pattern was observed for the word pair ambiguous versus definite. One interpretation could be that because US and German participants perceived the phone as being ambiguous, they also rated it as being geared towards experts. The only word pair where all three countries clearly had different opinions was between (a) requires perseverance to reach results and (b) allows to achieve quick results. While the Chinese test users indicated a neutral position on this issue, the German persons were rather of the opinion that (a) was true. The US test participants on the other side tended towards (b). The following section further analyzes the results from the semantic differential on the background of intercultural issues.

6 3.2. Cultural Dimensions Table 1: Cultural Dimension Scores China Germany USA Hofstede 2 Study Findings Difference Hofstede Study Findings Difference Hofstede Study Findings Difference PD IDV MAS UAI LTO PD = Power Distance, IDV = Individualism, MAS = Masculinity, UAI = Uncertainty Avoidance, LTO = Longterm Orientation. A higher number (on a scale from 0 to 100) correlates with a stronger characteristic of the dimension within the respective country. 2 Estimated scores based upon the average of Hofstede s scores of Japan, Taiwan, South Korea and Hong Kong Difference China 0 Difference Germany -10 Difference USA PD IDV MAS UAI LTO Figure 1: Differences between Hofstede s scores and score found in this study. Figure 1 shows the differences between the Hofstede scores and the scores from this study. Positive scores mean that our scores were higher than Hofstede s whereas negative scores mean that our scores were lower than Hofstede s scores Differences Within Countries With the exception of Long-term Orientation, all cultural dimension scores for China had to be estimated. The found scores clearly deviate from Hofstede s score. For China, the two biggest difference values of all the three countries were found. The Individualism score is much higher than that found by Hofstede. This finding meets our second hypothesis that deviated scores lean towards the German score as stated by Hofstede. Since Hofstede states significantly higher Individualism scores for Germany, this result can be interpreted as reflecting a German cultural influence on the impression of the phone. Following the same line of argumentation, the same holds true for Long-term Orientation. The score that was found is much lower than Hofstede s score, thus indicating that the phone was perceived as being un-asian or geared more towards quick insight and immediate success. The Masculinity score was also much lower than Hofstede s score. Since Germany has a higher Hofstede score on this dimension than China, this result cannot be explained in the same way as for Individualism and Long-term Orientation and therefore does not support our hypothesis

7 number 2. However the phone was geared more towards women than men, which the masculinity score seems to reflect. So we assume that this end customer focus overlays the effects that we predicted according to our second hypothesis in terms of the cultural dimension of Masculinity. For Germany, we expected to find approximately the same scores as Hofstede (hypothesis number one) because it is a product that was developed in Germany and therefore should reflect German cultural influence. As the results show, this hypothesis cannot be confirmed. Only Individualism and Uncertainty Avoidance deviated less than 10 score points from Hofstede s findings. Power Distance was rated clearly higher, meaning that the German test users were of the opinion that the phone was acting more formal than they would have expected. Long-term Orientation also scored higher, which could lead to the conclusion that the phone was more than was expected to be perceived as being geared towards persons who would be interested in using and understanding it in detail without much time pressure. Similar to the findings for China, the Masculinity score was lower than expected. The same rationale can be used here: the phone was geared towards women. In sum the US scores deviated the least from Hofstede s scores. Very low differences were found for Individualism, Masculinity and Uncertainty Avoidance. Consequently the phone meets the US test persons expectations on these dimensions. The focus on women does not seem to have a significant effect. Power Distance is scored significantly higher than reported by Hofstede and because Hofstede states lower Power Distance scores for Germany - therefore our second hypothesis could not be confirmed. However the finding is in line with the scores given by the German test users. They also perceived the phone as being formal and hard to approach. Long-term Orientation is higher than the Hofstede scores. This follows our second hypothesis even though Germany s scores as being reported by Hofstede are only slightly higher than the US scores. This finding also matches with the German test user scores, leading to the same interpretation that the culture inherent in the phone is more than was expected to be perceived as being geared towards persons who would be interested in using and understanding it in detail without much time pressure Differences Between Countries It is interesting to note cultural similarities in the findings between the countries. As mentioned above, Germany and the US show similar scores for Long-term Orientation. That there is hardly any difference between the two countries corresponds with Hofstede s findings and reflects the fact that both countries are typical examples of Western culture. Germany and China show similar scores on the Uncertainty Dimension. Hofstede also reports almost similar values for these countries on this dimension - even though on a higher level. As opposed to the US culture, the findings confirm that people in Germany and China have a lower tolerance for unstructured situations and deviant ideas or behaviors. China and Germany show similar scores for Individualism, while the Hofstede scores are quite different (Germany being much higher). As mentioned above, the Chinese scores seem to have adapted to the German scores. China and the USA have similar scores on the culture dimension Power Distance, yet the Hofstede scores are different for these two countries. The test persons from both countries agree that the cultural influence shaped the phone towards being formal and hard to approach. This result differs from the American perspective more than from the Chinese.

8 4. Conclusion and outlook Due to the small number of participants the results should not be over-generalized; however, they do show similarities and differences between the countries. Especially interesting is that all modules that were used to assess user feedback on the product showed culture-related results. This will allow us to analyze cultural similarities and differences more in depth than could be achieved in this study. Finding differences in usage, impression and rating of a product confirms not only earlier research findings, but also the strategy of global companies to really draw the consequence to have local usability efforts. We also experienced the consequence that this decentralized work results in a higher communication and synchronization effort. However because this study was to a large extent detached from the development process these issues were not a significant hindrance. The newly developed semantic differential scale provided some interesting results. It proved to be of value to assess general impressions about the phone. Independent from that it could additionally be used as a tool that allows an intercultural interpretation of the responses. To advance from the exploratory nature of the scale towards a scientific tool, the internal validity needs to be further addressed in next research steps. Due to time constraints much of the study data could not be analyzed. This will be carried out together with applying the methodology and tools to other product domains. 5. References Hassenzahl, M. (2002). The effect of perceived hedonic quality on product appealingness. International Journal of Human-Computer Interaction, 13, pp Hofstede, G. (1981). Culture's consequences: International differences in work-related values. Newbury Park, CA: Sage. Hofstede, G. (1991). Cultures and Organizations: Software of the Mind: Intercultural Cooperation and its Importance for Survival. New York: McGraw-Hill. ISO (Issue 1993). Ergonomic requirements for office work with visual display terminals (VDTs) - Guidance on usability. Berlin: Beuth. Marcus, A. (2001). Cultural Dimensions and Global Web Design, AM+A, Röse, K. (2002). Models of culture and their applicability for designing user interfaces. In: Luczak, H.; Cakir, A.: WWDU 2002, Work With Display Units, World Wide Work. Proceedings of the 6th International Scientific Conference on Work with Display Units, Berchtesgaden, May 22-25, 2002, pp Trompenaars, F. (1993). Riding the waves of culture: understanding cultural diversity in business. London: The Economist Press.