The Influences of Media Choice on Help Desk Performance Perception Zoonky Lee Yongbeom Kim Sang-Gun Lee University of Nebraska-Lincoln Fairleigh Dickinson University University of Nebraska-Lincoln zlee@unlnotes.unl.edu kim@alpha.fdu.edu sglee@unlserve.unl.edu Abstract Along with the increasing complexity of IT, frequent changes and updates to new technologies have made developing an effective help desk challenging. The main purpose of this study is to investigate the effect of different media on the end-users' satisfaction of help desk functions. The major theme of this paper is to show that when electronic media are combined and properly used with conventional human-based media, higher end-user satisfaction can be achieved. Media richness theory is used to develop hypotheses about the relationship between media choice and end-user satisfaction for help desk functions. Survey results support that use of richer media is related to help desk satisfaction through assurance, and end-users who use various types of communication media have a higher satisfaction level when they match the media choice to different types of problems. The result suggests that automating help desks should be considered as a way to provide more various options to end-users, not a total solution for help desks, and automation should be more carefully designed depending on problem types. 1. Introduction In today s highly IT-mediated work environments, IT support for end-users becomes one of the critical concerns in the organization. Along with the increasing complexity of IT, frequent changes and updates to new technologies have made developing an effective help desk challenging [37]. According to a Gartner group survey, the annual total cost of PC ownership increased from $19,296 to $41,536, and end user operation costs have doubled from 1987 to 1992. With the growing demands and expectation of endusers, organizations are looking for ways to provide better help desk services. A number of commercial products using artificial intelligence techniques such as expert systems [17, 26] and case-based reasoning [1, 2] are available. Remote on-line troubleshooting and Internet-based products are examples of more advanced technologies to support help desks [24]. Outsourcing the help desk function also becomes a viable option to many organizations [5, 6]. While organizational efforts to automate systems and introduce knowledge-based tools are ways to control costs, maintaining the service quality has remained as an important issue [3]. The primary concern here is how to combine these new technology-enabled tools with conventional human-based support to provide an effective and efficient end-user support. The main purpose of this article is to investigate the effect of different media on the end-user satisfaction of help desk functions. While in-person and telephone are the conventional media to contact the help desk, the automation of help desk functions also allows users to use more electronic media. The major theme of this paper is to show that when these available electronic media are combined and properly used with conventional human-based media, higher end-user satisfaction can be achieved. The results of this study will enable organizations to better design their help desk functions. 2. Research Model and Hypotheses Development Media Choice Face-to-Face Telephone Fax E-mail Internet Rich Lean Figure 1. Research Model Problem Types Assurance Responsiveness End user Satisfaction To investigate the relationship between media choice and end-user satisfaction of help desk functions, a research model was developed as in Figure 1. In this model, we used two constructs from SERVQUAL [25] that are thought of as closely related to end-user satisfaction in help desk: assurance and responsiveness. 0-7695-0981-9/01 $10.00 (c) 2001 IEEE 1
Assurance is defined as the knowledge and courtesy of employees and their ability to convey trust and confidence. Responsiveness refers to the willingness to help customers and to provide prompt service [18]. Based on the research model, we developed hypotheses about the relationship between media choice and end-user satisfaction for help desk using media richness theory (MRT). MRT has two parts: the communication media differ in their relative abilities to reduce uncertainty and resolve equivocality, and the choice of right media depending on the task characteristics governs the organizational effectiveness [8, 9]. On the basis of the first part of MRT, we derive the relationship between media choice and different aspects of user satisfaction. Then, based on the second part, we develop hypotheses regarding the effect of matching problem types and media choice on user satisfaction. According to MRT, media that facilitate the immediate exchange of a wide range of communication cues are often regarded as rich media, while those that allow the exchange of a restricted range of cues over a longer period are considered lean media [32]. Equivocality, the state of ambiguity, the existence of multiple and conflicting interpretation about organizational situation [9, 10, 34, 36], usually requires rich media to resolve. Uncertainty, defined as the difference between the amount of information required to perform the task and the amount of information already possessed by the organization [15], can be reduced with lean media. A number of empirical studies have demonstrated that each communication media indeed has its richness trait [8, 9, 11, 13, 32,]. Faceto-face communication is considered to be the richest media as it provides more cues [32], interaction [7, 8, 38] and broader bandwidth [7] than any other media. Recently developed electronic media, such as e-mail and the Internet, are reported to be relatively lean media compared to conventional media such as telephone and fax [7, 20, 27]. Although more recent studies begin to argue that appropriateness of media depends on social processes [21, 22, 23, 31] and learning [17, 19], the relative media richness has been generally preserved [22]. It is expected that end-users using richer media, characterized by more interaction, more cues, and broader bandwidth, will experience more assurance on the service, increasing their satisfaction with the service quality of help desk [33]. H1: Use of richer communication media will lead to higher end-user satisfaction for help desk function through increased assurance. The second body of MRT predicts that matching media to the characteristics of a task will increase management effectiveness: using richer media for equivocal tasks and leaner media for non-equivocal tasks will increase performance. The empirical study of media choice to organizational effectiveness, however, has not been clearly demonstrated [14, 16, 19, 28, 29, 35]. Studies have found that the managers' choice of medium for certain tasks and environments deviates from the hypothesis predicted by MRT. In interpreting the results, others have argued that the proposition from MRT remained untested in those studies since most studies evaluated media choice rather than measuring media use and performance [12, 30]. If we apply MRT to our helpdesk context, it is predicted that end-users who match media choice on the basis of the complexity of questions will show a higher satisfaction. However, this proposition is difficult to test since we all have different perception on complexity of questions, and we might end up making the same mistake that Dennis and Kinney [12] pointed out: " virtually all have examined the perceptions of media fit by surveying the media choice of message senders, not by examining the actual performance effects of media use ". In this study, we propose that end-users should utilize different types of media for different types of questions when dealing with the help desk [4]. In other words, those end-users who stick to limited number of media choices will be less satisfied with the service of the help desk because they cannot match the characteristics of media for different types of questions. Therefore, we suggest that those end-users who can use various types of media will get a higher satisfaction level when they are involved in various types of questions. H2: End-users who use more various types of media will have a higher degree of satisfaction than end-users who have a limited choice of media when dealing with more different types of questions. H2a: End-users who use more various types of media will have a higher perception of assurance toward the help desk function than end-users who have a limited choice of media when dealing with more different types of questions. H2b: End-users who use more various types of media will have a higher perception of responsiveness toward the help desk function than end-users who have a limited choice of media when dealing with more different types of questions. 0-7695-0981-9/01 $10.00 (c) 2001 IEEE 2
3. Research Method For the purposes of this study, we used surveys of end-users to test hypotheses. The survey was distributed to 214 end-users at two organizations during the summer of 1999; one from the service and the other from the manufacturing industry with 1000 or more employees worldwide, and located mainly on the northeast coast of the United States. These organizations have in-house help desk functions. One hundred fifty-five respondents (72.4% return rate) agreed to participate. Thirty-six percent of the participants came from the service industry, and sixty-five percent from the manufacturing industry. In terms of overall computer proficiency of participants, most participants were proficient: power-user (9.7 percent), above-average (40 percent), and average (45.8 percent). Only 4.5 percent of participants were novice. First, we asked end-users their frequency of using different media in contacting help desks. The media in question are in-person, telephone, fax, e-mail and the Internet, and the frequency was measured using 5-point Likert-type scales (from never to always). Second, we took instruments from SERVQUAL to measure assurance and responsiveness. Although original SERVQUAL carries five different constructs, we used measures related to assurance and responsiveness. Table 1 shows the factor analysis results. Items Assurance Response HD staff was answering courteous 0.915-0.103 HD staff was professional and enthusiastic 0.881 0.130 HD staff was knowledgeable in the subject area 0.785 0.168 How long before your problem was resolved? 0.144 0.932 Problem was solved in a timely manner 0.003 0.769 Eigen Value 2.250 1.516 Percentage of Variance Explained 45.0% 30.3% Cronbach's alpha 0.847 0.661 Table 1. Factor Analysis of Satisfaction on Help Desk As shown in Table 1, the factor analysis shows a clear structure, suggesting clear discriminant validity for these constructs. These two factors account for more than 75% of variance. The loadings of each of five measures on its respective factor was well over 0.75, and all cross-loadings were smaller than 0.17. Predictive validity was also established by correlating these two measures with an overall satisfaction measure that asks the general satisfaction level of help desk. The correlation between assurance and responsiveness with an overall satisfaction measure was 0.435 and 0.616 respectively, and both are significant at the 99% level. Finally, we asked end-users for what kinds of problems they contact the help desk. Six items asked are: hardware problems (monitor, CPU, Printer, etc.), off-the shelf application software problems (word processor, e-mail, etc.), customized application software problem, operating system problems (windows, NT, Unix functions such as file management), log-on problems (user ID, password, etc.), and telecommunication problems (remote access, modem, etc.). Then, the number of items checked by each respondent was added to see how many different types of problems each end-user experienced. The number approximately follows the normal distribution, with a mean 2.40 and a standard deviation 1.31. 4. Data Analysis First, we asked the frequency of using five different communication media in contacting the help desk function. As shown in Table 2, most end-users use telephone, e-mail, and in-person (face-to-face). Respondents rarely use fax and the Internet, as their average frequency is less than 1.15. Constructs Mean Std. Deviation In person 1.5871 1.1610 Telephone 4.4129 1.2473 Fax 1.1355 0.6554 Email 1.7935 1.1936 Internet 1.1097 0.5168 Table 2. Mean of Using Rate of Communication Media When Contacting Help Desk To examine Hypothesis 1, simple correlation analysis was used. Each media use was correlated with both assurance and responsiveness (see Table 3). Constructs Assurance Responsiveness In person 0.16** -0.02 Telephone 0.10 0.11 Fax 0.03 0.07 Email 0.10-0.01 Internet 0.05-0.01 ** significant at α=0.05 level Table 3. Correlations between Media Use and Satisfaction As we hypothesized, use of rich media is related to the assurance of the help desk. The Pearson correlation between frequency of in-person in contacting the help desk and assurance is 0.16, which is significant at the 95% level. Relatively less rich media such as e-mail and the Internet were not correlated with customer satisfaction. 0-7695-0981-9/01 $10.00 (c) 2001 IEEE 3
.2 To examine Hypothesis 2, we first performed a cluster analysis. Since fax and the Internet are rarely used, we grouped based on use of in-person, telephone and e-mail only. Table 4 shows the result of the cluster analysis. We identified two groups of end-users: one group utilizing all of these three media, and the other using telephone only. We named the first group hybrid group (HG), and the second group telephone-centralized group (TG). The number of end-users in HG and TG group was 58 and 97, respectively. Satisfaction on Assurance.1 -.0 -.1 -.2 -.3 1 2 Hybrid Group Telephone Group Frequency Hybrid Group (HG) Telephone-centered Mean Group (TG) In-person 2.54 1.13 Telephone 3.30 4.94 Email 2.74 1.34 Total Number 58 97 Levels of Problem Complexity Figure 2. Assurance and Interaction.2 Table 4. Cluster Analysis of End-users depending on Media Use Before examining the hypothesis 2, we tested one-way ANOVA to detect differences between the two groups. The two groups have no difference in computer proficiency (F=1.99, p=0.16), frequency of contacting the help desk (F=0.535, p=0.466), number of different problems they ask the help desk (F=0.988, p=0.322) and their perception of responsiveness by the help desk (F=0.349, p=0.556). We found a marginal difference in the perception of assurance (F=3.139, p=0.78). A slope analysis was used to test the interaction effect between media choices and the number of different types of questions. Media choices were grouped into two (i.e. HG and TG) and the number of different types of questions was grouped into two (low number vs. high number). In general, the TG group was slightly higher in assurance satisfaction, which was only marginally significant. When we categorized end-users depending on how many different questions they ask for help, however, we began to see different results. The TG group significantly decreases its satisfaction level of assurance as they begin to ask more different types of questions, but the assurance level remained unchanged for the HG group (Figure 2). The HG group increases the satisfaction on responsiveness as they deal with more different types of questions, while the TG group decreases their satisfaction on responsiveness (Figure 3). Satisfaction on Responsiveness.1 0.0 -.1 -.2 1 Figure 3. Responsiveness and Interaction 5. Findings Levels of Problem Complexity The first hypothesis tests whether use of richer media is related to help desk satisfaction through assurance. The results show that use of the in-person medium is significantly correlated with assurance, and the other less rich media are not correlated with assurance. However, the use of telephone, one of the richest media, is not correlated with assurance. If we look at the items for measuring assurance, we can see these items are about the help desks staff being courteous, professional, enthusiastic, and knowledgeable (see Table 1). It seems that end-users feel more assured of help desk service when they use the face-to-face medium. On the other hand, none of the media choices were closely related to responsiveness, indicating that simply choosing any type of media does not lead to timely solutions. The classification of end-users depending on media choice results in two groups: the hybrid group (HG) who uses all available media, and the telephone group (TG) in which only telephones are used to contact the 2 Hybrid Group Telephone Group 0-7695-0981-9/01 $10.00 (c) 2001 IEEE 4
help desk. Since these two groups do not differ in computer proficiency and frequency of contacting the help desk, we assume that it is a personal disposition that makes two groups. The identification of two groups also suggests that all end-users need human interaction in contacting the help desk as we do not come up with a group with only the electronic media. Simply using telephone only or mixing different media does not affect help desk satisfaction much although using the in-person medium was marginally higher in assurance. The satisfaction, however, shows a clear difference when we investigated the satisfaction level considering the number of different types of problems end-users ask. For end-users who contact the help desk with the same type of questions (e.g. low in different types of questions), satisfaction was higher in terms of assurance and responsiveness when they use telephone as a primary medium to contact the help desk. Especially, those end-users using telephone only feel more strongly that the help desk provides timely solution than HG does. However the satisfaction level drops sharply when this group begins to ask more different questions. Satisfaction on assurance and responsiveness drops from 1.2 to -2.1, and from 0.8 to - 1.1, respectively. On the contrary, the satisfaction level of HG stays the same or increases as they begin to ask more different types of questions: assurance does not decrease, and responsiveness increases. The result suggests that mixing and utilizing different media for different types of questions contributes to the timely solution of problems. 6. Conclusion In this article, we investigated the relationship between media use to contact the help desk and enduser satisfaction. Based on MRT, we hypothesized that use of richer medium will increase end-user satisfaction on assurance. We also hypothesized that end-users who utilize various communication media selectively depending on the characteristics of problems would get a higher degree of satisfaction on help desk functions. The results of data analysis showed that use of inperson media is related to increase in end-users' perception on service assurance. The results also showed that those end-users who use various types of communication media have a higher satisfaction level in terms of help desk assurance and responsiveness only when they are involved with many different types of problem tasks. The result is interesting because endusers using only a limited choice of media (here telephone) show the satisfaction level decreases with the number of different problems increase, indicating that matching one media choice to all different types of problems only decreases the satisfaction level. Another interesting result from data analysis is the identification of two different patterns of media usage groups in contacting help desks: hybrid and telephoneonly group. We could not find any difference in the computer proficiency or the problem types they asked between these two groups. Furthermore, the two groups do not show any significant difference in the help desk satisfaction level. The results of this research have some significant theoretical implications. First, we applied the concept of media richness theory to the domain of help desk function. While there are a good number of different empirical studies that have applied the MRT, most of studies were based on scenarios and only a few studies have applied it to the real situations. We empirically tested the media/task matching frame to the media choice/help desk satisfaction, and confirmed the theory. Second, our research suggested an importance of relationship among different media choices. The results showed that there are patterns of media usage, and a high correlation exists between usages of one type of medium with another. For example, use of in-person as a medium to contact the help desk is highly correlated with use of e-mail, but not with use of telephone. It seems to suggest that we have to understand the relationships of media or patterns of using media rather than focusing on one medium when we study the media choice or the impact of media choice problems. Practically, this study provides some guidelines in designing more effective help desks. First, the results indicate that organizations should understand the impact of automation of help desk functions on end-user satisfaction. While organizations have invested a huge amount of money to automate help desk functions in an effort to save an ever-increasing end-user supporting cost, they do not seem to understand its impact on customer satisfaction. The cost saving and better satisfaction is not exactly a tradeoff as was shown in our result; providing only telephone services does not increase customer satisfaction although it might be a more costly option. Instead, the results suggested that providing various media choices increase the customer satisfaction only when end-users discernibly choose the media depending on the problem types. It suggests that organizations should provide automation of help desks for some type of questions that are best solved with automation, and guide users to use them. In summary, automating help desks should be considered as a way to provide more various options to end-users, not a total solution for help desks, and automation should be more carefully designed depending on problem types. Second, it is necessary to provide effective end-user training. End-users need to be trained to use different media depending on different types of problems. Our data indicated that the most dominant way of contacting 0-7695-0981-9/01 $10.00 (c) 2001 IEEE 5
the help desk is still by telephone. The results also showed that some end-users use various types of media ranging from face-to-face contact to e-mail. However, our results showed that satisfaction with the help desk is a function of both media choice and types of problems, not of media choice only. The result suggests that endusers be trained to match media choice to the types of problems. 7. References [1] Aamont, A. and Plaza, E., (1994). Case-based Reasoning: Foundational Issues, Methodological Variations, and Systems Approaches, AI Communications, Vol. 7, No. 1, pp.39-59. [2] Aha, D.W. and Wettschereck, D., (1997). Case-based Learning: Beyond Classification of Feature Vectors, Proceedings of 9 th European Conference on Machine Learning, Prague, Czech Republic, April, pp.329-336. [3] Boronico, J., Zirkler, A., and Siegel, P., (1996). Quantifying the Tradeoffs between Cost and Quality for System Service Support, Journal of Applied Business Research, Fall Vol. 12, No. 4, pp. 70-82. [4] Campbell, D. J., (1988). Task Complexity: A Review and Analysis, Academy of Management Review, Vol. 13, No. 1, pp.40-52. [5] Chalos, P. and Sung, J., (1998). Outsourcing Decisions and Managerial Incentives, Decision Sciences Vol. 29, No, 4, Fall pp.901-919. [6] Chapman, R. B. and Andrade, K., (1997). Insourcing after the Outsourcing, American management Association, New York. [7] Chidambaram, L. and Jones, B. (1993). "Impact of communication medium and computer support on group," MIS Quarterly, Vol. 17, No. 4, Dec., pp.465-492. [8] Daft, R.L. (1986). Organizational Theory and Design, Second Edition, West St. Paul, MN. [9] Daft, R.L. and Lengel, R.H. (1986). Organizational Information Requirement, Media Richness and Structural Design, Management Science, Vol. 32, No 5, pp. 554-571. [10] Daft, R.L. and Macintosh, N. B. (1981). A Tentative Exploration into the Amount and Equivocality of Information Processing in Organizational Work Units, Administrative Science quarterly, Vol. 26, No 2, June, pp. 191-233. [11] Daft, R.L., Lengel, R.H. and Trevino, L.K. (1987). Message Equivocality, Media selection, and Manager Performance: Implications for Information Systems, MIS Quarterly, Vol. 11, No 3, pp. 354-366. [12] Dennis, A. and Kinney, S., (1998). Testing Media Richness Theory in the New Media: The Effect of Cues, Feedback, and Task Equivocality, Information Systems Research, Vol. 9, No. 3, pp. 256-274. [13] El-Shinnawy, M. M. and Makus, M.L., (1992). Media Richness Theory and New Electronic Communication Media: A study of Voice Mail and Electronic Mail, Proceedings of the Thirteenth International Conference on Information Systems, Dallas, TX, pp, 91-105. [14] Fulk, J., Steinfield, C.W., Schmitz, J., and Power, J.G., (1987). A Social Information Processing Model of Media Use in Organizations Communication Research, Vol. 14, No. 5, pp. 529-552. [15] Galbraith, J. (1973). Strategies of Organizational Design, Addison-Wesley, Reading, Massachusetts. [16] Gefen D. and Straub D.W., (1987). Gender Differences in the Perception and Use of E-mail: An Extension to the Technology Acceptance Model, MIS Quarterly, December, Vol. 11, No. 4, pp. 389-400. [17] Goodman P.S., and Darr E.D., (1988). Computer-Aided Systems and Communities: Mechanism for Organizational Learning in Distributed Environment, MIS Quarterly, December, Vol. 12, No. 4, pp. 417-440. [18] Kettinger, William, J., and Lee, Choong C., (1994). "Perceived Service Quality and User Satisfaction with the Information Services Function", Decision Sciences, Vol. 25, No. 5/6, September-December, 737-766. [19] King, R. C. and Xia, Weidong, (1997). Media Appropriateness: Effects of Experience on Communication Media Choice, Decision Sciences, Vol. 28, No. 4, Fall, pp. 877-910. [20] Kydd, C. T. and Ferry, D. L., (1994). "Managerial use of video conferencing," Information & Management, Vol. 27, Iss. 6, pp.369-376. [21] Markus, L., (1988). Information Richness Theory, Managers, and Electronic Mail, presented at the Academy of Management National Meeting, Anaheim, CA. [22] Markus, L., (1994). Electronic Mail as the Medium of Managerial Choice, Organization Science, Vol. 5, No. 4, November pp. 502-527. [23] Ngwenyama O. K. and Lee, A., (1997). Communication Richness in Electronic Mail: Critical Social Theory and Contextuality of Meeting, MIS Quarterly, June, Vol. 21, No. 2, pp. 145-167. [24] Oxford, C., (1998). Strategies for Resolving the Help Desk Crisis, White paper, Help Desk 2000 TM. [25] Parasuraman, A., Zeithaml, V., and Berry, L. (1991). Refinement and Reassess of the SERVQUAL Scale, Journal of Retailing, Vol. 67, Iss. 4, pp. 420-450. 0-7695-0981-9/01 $10.00 (c) 2001 IEEE 6
[26] Puppe, F., (1993). Systematic Introduction to Expert Systems: Knowledge Representations and Problem-Solving Methods, Springer-Verlag, Berlin. [27] Rice, R.E. and Shook, D. E., (1990), "Relationships of Job Categories and Organizational Levels to Use of Communication Channels, Including Electronic Mail: A Meta- Analysis and Extension," The Journal of Management Studies, Vol. 27, Iss. 2, Mar., pp.195-230. [28] Rice, R.E., (1987) Computer-mediated Communication System and Organizational Innovation, Journal of Communication, Vol. 37, No. 4, pp. 65-94. [29] Rice, R.E., (1990) Computer-mediated Communication System Network Data: Technical Concerns and Empirical Examples, International Journal of Man-Machine Studies, Vol. 32 No. 6, pp. 627-647. [30] Rice, R.E., (1992) "Task Analyzability, Use of New Media, and Effectiveness: A Multi-Site Exploration of Media Richness," Organization Science, Vol. 3, pp. 475-500. [31] Rice, R.E., (1993). Media Appropriateness: Using Social Presence Theory to Compare Traditional and New Organization Media, Human Communication Research, Vol. 14 No. 1, pp.451-484. [32] Short, J., Williams, E., and Christie, B., (1976). The Social Psychology of Telecommunications, John Wiley, New York, NY. [33] Swink, M. and Robinson, P. (1997). Complexity Factors and Intuition-based Methods for Facility Network Design, Decision Science Vol.28, No.3, Summer 1997, pp.583-614. [34] Trevino, L.K., Lengel, R.H., and Daft, R.L., (1987). Media Symbolism, Media Richness, and Media Choice in Organizations, Communication Research, Vol. 14. No. 5., pp.553-574. [35] Walther, J.B., (1995). Relational Aspects of Computer- Mediated Communication: Experimental Observations over Time, Organizational Science, Vol.6, No. 2, pp. 168-185. [36] Weick. K. (1979). The Social Psychology of Organizing, Addison-Wesley, Reading, Massachusetts. [37] Whiting, R., (1997). Panic at the Help Desk, Software Magazine, Vol. 17, Iss. 11, Oct., pp. 92-95. [38] Zigurs L., Poole, M.S., and DeSanctis, G.L., (1988). A Study of Influence in Computer-Mediated Group Decision Making, MIS Quarterly, December, Vol. 12, No. 4, pp. 625-644. 0-7695-0981-9/01 $10.00 (c) 2001 IEEE 7