11 Success of the Help Desk: Assessing Outcomes

Transcription

1 11 Sccess of the Help Desk: Assessing Otcomes I dread sccess... I like a state of continal becoming, with a goal in front and not behind. George Bernard Shaw Key Findings Respondents help desks tend to meet most representative goals often or almost always. Respondents help desks have their greatest positive impact on central IT service reptation and the perceived vale of central IT. Instrctional and administrative activities and the workloads of central and nit-specific IT staff follow, with the lowest positive impact recorded being in the area of research activities. Positive help desk impact is greater where respondents report greater help desk matrity, more advanced strategic plan stats, stronger agreement that the help desk ses metrics effectively to improve ser service, and a brisker pace of adoption of new technologies by the instittion as a whole. More than half of respondents rate overall help desk service qality as either very good or excellent. Respondents say their clients wold rate help desk service qality lowest in the areas of research spport and spport for administrative and instrctional applications. Service qality is positively associated with adoption of or list of for basic ITSM practices and inclsion of the help desk in activites related to them; with the se of metrics to docment the help desk s performance; and with the existence of a strategic plan for the help desk. In general, the richer the help desk toolset, the better the help desk s overall service qality. Service qality varies dramatically with the help desk s matrity level. The more matre the help desk, the higher the service qality rating. In all eight of the areas we asked abot, the extent of the help desk s positive impact is greater where help desk service qality is higher. The sccess of the higher edcation IT help desk is, in large part, reflected in its ability to meet its goals. Or respondents told s in general and on average that they often or almost always do what they set ot to do in nine representative areas. However, they also told s that they wold rate the overall qality of the services the help desk provides somewhat lower 3.55 on a five-point scale, placing them sqarely between good and very good. These findings sggest that meeting goals isn t the only key to an organization s 2007 EDUCAUSE. Reprodction by permission only. EDUCAUSE Center for Applied Research 113

2 sccess. For example, sccess is not assred if the wrong goals are met or if low-priority goals are met at the expense of other matters whose importance may have been nderestimated and for which goals may never have been articlated. This chapter explores factors affecting help desk otcomes: the sccess or respondents have had at meeting the goals we discssed in Chapter 9, the areas in which they feel the help desk has a positive impact, and their estimations of their help desks overall service qality. As we shall see, each of these measres is associated with several help desk management practices we discssed in previos chapters. Sccess in Meeting Goals Having goals in place is a good thing, in general, and in Chapter 9 we saw that majorities of respondents had docmented goals in place for help desk service availability, ser satisfaction, and call/incident resoltion. Smaller percentages of respondents had set for themselves varios others of the nine goals we asked abot. Of corse, jst having a goal in place is no assrance that it will be met, with all the implications that failre to meet goals has for service qality and the effectiveness of the help desk. The freqency with which or respondents help desks met their goals was srprisingly niform, as Table 11-1 demonstrates, ranging from a low of 3.83 (between sometimes and often ) to a high of 4.67 (between often and almost always ). The goal most freqently met was spport service availability. As discssed in Chapter 9, meeting this goal might be as simple as opening the help desk office doors dring posted operation hors or assigning a staff member to the help desk telephone line and accont. This probably explains the difference of 0.45 points on or five-point scale between this goal s mean freqency and that of the next and considerably more complex one, incidents resolved per nit time. Other goals reported as being met with a mean freqency greater than 4.0 ( often ) inclde incidents resolved per nit time, incidents resolved per staff member per nit time, percentage of ser satisfaction, and calls resolved at first contact. Goals reported as being met with a mean freqency above 3.0 ( sometimes ) bt below 4.0 inclde telephone wait times, SLA reqirements flfilled, complex-incident resoltion time, and telephone call abandonment rate. Positive Impact of the Help Desk Looking at perceptions of the help desk s sccess in meeting its goals gives s focsed insight into particlar help desk practices bt doesn t help s gage the help desk s positive Table Freqency with Which Help Desk Goals Are Met Help Desk Goal N Mean* Std. Deviation Spport service availability Incidents resolved per nit time Incidents per staff member Percentage ser satisfaction Calls resolved at first contact Telephone wait times SLA reqirements flfilled Complex-incident resoltion time Call abandonment rate *Scale: 1 = almost never, 2 = seldom, 3 = sometimes, 4 = often, 5 = almost always 114

3 impact on its constitents. The ideal way to assess impact, of corse, is to srvey the client poplation directly something many of or respondents do reglarly bt which is beyond the scope of this srvey. So to help sharpen or sense of help desk benefits, we asked respondents to evalate the positive impact the central IT help desk had in the following eight areas: the instittion s instrctional activities, the instittion s administrative activities, the instittion s research activities, central IT services general reptation, camps perception of the vale of crrent central IT services, camps administration s willingness to fnd additional central IT initiatives, redcing central IT specialists' workload, and redcing nit-specific IT specialists' workload. Figre 11-1 portrays the responses, with impact areas arrayed from left to right in ascending order of high positive impact responses. 1 Respondents reported that the help desk had the least positive impact on research activities: Nearly two-thirds said the extent of the help desk s positive impact was not at all or a little. Slightly more than a qarter said impact was moderate, and fewer than 1 in 10 said it was high. (Note that only 87.4 percent of or srvey poplation responded to this qestion, reflecting the many instittions at which research is not an instittional priority.) Research was the only impact area for which the percentage of low responses exceeded both moderate and high, and one of only two in which moderate response percentages exceeded high ones. As we saw in Chapter 5 (Figre 5-6), research applications is one of the for areas for which help desks provide assistance least often, no dobt helping to explain why respondents rated the help desk s impact in this area so low. Respondents gave s well-assorted responses regarding the help desk s positive 90% 80% Percentage of Instittions in Category 70% 60% 50% 40% 30% 20% 10% Figre Extent to Which Help Desk Positively Impacts Selected Areas 0% Research activities (N = 397) Willingness to fnd central IT (N = 409) Workload of nit-specific IT staff (N = 390) Instrctional activities (N = 444) Workload of central IT staff (N = 437) Administrative activities (N = 449) Perceived vale of central IT (N = 446) Central IT reptation (N = 449) Low Moderate High Area of Impact Definitions: low = not at all + a little, moderate = somewhat, high = considerably + extensively EDUCAUSE Center for Applied Research 115

4 impact on the administration s willingness to fnd additional central IT initiatives. Slightly more said positive impact was high than said it was low, while nearly 4 in 10 said it was moderate. In all the other areas, positive impact was generally acknowledged to be high. Majorities said positive impact was high, except in the area of nit-specific IT staff workload. Respondents rated the help desk s positive impact highest for central IT reptation and central IT s perceived vale. Table 11-2 presents mean vales for positive help desk impact. It shows, for example, that mean positive impact on research, at 2.16, was jst above a little and central IT reptation, at 4.31, was nearly twice as great and approached the midpoint between considerably and extensively. In several areas, mean extent of the help desk s positive impact was significantly associated with other measres. Table 11-3 illstrates the strongest of those associations: Positive impact on central IT reptation, central IT s perceived vale, the workload of central and nit-specific IT staff, and the instittion s research activities were all higher where the help desk matrity level was toward the optimized end of or matrity spectrm. Positive impact was also significantly associated with strategic plan stats in the area of the instittion s research activities. Where the instittion had either a stand-alone strategic plan or a strategic plan integrated into the central IT strategic plan, the help desk s positive impact on the research area was significantly higher. Also in the research area, positive impact was significantly associated with agreement that the help desk ses metrics effectively to improve ser service for research activities. Higher positive impact was associated with a higher level of agreement. From these findings we can infer that having achieved a higher matrity level gives the help desk some advantage in positively impacting a range of camps activities. Along with matrity, having a strategic plan in place and sing metrics effectively to improve ser service are characteristics shared by many instittions whose respondents said their help desks positively impacted research activities. Overall Service Qality In general, or respondents took a positive tone in completing the sentence in or srvey that began, In terms of overall service qality, I believe or central IT help desk is... The list from which they chose inclded poor, fair, good, very good, and excellent. As Figre 11-2 illstrates, only 0.7 percent characterized their help desk service qality as poor, 11.4 percent said it was fair, 32.9 percent said it was good, 41.8 percent said it was very good, and 13.2 percent said it was excellent. When converted Table Extent of Help Desk s Positive Impact on Selected Areas Service Area N Mean* Std. Deviation Central IT reptation Perceived vale of central IT Administrative activities Workload of central IT staff Instrctional activities Workload of nit-specific IT staff Willingness to fnd central IT Research activities *Scale: 1 = not at all, 2 = a little, 3 = somewhat, 4 = considerably, 5 = extensively 116

5 Table Significant Positive Associations Between Extent of Positive Impact and Other Measres Agreement That Help Service Area Help Desk Matrity Level Strategic Plan Stats Desk Uses Metrics Effectively Central IT reptation X Perceived vale of central IT X Administrative activities Workload of central IT staff X Instrctional activities Workload of nit-specific IT staff X Willingness to fnd central IT Research activities X X X to a nmeric scale ranging from 1 to 5, the mean response was 3.55 (standard deviation 0.884), jst over halfway between good and very good, and the median response was 4, or very good. Using the same scale from poor to excellent, we asked respondents to tell s how they thoght central IT help desk sers wold rate the spport assistance provided for specific types of assistance. Table 11-4 lists those items and the reslts, which provide a finer-grained pictre of the perceived qality of help desk services. With the exception of assistance for research applications, the mean ratings for all types of assistance ranged between 3.26 and 3.79 roghly a qarter of a point above good to a qarter of a point below very good. The relatively low mean rating for assistance with research applications is consistent with the findings illstrated in Chapter 5 (Figre 5-6), which show that more than twothirds (67.4 percent) of respondent instittions reported providing assistance with research applications seldom or almost never. The finding abot research applications reported above helps explain that high percentage; it sggests that to some extent the research commnity was not contacting the central IT help desk for assistance with its applications becase, in or respondents estimation, researchers opinions of help desk assistance were relatively low. The low mean ratings of assistance for administrative and instrctional applications are of some concern. Along with research, these areas tend to be highly strategic for the instittion and figre prominently in instittional goals. Ratings of help desk services are higher in more tactical areas sch as spport for identity management, desktop essentials (sch as operating systems, secrity, and privacy), and commnication applications. We saw in Chapter 4 that providing infrastrctre and services that frther the instittion s strategic goals was the goal of more than two-thirds of or respondents central IT organizations. Ths, while overall service qality may be rated high, it concerns s that the help desk s services, as reflected in these mean ratings, appear not to be operating as well at the strategic level as at the tactical. 2 Service Qality and ITSM Practices Slightly higher levels of mean help desk service qality are significantly associated with the implementation of formal gidelines for the basic ITSM practices of capacity planning, availability planning, change manage- EDUCAUSE Center for Applied Research 117

6 Poor, 0.7% Excellent, 13.2% Fair, 11.4% Figre Overall Service Qality of Help Desk (N = 447) Good, 32.9% Very good, 41.8% Table Ratings of Help Desk Assistance Type of Assistance N Mean* Std. Deviation Identity management Desktop essentials Commnication applications Personal prodctivity applications Camps IT infrastrctre Instrctional applications Administrative applications Research applications ment, and release management. Table 11-5 presents the relationship: Among respondents who had adopted a given practice, we see a positive difference of abot a third of a point on a five-point qality scale over those who had not adopted that practice. By itself, SLA implementation, another ITSM-related practice, is not significantly associated with overall service qality. In Chapters 9 and 10, as part of or discssion of the partnership between the help desk and the central IT organization, we discssed the adeqacy of help desk staff inclsion in central IT s activities in for ITSM-related areas. As Table 11-6 shows, the greater the nmber of ITSM activity areas in which the help desk is adeqately inclded, the better the overall qality of help desk services. The data as presented above mask marked increases in the mean nmber of ITSM activities help desk staff are inclded in, especially at the high end of the qality scale. Figre 11-3 orients the data differently and reveals the pattern. Becase only three respondents reported poor service qality, we disregard 118

7 Table Overall Service Qality, by Adoption of Basic ITSM Practices ITSM Practice N Mean* Std. Deviation Capacity Planning No formal gidelines Formal gidelines in place Total System Availability Planning No formal gidelines Formal gidelines in place Total Change Management No formal gidelines Formal gidelines in place Total Release Management No formal gidelines Formal gidelines in place Total Table Overall Service Qality, by Nmber of For Basic ITSM Activities in Which Help Desk Is Adeqately Inclded Nmber of ITSM Activities That Inclde Help Desk N Mean* Std. Deviation None One Two Three For Total that finding; where qality is better, we see a clear pward trend in the nmber of ITSM activities with adeqate help desk inclsion. Respondents reporting excellent service qality also report nearly twice as many ITSM activities with adeqate help desk inclsion as those whose service qality is only fair. In smmary, these findings show that ITSM practices and help desk service qality go hand in hand. Adoption of formal gidelines for each of the ITSM practices we asked abot was associated with higher overall service qality. And the association between qality and the extent to which help desk personnel are inclded in the central IT organization s ITSM activities was even more dramatically positive. Service Qality and Basic Organizational Parameters The central IT help desk s overall service qality was significantly associated with several EDUCAUSE Center for Applied Research 119

8 Figre Mean Nmber of For Basic ITSM Activities in Which Help Desk Is Adeqately Inclded, by Overall Service Qality Mean Nmber of Activities Poor (N = 3) Fair (N = 51) Good (N = 147) Very good (N = 187) Excellent (N = 59) Overall Service Qality basic characteristics of the help desk, the central IT organization, and the instittion. As Table 11-7 shows, perceived adeqacy of help desk fnding increased from a mean of 2.94, jst below good, where fnding was seen as mch less than adeqate, to 3.92, jst below very good, where fnding was seen as more than adeqate. Only one respondent said help desk fnding was mch more than adeqate, so the mean of 3.0 at that level is best ignored. We fond no significant association between overall help desk service qality and the nmber of fll-time eqivalent staff working at the help desk. Service qality is also associated with the pace at which instittions and their central IT organizations said they adopted new technologies. Table 11-8 portrays the differences in overall service qality for each of three paces of adoption. In general, overall help desk service qality was a bit higher relative to the instittion s adoption pace than to the central IT organization s pace. Whichever entity s pace we looked at, mean service qality was highest among early adopters (3.81 to 3.89, closer to very good than to good). Among mainstream adopters, mean service qality was a third of a point lower (3.54 to 3.56, halfway between good and very good), and among late adopters it was three- or fortenths of a point lower still (3.15 to 3.32, relatively low in the good range). These findings sggest that at most respondent instittions the additional spport brden of early adoption of new technologies does not negatively impact overall service qality. We might even speclate that earlier adoption of new technologies lends a positive glow to help desk services, at least in or respondents eyes. Finally, Figre 11-4 shows that as help desk qality increased, so did mean agreement that camps expectations of the help desk were aligned with its resorces. Mean overall service qality was lower where respondents disagreed or strongly disagreed that camps expectations and help desk resorces were aligned, or where respondents were netral. In these categories, mean service qality ranged from 3.24 to 3.38, less 120

9 Table Overall Service Qality, by Adeqacy of Help Desk Fnding Adeqacy of Fnding N Mean* Std. Deviation Mch less than adeqate Less than adeqate Adeqate More than adeqate Mch more than adeqate N/A Total Table Overall Service Qality, by Pace of Adoption of New Technologies (N = 444) Adoption Pace N Mean* Std. Deviation Pace of Instittional Adoption Early adopter Mainstream adopter Late adopter Total Pace of IT Organization Adoption Early adopter Mainstream adopter Late adopter Total than halfway between good and very good. Where respondents agreed that expectations and resorces were aligned, service qality was mch closer to very good than to good. Among the 21 respondents who agreed strongly abot alignment, mean overall service qality was a third of the way between very good and excellent. It ths appears that where the alignment of camps expectations and help desk resorces is perceived as better, service qality is markedly better. Where alignment is seen as poorer, service qality is also poorer, thogh the relationship is more diffse. Service Qality and Help Desk Toolsets Overall help desk service qality is also associated with the richness of the varios toolsets the help desk ses. For example, in Table 11-9 we see that mean help desk service qality increases with increasing agreement that the instittion ses self-service tools effectively to redce help desk demand. We also fond that service qality increased as the nmber of atomated help desk fnctions increased and was significantly greater for respondent instittions whose atomated help desk fnctions were part of an integrated system. In general, service qality increased as the nmber of help desk staff spport tools implemented increased. The relationship between service qality and the nmber of ser tools the help desk had implemented was more complex: We fond a striking difference in qality between respondent instittions with no ser tools implemented (mean overall service qality of 3.30, stan- EDUCAUSE Center for Applied Research 121

10 Figre Mean Overall Service Qality, by Agreement That Camps Expectations and Help Desk Resorces Are Aligned Mean Overall Service Qality* Strongly disagree (N = 33) Disagree (N = 131) Netral (N = 98) Agree (N = 157) Strongly agree (N = 21) Level of Agreement Table Overall Service Qality, by Agreement Abot Effective Use of Self-Service Featres Level of Agreement N Mean* Std. Deviation Strongly disagree Disagree Netral Agree Strongly agree Total dard deviation 0.858) and the 14 instittions where all for had been implemented (mean 4.63, standard deviation 0.633), bt among respondents who had implemented one, two, or three sch tools, mean service qality was fairly niform, ranging from 3.62 to All the findings reported in this section reinforce the idea that better help desk toolsets are associated with better help desk service qality. Ths it appears that if qality help desk services are a goal, at least one contribtor to small improvements wold be investment in a robst set of help desk tools. Service Qality, Goals, and Planning Figre 11-5 shows the clear, significant association between mean overall service qality and the nmber of goals the help desk met often or almost always. Instittions reporting no help desk goals rated mean overall service qality at 3.04 (standard deviation 0.892), good on or five-point scale. Service qality increases steadily as the nmber of goals met increases. Where all nine goals were met often or almost always, mean overall service qality was 4.19 (standard deviation 0.681), jst over very good. 122

11 Mean Overall Service Qality* Figre Mean Overall Service Qality, by Nmber of Goals Met Often or Almost Always Nmber of Help Desk Goals Respondents meeting intermediate nmbers of goals had service qality ratings between these nmbers. Viewed as mean nmber of help desk goals met often or almost always, by overall service qality, this relationship holds as well. Among respondent instittions where service qality was poor or fair, the mean nmber of goals was 1.30 (standard deviation 1.500). Where qality was very good or excellent, the mean nmber of goals was three times higher, at 4.80 (standard deviation 2.940). Strategic planning and mean overall service qality were also significantly associated. Where no strategic plan was in place, mean overall service qality was 3.40 (standard deviation 0.895). Where a strategic plan for the help desk was integrated into the IT instittion s strategic plan, mean overall service qality was almost half a point higher, at 3.86 (standard deviation 0.776). Where the strategic plan for the help desk stood alone, mean service qality lay between these figres. These findings reinforce what seems obvios: A healthy set of goals and a strategic plan for the help desk go hand in hand with service qality. Service Qality and Metrics In or data set, positive associations are present bt not strong between mean overall help desk service qality and both the nmber of metrics the help desk analyzed and the nmber of constitencies to which it reported those metrics. On the other hand, respondents agreement that the help desk ses metrics effectively to improve ser service is more persasively associated with service qality (see Table 11-10). Respondents strongly disagreeing had a mean service qality of 2.86, jst nder good, while those strongly agreeing had a mean service qality of 4.05, jst over very good. We fond that reglarly analyzing more metrics and reporting them to more constitencies were not associated with impressive gains in overall help desk service qality. However, when we asked a more probing qestion abot the effectiveness of the help EDUCAUSE Center for Applied Research 123

12 Table Overall Service Qality, by Agreement That Help Desk Uses Metrics Effectively Level of Agreement N Mean* Std. Deviation Strongly disagree Disagree Netral Agree Strongly agree Total desk s se of those metrics, or respondents reported a more compelling positive association with service qality. While service qality may benefit from having metrics in place and reporting them to the appropriate entities, these findings sggest that these steps alone are not sfficient. Or analyses fond no significant association at all between service qality and either the nmber of methods sed to assess help desk ser satisfaction or the se of any of the individal assessment methods. While assessing ser satisfaction may very well have benefits in planning, steering, and evalating help desk services, it appears neither to drive overall help desk service qality nor to be driven by it. Service Qality and Commnication of Costs and Vale The extent to which the camps is aware of the central IT help desk s costs and vale is positively associated with help desk service qality. As Table shows, for each of the for statements we proposed, respondents who strongly agreed had sbstantially higher mean overall service qality than those who strongly disagreed. For the statements abot help desk costs, service qality among those who strongly agreed was abot nine-tenths of a point higher on or five-point scale. For the statements abot help desk vale, the difference was between 1.6 and 1.7 points. From these data it appears that sccessfl commnication of the costs and vale of help desk services goes hand in hand with service qality. Service Qality and Help Desk Matrity As discssed in Chapter 10, we asked respondents to characterize their central IT help desks in terms of their organizational matrity. Not srprisingly, matrity is dramatically associated with mean overall help desk service qality, as depicted in Figre Mean overall service qality at instittions whose help desks are at the optimized level is nearly two points higher on or five-point scale than among instittions with help desks at the initial level. This section s findings speak to a dramatic and statistically strong 3 association between overall service qality and help desk process matrity. As always, however, it is impossible from the analyses we have done to confidently infer the direction in which the association between these measres operates. Do more matre help desks reslt in higher-qality services? Does having a higher-qality service site incline the respondent to say the help desk is more matre? Or does some other factor or combination of factors positively impact both service qality and matrity? It is probably safe to sggest that help desk leaders who prse both matre practices and service qality will find synergy on their side. 124

13 Table Overall Service Qality, by Agreement Abot Commnication of Costs and Vale Level of Agreement N Mean* Std. Deviation Costs Are Well Docmented Strongly disagree Disagree Netral Agree Strongly agree Total Costs Are Well Understood Strongly disagree Disagree Netral Agree Strongly agree Total Vale Is Well Docmented Strongly disagree Disagree Netral Agree Strongly agree Total Vale Is Well Understood Strongly disagree Disagree Netral Agree Strongly agree Total Service Qality and Positive Impact of the Help Desk Looking at mean positive impact as it relates to help desk service qality, Figre 11-7 shows a general pattern of higher help desk service qality where the help desk has high positive impact and declining help desk service qality where positive impact is lower. For two areas, however, the difference was especially great: central IT s perceived vale and its reptation. In these cases, service qality appears to fall away qickly as the help desk s perceived positive impact declines. Again, we cannot say for certain in which direction this association operates, bt it seems intitive that where service qality is low, what sffers most is central IT s reptation and perceived vale. EDUCAUSE Center for Applied Research 125

14 Figre Mean Overall Service Qality, by Help Desk Matrity Level Mean Overall Service Qality* Initial (N = 35) Repeatable (N = 125) Standardized (N = 178) Managed (N = 78) Optimized (N = 30) Matrity Level 5.0 Figre Mean Overall Service Qality, by Extent to Which Help Desk Has Positive Impact Mean Overall Service Qality* Research activities Workload of nitspecific IT staff Willingness to fnd central IT Administrative activities Instrctional activities Workload of central IT staff Central IT reptation Perceived vale of central IT Limited positive impact Moderate positive impact Area of Impact Extensive positive impact Definitions: limited positive impact = not at all or a little, moderate = somewhat, extensive = considerably or extensively 126

15 Smmary and Implications In general, or respondent instittions did well at meeting their docmented goals, with mean freqencies of sccess ranging from 3.83, jst below often, to 4.67, not far below almost always. Goals met most often were for spport service availability, incidents resolved per nit time, incidents resolved per staff member per nit time, percentage of ser satisfaction, and calls resolved at first contact. Presmably these goals prevalence reflects their priority among most of or respondent instittions help desks. Among the nine areas we asked abot, the help desk had its greatest positive impact on central IT service reptation and perceived vale. Becase responses for these two similar areas were nearly identical, we assme that the distinction we were trying to draw, between doing things right and doing the right things, may have been nclear to many of or respondents. In any case, the work of the help desk seems to have a strong inflence pon perceptions abot central IT, reinforcing the idea that the help desk is in many practical ways the pblic relations face of the IT organization. The help desk also has a sbstantial effect on instrctional and administrative activities and the workloads of central and nit-specific IT staff. Srprisingly, the help desk had little positive impact on the camps administration s willingness to fnd additional IT initiatives, which sggests to s that sch fnding is most often based on other considerations. The help desk has the least positive impact in the area of spport for research activities. This is explained at least in part by or finding that the research area is one in which or respondents help desks least often provide assistance. Positive impact in several areas was higher where the help desk was more matre; in the research area, it was also higher where strategic plan stats was more advanced and where agreement was greater that the help desk ses metrics effectively to improve ser service. We speclate that help desks that try harder in these ways may improve their chances of providing beneficial services to the demanding research constitency. Overall service qality is or primary measre of help desk otcomes. In general, we fond that respondents were positive abot the qality of the services they provide, with more than half rating overall qality either very good or excellent. Only three respondents characterized their help desk qality as poor, and only 51 (11.4 percent) rated it as fair. While it might be tempting to interpret these low negative responses as exaggeration or braggadocio, we saw in or investigation of help desk matrity that nearly three times as many respondents (35.7 percent) were willing to rate themselves in the lower two of or five matrity categories. While there may still be some self-praising bias in respondents service qality ratings, the low nmbers reported for poor and fair overall service qality may also reflect natral selection at work: Help desks that provide trly poor-qality service may simply not srvive. When asked for their ratings of service qality in specific areas, respondents rated research spport lowest, midway between fair and good. This goes along with or finding that help desks report the research area as one in which they least often provide assistance; only 337 of or 454 respondents offer any assistance for research applications, and nearly half of them report providing sch assistance seldom or almost never. We don t know whether this is becase the help desk doesn t have the skills needed to assist in the research enterprise or is simply not asked to do so; we sspect that in some cases both explanations apply. Administrative and instrctional applications also received relatively low marks, averaging less than halfway between good and very good. These areas can be very demanding and are freqently spported by EDUCAUSE Center for Applied Research 127

16 specialized help desks; or findings may reflect both of those factors, bt they also sggest that the average respondent s help desk is of only tactical importance in these areas of strategic importance to the instittion. The remainder of the areas we asked abot IT infrastrctre, personal prodctivity applications, commnication applications, desktop essentials, and identity management were rated more than halfway between good and very good. Service qality was significantly positively associated with the adoption of formal gidelines for ITSM practices sch as capacity planning, availability planning, change management, and release management. In general, the more of these practices an IT organization adopts, the higher the level of service qality. The greater the extent to which the central IT organization incldes help desk staff in those activities, the better or respondent instittions mean overall help desk service qality. None of these findings is particlarly srprising; a primary goal of ITSM practices is the improvement of service qality. While or analyses don t lend themselves to identifying casality, these findings related to ITSM and help desk service qality at least give s no reason to sggest that ITSM does not work. The se of SLAs was not significantly associated with mean overall service qality, despite SLAs important role in ITSM. This does not sggest that SLAs are ineffective, thogh. Becase only 20.5 percent of or respondents are sing SLAs, sorting those responses into or five service qality categories reslted in statistically nsable sbsample sizes. Not srprisingly, service qality was also higher where help desk fnding was adeqate or better, the instittion and the central IT organization were more aggressive in adopting new technologies, the help desk s toolset was robst and integrated, a strategic plan for the help desk was in place, and respondents rated the help desk as more effective in the se of metrics to improve ser service. Reinforcing both conventional wisdom and ITSM doctrine, we fond that commnication between the help desk and the camps was also associated with help desk qality. In general, the better the instittion s help desk service qality, the more likely respondents were to agree that help desk costs and vale were well docmented and well nderstood by the help desk s constitents. Overall help desk service qality varies dramatically with the help desk matrity rating. The more matre the help desk, the higher the service qality rating. This association might owe some of its strength to some respondents failre to distingish between help desk matrity and overall service qality; for them, or two qestions wold have been asking the same thing, and their responses wold have been highly correlated. We discont this explanation for two reasons. First, while or qestion abot service qality offered no definitions for its options poor throgh excellent, we did provide clear definitions for the matrity-level options we offered, and only one of them mentioned qality. Second, as mentioned above, the distribtion of responses to the qality qestion was mch more generally positive than for the matrity qestion. Finally, in all eight of the service areas we asked abot, greater positive impact is associated with higher service qality levels. This trend is consistent across the dataset, althogh in the areas of central IT s reptation and perceived vale, qality is more dramatically associated with positive impact than in the others. Or interpretation is that while the other areas appear to reflect service qality to some extent, qality heavily inflences reptation and perceived vale. 128

17 Endnotes 1. Note that or initial five-point scale inclded the responses not at all, a little, somewhat, considerably, and extensively. To simplify presentation we compacted that scale into a three-point scale inclding low ( not at all + a little ), moderate ( somewhat ), and high ( considerably + extensively ). The significance of the statistical relationships within the data was not meaningflly changed by this compression of the response scale. 2. For a discssion of the help desk s strategic vale, see the accompanying case stdy: Jdith A. Pirani and Robert Albrecht, Bowdoin College and Colgate University: Using the Help Desk Strategically to Revitalize the IT Organization (Case Stdy 7) (Bolder, CO: EDUCAUSE Center for Applied Research, 2007), available from edcase.ed/ecar. 3. The ANOVA significance of this association is.000, indicating a vanishingly small probability that the relationships among the nmbers are de to chance. ECAR seldom reports associations whose significance is higher than.000. For this association, as for all or comparisons of means, we also calclated Eta, a coefficient of nonlinear variation that speaks to the strength of an association. For this association, Eta was.553, indicating one of the strongest associations among all those reported in this stdy. EDUCAUSE Center for Applied Research 129