Acheck of vital signs is the first step in any

Transcription

1 Bechmarkig to Iform Plaig The EDUCAUSE Core Data Service By Leah Lag Acheck of vital sigs is the first step i ay physical examiatio. A urse measures ad records curret blood pressure, heart rate, respiratio rate, ad temperature: for example, 110 over 79, 77 beats per miute, 13 breaths per miute, ad 97.8 degrees. These four metrics are the iitial idicators of the patiet s well-beig; however, o their ow they are ambiguous. To evaluate whether these metrics idicate health or cause for cocer, medical practitioers must also be familiar with a set of bechmarks based o ormal huma physiology: 120 over 80, 50 to 80 beats per miute, 16 to 20 breaths per miute, ad 98.6 degrees. Whe vital sigs fall outside of a acceptable rage of these bechmarks, practitioers begi a ivestigatio for uderlyig causes. 40 EducausEreview MAY/JUNE 2015 Illustratio by Mark Alle Miller, 2015

2 Bechmarkig to Iform Plaig: The EDUCAUSE Core Data Service Metrics ad bechmarks are ot just for medical professioals; they ca be used by ayoe i ay field. Istitutioal leaders ca use metrics ad bechmarks to documet curret circumstaces ad evaluate orgaizatioal health. Geerally, metric is defied as a stadard of measuremet. A metric ca be used to help uderstad a orgaizatio. The term bechmark is defied as somethig that ca be used as a way to judge the quality or level of other, similar thigs. Most commoly, bechmarkig is thought of as comparig a metric with a gold stadard or best practice, or other iteral or exteral referece poit, as a way to evaluate a orgaizatio. Why Bechmark? To ru their istitutios efficietly ad effectively, leaders at today s colleges ad uiversities must have a clear uderstadig of their ow fiacial, staffig, ad operatioal status. This iformatio is critical to makig iformed decisios ad optimizig the impact of iformatio techology. Leaders ca ehace their decisio makig by evaluatig their fiacial, staffig, ad operatioal status agaist comparable iformatio for peers, aspirat peers, or eve their ow istitutio. For example, a metric o total cetral IT spedig per istitutioal FTE (studets, faculty, ad staff) ca be used to estimate the amout of IT support that idividuals at a istitutio receive. Bechmarkig this metric agaist peer istitutios or a istitutio s past performace ca help leaders make the case for additioal resources. I fact, leaders at both Marylhurst Uiversity ad America Uiversity did just that. Durig a orgaizatioal trasitio at Marylhurst Uiversity, the IT departmet used metrics ad bechmarks from the EDUCAUSE Core Data Service (CDS) to itroduce ew istitutioal leaders to the curret IT eviromet ad set the cotext for future plas. 1 Natha Phillips, Director of the Ceter for Learig ad Techology at Marylhurst Uiversity, reported: CDS data helped us provide a strog case for icreasig our support services, sice we have bee uderstaffed compared with Sice 2002, the EDUCAUSE Core Data Service (CDS) has bee providig higher educatio CIOs ad seior IT leaders with the bechmarks they eed to make strategic decisios about iformatio techology at their istitutios. O average, more tha 800 istitutios (both withi ad outside the Uited States) participate i a survey about IT fiacials, staffig, ad services. For their time ad effort, survey participats are rewarded with access to CDS Reportig, a self-service tool that eables istitutioal leaders to bechmark their IT orgaizatios agaist those of their peers. I additio to gaiig access to CDS Reportig, IT leaders also participate i the CDS for the followig reasos: To study their IT orgaizatio To bechmark agaist past performace To look at treds over time To start gatherig ad usig metrics To have data available just i case Figure 1. How to Bechmark 7. Improve. 6. Cosider possible outcomes. 5. Develop a pla for reportig. 2. Idetify the audiece. 1. Idetify goals. 3. Idetify data sources. 4. Evaluate data quality. similarly sized istitutios. Similarly, America Uiversity tracks resource allocatio over time agaist CDS bechmarks. Accordig to Terry Feradez, Seior Director of IT Customer Service ad Support at America Uiversity: Each year whe the results are released, we have used them to bechmark ourselves agaist our peer istitutios. It has bee most helpful to compare staffig levels ad budgets for the IT orgaizatio as a whole, as well as idividual focus areas. We have bee able to successfully advocate for additioal or reallocated resources, based o the ormalized data. How to Bechmark Colleges ad uiversities that bechmark successfully follow seve steps (see Figure 1). 2 Step 1. Idetify goals. Geeral bechmarkig assessmets will start with goals such as idetify best practices or commuicate the value of iformatio techology. More specific bechmarkig assessmets may aim to track progress o strategic iitiatives or the 42 EducausEreview MAY/JUNE 2015

3 EDUCAUSE 2015 Top 10 IT Issues. 3 Questios to ask iclude the followig: What is the purpose of this bechmarkig assessmet? If there are several goals, how are these goals prioritized? Step 2. Idetify the audiece. It is importat to cosider who is requestig the iformatio ad to have a idea of their motivatios, their uderstadig of the cotet, ad their preferece for cosumig iformatio. These details will be useful i rightsizig the bechmarkig assessmet. For example, a data-savvy audiece with missio-critical cocers may require more data accuracy tha a group of data ovices who are just startig to use metrics. Questios to ask iclude the followig: Is data beig gathered for leaders, users, or service owers? To what extet will the audiece rely o this data to iform decisio makig? What level of specificity i the data will the audiece expect? Step 3. Idetify data sources. Data ca be foud aywhere from server logs to aual reports to exteral sources like the EDUCAUSE CDS. The data used will largely deped o the goals established i Step 1. Table 1 shows a example of how CDS data ad bechmarks ca be used to track progress o the EDU- CAUSE 2015 Top 10 IT Issues. Questios to ask iclude the followig: What data is available today? Is additioal data eeded to achieve the goals established i Step 1? Agaist which groups or istitutios will the compariso be made? Demographic peers? Geographic peers? Aspiratioal peers? Step 4. Evaluate data quality. Oce the data has bee gathered, the ext step is to determie whether it cotais ay errors. Also, cosider whether the data Table 1. Usig Cds Data to Track Progress o the Educause 2015 Top 10 IT Issues 2015 Top 10 IT Issue Supportig Metrics 1 Hirig ad Retaiig Qualified Cetral IT traiig spedig per cetral IT staff Staff, ad Updatig the FTE Kowledge ad Skills of Existig Cetral IT FTEs per 1,000 istitutioal FTEs Techology Staff 2 Optimizig the Use of Most commo teachig ad learig support Techology i Teachig ad services for faculty Learig i Collaboratio with Academic Leadership, Icludig Uderstadig the Appropriate Level of Techology to Use 3 Developig IT Fudig Models Percetage of cetral IT spedig o ruig, That Sustai Core Service, growig, ad trasformig the istitutio Support Iovatio, ad Cetral IT compesatio, ocompesatio, Facilitate Growth ad capital spedig as a percetage of total cetral IT spedig Cetral IT outsourcig spedig as a percetage of total cetral IT spedig 4 Improvig Studet Outcomes Systems most commoly mobile friedly through a Istitutioal Approach That Strategically Leverages Techology 5 Demostratig the Busiess Total cetral IT spedig per istitutioal FTE Value of Iformatio Techology (studets, faculty, ad staff) ad How Techology ad the Total cetral IT spedig as percetage of IT Orgaizatio Ca Help the istitutioal expeses Istitutio Achieve Its Goals 6 Icreasig the IT Orgaizatio s Systems most likely to be replaced i the ext Capacity for Maagig Chage, three years Despite Differig Commuity Needs, Priorities, ad Abilities 7 Providig User Support i the Istitutios offerig tier 2/level 2 service or New Normal Mobile, Olie higher Educatio, Cloud, ad BYOD Istitutios offerig self-service help optios Eviromets Access poits that are or ac Systems most commoly mobile friedly 8 Developig Mobile, Cloud, Istitutios with madatory iformatio ad Digital Security Policies security traiig That Work for Most of the Istitutioal Commuity 9 Developig a Eterprise IT Systems most likely to be replaced i the ext Architecture That Ca Respod three years to Chagig Coditios ad New Opportuities 10 Balacig Agility, Opeess, ad Istitutios that have coducted ay sort of IT Security security risk assessmet MAY/JUNE 2015 Educausereview 43

4 Bechmarkig to Iform Plaig: The EDUCAUSE Core Data Service eeds to be reformatted or whether secodary variables must be calculated usig the origial data source. Evaluatig data quality is importat eve whe usig CDS data, which is selfreported through a aual survey. Those who aalyze CDS data should evaluate whether the budget ad staffig umbers reported are i lie with what is expected, ad they should report whe data look suspicious. 4 Istitutioal leaders are also ecouraged to reach The first step toward strategically fudig iformatio techology is to idetify IT budget parameters based o the type, the populatio, ad the budget of the istitutio. out to peers to validate data ad better uderstad the cotext behid possible outliers. Feradez, from America Uiversity, has oted: We have bee able to use the data to make more iformed decisios ad follow up directly with peer istitutios that have similar circumstaces. Questios to ask iclude the followig: Are there ay outliers? Why are they outliers? Is the data source trustworthy, or does the data eed to be validated before usig it? Step 5. Develop a pla for reportig. Whe reportig results, preseters must cosider how the iformatio will be shared ad whether the data will be reported at the right frequecy, i the right formats, ad to the right people. They must cosider how the data should be cotextualized ad what groudwork should be set to explai the itet ad limitatios of the data. With ew bechmarkig assessmets, it may be wise to test the iitial report with a iteral audiece ad with peers from other istitutios to gather feedback about data quality, settig a appropriate cotext aroud the data ad the report format, before sharig results with executives. This may prevet ay out-of-had dismissal of results a reactio that ca occur whe the audiece has doubts about the data. It ca also esure that IT maagers are aware ad supportive of sharig their service s bechmarks with istitutioal leadership. Questios to ask iclude the followig: Will this iformatio be preseted i perso or i writig? What backgroud or cotextual iformatio should be preseted to explai the data ad its limitatios? How ofte will it be ecessary to refresh ad reproduce this report? Step 6. Cosider possible outcomes. A critical compoet of ay bechmarkig assessmet is a pla for how the results will be used. Bechmarkig is ofte used to idetify aomalous circumstaces; uexpected results should be the startig poit of a ivestigatio. Key stakeholders should be i agreemet o courses of actio ad possible outcomes. I additio, before reportig the results of a assessmet, preseters should review the pla ad take a few momets to aticipate how the results may ifluece decisio makig. Questios to ask iclude the followig: What decisios eed to be made with this iformatio? Is the report clearly drivig toward those decisios? Will the audiece wat to take actio after hearig the results? What additioal ivestigatios could be coducted to lear more about aomalous results? Step 7. Improve. Fially, i the ame of cotiuous improvemet, it is always importat to explore optios for improvig ay or all elemets of a bechmarkig assessmet. Questios to ask iclude the followig: Was the data that was icluded i the aalysis the right data to use? Was the report or bechmarkig tool useful? Should ay additioal audieces see the report ext time? Bechmarkig Fiacials, Staffig, ad Services The first step toward strategically fudig iformatio techology is to idetify IT budget parameters based o the type, the populatio, ad the budget of the istitutio. The, based o istitutioal priorities ad the curret IT eviromet, IT leaders ca determie a spedig portfolio that will get them to where they wat to be. Breakig the budget dow by dollars spet (1) o ruig, growig, ad trasformig the istitutio, (2) o each IT domai area, ad (3) for capital ad operatig work will help them determie the right bled of iovatio spedig to operatig spedig 44 EducausEreview MAY/JUNE 2015

5 Bechmarkig to Iform Plaig: The EDUCAUSE Core Data Service for all areas of iformatio techology. Bechmarks o IT fiacials ca help IT leaders aswer the followig questios: What is a practical rage for total IT budget based o the type, the populatio, ad the budget of the istitutio? Are chages i budget from the previous fiscal year i lie with chages i peer budgets? What is a appropriate distributio of spedig for the istitutio? Staffig models are evolvig. Hirig ad retaiig qualified staff, ad updatig the kowledge ad skills of existig techology staff, is the #1 Top 10 IT Issue i Services are beig outsourced, but istitutios eed staff to maage outsourcig, ad they eed more services ad badwidth to support BYOD. Does this mea fewer staff, or does it mea similar umbers of staff with differet skillsets? Through this evolutio, IT leaders will wat to keep a eye o several bechmarks: ratio of cetral IT staff to istitutioal FTE; studet workers as a percetage of total cetral IT FTE; percetage of IT staff across IT domai areas; ad traiig spedig per IT staff. Keepig a eye o how others are staffed ad kowig how peers balace their staff portfolio ca help IT leaders fid the right fit. Kowig what peers are spedig o staff traiig ca likewise help IT leaders budget for updatig the skillsets of existig staff. Bechmarks o IT staffig ca help IT leaders aswer the followig questios: What is a practical rage for staff size based o istitutio type ad size? What is the right bled of staff? What is a appropriate budget to retrai curret staff? Fially, i a chagig eviromet it is importat to kow which services are i demad ad which are fadig i importace, which services should stay local ad which ca be outsourced, ad which services must have mobile deploymet or be accessible via the cloud. Providig the right services i the most efficiet maer is critical. The EDUCAUSE CDS has data that ca help IT leaders uderstad how peers are supportig users i mobile computig, olie educatio, ad cloud/ BYOD eviromets. CDS data o faculty support services ca help IT leaders determie how to help faculty optimize the use of techology i teachig ad learig, whereas data (icludig vedor/ product, deploymet, ad maagemet strategy) o fifty differet iformatio systems ca help IT leaders strategize for a eterprise architecture that is right for the istitutio. Bechmarks o IT services ca help IT leaders aswer the followig questios: A New Geeratio of IT Assessmet Tools Aew culture of data-drive decisio makig, together with risig pressure from boards ad legislatures to measure istitutioal performace, is feedig demad for better, cheaper, ad faster ways to assess iformatio techology. With support from the Lumia Foudatio, EDUCAUSE is workig to improve the value ad efficiecy of the cotributio of iformatio techology to higher educatio by developig a iovative ew service for IT bechmarkig ad assessmet. This ew olie service will help IT leaders commuicate the value ad relevace of iformatio techology to o-it leaders, as well as help them assess ad bechmark maturity ad techology deploymet for a suite of strategic iitiatives. Maturity idices measure the capability to deliver IT services ad applicatios i a give area. They examie multiple dimesios of progress, techical ad otechical such as culture, process, expertise, ivestmet, ad goverace. They eable istitutioal leaders to determie where they are i deliverig a service, ad where they aspire to be. Deploymet idices measure the degree to which a istitutio has deployed the techologies related to deliverig a service, based o a stadard scale reflectig stages of deploymet. This olie service will build o the CDS database ad reportig tools ad will leverage other EDUCAUSE data ad aalytical frameworks to create a sapshot of iteral performace ad to allow compariso with peer istitutios or other desired compariso groups. Developmet ad prototypig of the ew service is uder way, ad the service will eter productio i the first quarter of What services should be provided? How should the istitutio provide those services? How ca service owers evaluate service efficiecy or effectiveess? What other istitutios are usig the same vedor (or are cosiderig the same vedor) for a particular applicatio? Next Steps The followig recommedatios ca help IT leaders take full advatage of the EDUCAUSE CDS. Measure distributed IT resources. To make iformed IT decisios, istitutioal leaders eed a full picture of iformatio techology campus-wide, ot just of the cetral IT orgaizatio. However, data o distributed IT resources remais elusive. I partership with NACUBO, the ECAR-COST Workig Group ( is curretly researchig barriers to gatherig this data ad is developig strategies to 46 EducausEreview MAY/JUNE 2015

6 Bechmarkig to Iform Plaig: The EDUCAUSE Core Data Service overcome the barriers. As a first step, CIOs are ecouraged to work with istitutioal leadership, icludig huma resources, to develop practices for gatherig more reliable iformatio about distributed IT resources. The CDS database is oly as strog as the umber of istitutios that cotribute data each year. The bechmarkig capability of the data is maximized with robust peer groups. Participate. From the Istitutioal Research departmet to the Huma Resources, comparative data about resources is beig collected at all levels of the istitutio. Leaders ca stay ahead of the game ad i cotrol of the IT bechmarkig at their istitutio by participatig i CDS. They should cosider addig data to the ext CDS survey, lauchig i July Use the data. Ayoe ca bechmark a istitutio agaist peer istitutios i similar Caregie Classificatios with the 2014 CDS Almaacs ( educause-core-data-service almaacs) or the 2014 CDS Bechmarkig Report ( library/resources/2014-cds-bechmark ig-report), a customizable slide presetatio that ca be used to itegrate core data ito campus presetatios ad strategic plaig. I additio to these publicly available resources, those at participatig istitutios ca use the data foud i CDS Reportig to iform their plaig. 6 Ted Bross, Associate Director for Admiistrative Iformatio Services at Priceto Uiversity, otes: While aggregatio of data puts a reasoable framework aroud large data sets ad ofte helps to suggest coclusios or fidigs, the importace of lookig at the raw data... caot be overestimated. Ecourage peers. The CDS database is oly as strog as the umber of istitutios that cotribute data each year. The bechmarkig capability of the data is maximized with robust peer groups. Ecouragig peers to participate i the survey ad to use the data esures a rich database for all. Stay iformed. EDUCAUSE offers umerous ways for IT leaders to fid iformatio about survey ope ad close dates, about completig the survey, about usig CDS data, ad much more. 7 The ladscape of higher educatio is chagig rapidly. As a facilitator of process chage, efficiecy gais, ad eriched learig, iformatio techology clearly adds value to the higher educatio istitutio. Usig metrics to bechmark higher educatio IT fiacials, staffig, ad services ca add further value by iformig ad reiforcig the decisio-makig process throughout this jourey. Notes 1. For more o how higher educatio IT leaders use CDS to iform their plaig, see the recorded webiar The EDUCAUSE Core Data Service: Core Metrics i Actio, educause-core-data-service-core-metrics-actio. 2. For more iformatio about gettig started with bechmarkig, view the short EDUCAUSE video Bechmarkig for CIOs, Susa Grajek ad the EDUCAUSE IT Issues Pael, Top 10 IT Issues, 2015: Iflectio Poit, EDUCAUSE Review, vol. 50, o. 1 (Jauary/February 2015), 4. Suspicious data may be reported by to coredata@educause.edu. 5. See Grajek et al., Top 10 IT Issues, For a itroductio to how to use CDS Reportig, view the CDS Reportig Demo video, cds-reportig-demo. 7. Subscribe to mothly ewsletters ( educause.edu/apps/coredata/ews/get_ .asp); joi the CDS discussio list ( COMMUNITY&A=1); atted CDS Office Hours ( ad explore the CDS website ( Leah Lag. The text of this article is licesed uder the Creative Commos Attributio- NoCommercial-NoDerivatives 4.0 Iteratioal Licese ( Leah Lag (llag@educause.edu) is Director of Aalytics Services for EDUCAUSE. 48 EducausEreview MAY/JUNE 2015