Design your own. KPI based Scorecard. Wednesday September PM 6.00 PM Mustang 6

Transcription

1 Design your own KPI based Scorecard Wednesday September PM 6.00 PM Mustang 6

2 Workshop overview This session is designed for professionals who operate in the field of IT performance management and allows them to create a KPI based scorecard using KPIs that actually make sense to the business The following are the objectives for this session: Understand the principles of successful IT Performance Management Are able to identify what KPIs are relevant to their business Learn to identify quality KPIs Using a Software as a Service environment can configure a scorecard and dashboard Are able to interpret information in a dashboard and use this for effective decision making This workshop focuses on the process for the creation of an IT KPI based scorecard and will not go into the details of data collection, data quality or data transformation 2

3 Workshop Agenda Session Title Description Type Time 1 The current state of IT Performance management in your organization Introduction of the participants and Discussion on maturity of IT Performance Management in your organization Assignment PM 2 Current state of the industry An overview of the current state of the IT Performance Management industry Presentation and Group Discussion PM 3 IT Performance Management An overview of the IT Performance Management process, terminology and concepts Presentation PM 4a How to Defining IT Performance management objectives and identifying key processes relevant to your organization Presentation b Define / Measure Define requirements for IT Performance Management: objectives, audience, scope Individual Exercise PM 5 Scorecard Design Create an IT KPI based Scorecard based on the current state of your organization and your requirements Presentation and Group Exercise PM 6 Review of Scorecards / Dashboards Analysis of a number of Scorecard and Dashboard examples Presentation and Group Discussion PM 7 Questions and Answers Q and A session and definition of next steps Group Discussion PM 3

4 Session 1 CURRENT STATE OF YOUR ORGANIZATIONS IT PERFORMANCE

5 The current state of IT Performance Management in your organization Q1) What is your opinion on the following statement: For too many organizations IT is a black box. Projects and systems are so complex that few CIOs can predict a direct impact on the business, making it hard to win budget and resources even in prosperous times Agree Agree to some extent Disagree Not sure Source: Information Week Hunting the Elusive IT Dashboard

6 The current state of IT Performance Management in your organization Q2) To what extent does IT management in your organization / your customers have a holistic picture on and decision making information about the performance of IT? They don t have a clear picture They could, but information is fragmented and dispersed over tools and reports They do, but a lot of effort goes into consolidating various reports into one view They have a holistic picture of their overall IT performance thanks to one or few regular concise reports There is no clear trend to be distinguished

7 The current state of IT Performance Management in your organization Q3) How would you rate the maturity level of your organization (or that of your customers) in terms of its IT performance management. Level 0: IT Performance Management is non-existent. Level 1: There is ad hoc monitoring in isolated areas. Level 2: Some measures are set with a clear link to business goals but are not communicated. Measurement processes emerge, but are not consistently applied. Level 3: Efficiency and effectiveness are measured and communicated and linked to business goals and the IT strategic plan. Continuous improvement is emerging. Level 4: There is an integrated performance measurement system linking IT performance to business goals by global application of a documented framework. Continuous improvement is a way of life.

8 The current state of IT Performance Management in your organization Q4) What is the main driver for IT Performance Management? Reduction of IT cost Establish the progress toward achieving goals Optimal resource allocation Be compliant with internal or external regulations Require an insight into performance against service levels Identify internal improvement opportunities Other

9 The current state of IT Performance Management in your organization Q5) What do you consider to be the key success factors for effective IT performance management? Having support from the business Proper guidance on the implementation Having a formalized service management framework in place (ITIL, COBIT) Having a useful software application to capture and convey performance measures Having an industry standard for metrics (common language) Having the proper resources in place (budget, people, infrastructure) Other. Please specify:

10 The current state of IT Performance Management in your organization Q6) Which tools are you currently using to report IT performance? Business Intelligence (e.g. IBM, Hyperion, SAP) Manual reporting (e.g. Excel) Internally developed software solution IT performance solution from external vendor (Metricus, M42 ) IT service management solutions (HP OpenView, BMC Remedy ) Service Level Management Monitoring Tools (Digital fuel, Oblicore...) Other. Please specify: We don t use any tools to report IT performance

11 Session 2 CURRENT STATE OF IT PERFORMANCE MANAGEMENT

12 IT Performance Management market maturity 65% of IT Managers agree with an InformationWeek statement that says IT managers have no idea what is happening inside their IT organization (according to an ITpreneurs survey) For too many organizations IT is a black box. Projects and systems are so complex that few CIOs can predict a direct impact on the business, making it hard to win budget and resources even in prosperous times. And when the CIO can't get a clear picture of the real-time data that underlies critical applications, infrastructure, and projects, IT too often ends up reacting to issues after users and customers are having problems. (Information Week March 2008) Source: Information Week Hunting the Elusive IT Dashboard Source: Trends in IT Performance Management, 2008/9, ITpreneurs survey and interviews among 99 IT executives and consultants 12

13 IT Performance Management market maturity The majority of organizations are not reporting on IT Performance due to the time and energy involved in doing this They could, but information is fragmented and dispersed over tools and reports They do, but a lot of effort goes into consolidating various reports into one view They have a holistic picture of their overall IT performance thanks to one or few regular concise reports Collecting data from various sources Converting data into logical numbers Building complex Excel sheets Highly Ineffective They don't have a clear picture 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% Source: Trends in IT Performance Management, 2008/9, ITpreneurs Survey and interviews amongst IT executives and consultants Percentage 2009 Percentage

14 IT Performance Management market maturity IT Performance Measurement is emerging and the basics are getting in place but there is no proper communication as yet 45% 40% 35% 30% 25% 20% 15% 10% 5% Level 0 Level 1 Level 2 Level 3 Level 4 IT Performance Management is non-existent. There is ad hoc monitoring in isolated areas. Some measures are set with a clear link to business goals but are not communicated. Measurement processes emerge, but are not consistently applied. Efficiency and effectiveness are measured and communicated and linked to business goals and the IT strategic plan. Continuous improvement is emerging. There is an integrated performance measurement system linking IT performance to business goals by global application of a documented framework. Continuous improvement is a way of life. 0% Level 0 Level 1 Level 2 Level 3 Level 4 Note: this is a self assessment Source: Trends in IT Performance Management, 2008/9, ITpreneurs Survey and interviews amongst IT executives and consultants 14

15 IT Performance Management market maturity The reduction of cost and the creation of clarity on the performance of IT in relation to business goals are the main drivers for IT Performance management initiatives Reduction of IT cost Establish the progress toward achieving goals Optimal resource allocation Require an insight into performance against service levels Identify internal improvement opportunities Be compliant with internal or external regulations Other 0% 20% 40% 60% 80% Source: Trends in IT Performance Management, 2008/9, ITpreneurs Survey and interviews IT executives and consultants 15

16 IT Performance Management market maturity Organizations look at IT best practices for support in identifying metrics and realize they need to have both resources in place and buy-in from the business Having a formalized service management framework in place (ITIL, COBIT) Having the proper resources in place (budget, people, infrastructure) Having support from the business Having an industry standard for metrics (common language) Having a useful software application to capture and convey performance measures Proper guidance on the implementation Other 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% percentage 2009 percentage 2008 Source: Trends in IT Performance Management, 2008/9, ITpreneurs Survey and interviews amongst IT executives and consultants 16

17 IT Performance Management market maturity Manual reporting using Excel is favored in most organizations today followed by the reporting capabilities provided by service management tools Manual reporting (e.g. Excel) IT service management solutions (HP OpenView, BMC Remedy) Internally developed software solution Business Intelligence (e.g. IBM, Hyperion, SAP) Service Level Management Monitoring Tools (Digital fuel, Oblicore) Other We don't use any tools to report IT performance IT performance solution from external vendor (Metricus, M42 ) 0% 10% 20% 30% 40% 50% 60% 70% 80% Percentage 2009 percentage 2008 Source: Trends in IT Performance Management, 2009, ITpreneurs Survey and interviews IT executives and consultants 17

18 Session 3 IT PERFORMANCE MANAGEMENT PROCESS 18

19 The IT Performance Management Process IT Performance Management is about measuring, improving, and demonstrating the value of IT IT Performance Management is the effective combination of methods, metrics, data, and tools that enables organizations to define KPIs that are relevant to them, understand their current performance against predetermined goals, and enables organizations to build on this information, initiate improvement activities, and achieve optimal IT performance in line with business requirements * Metricus definition for IT Performance Management 19

20 The IT Performance Management Process IT Performance Management is closely related to IT performance measurement. They are sometimes mistaken for each other. Strictly speaking, Performance Management is the larger domain and includes performance measurement as a component. IT Performance Management Define Measure Manage Improve IT Performance Measurement * Wikipedia Performance measurement is the process of assessing progress toward achieving predetermined goals*. Performance Management builds on that process, adding the relevant communication and action on the progress achieved against these predetermined goals. 20

21 The IT Performance Management Process In this workshop we will touch on the first three phases of the IT Performance Management lifecycle: define, measure and manage Define Define what is important to you and what you should measure Initiate improvement activities based on historic data and scenario planning Improve Measure Measure your IT performance through best practice KPIs and performance data Manage Manage the ongoing process and present decision making information to relevant stakeholders in your organization 21

22 What drives IT Performance Management What is not defined cannot be controlled. What is not controlled cannot be measured what is not measured cannot be managed or improved Just like other business departments, IT has to continuously improve and ensure alignment with the business Ultimately the only way for IT management to demonstrate value and control is by defining, measuring and managing IT performance A great idea, but the idea often gets stuck at not being able to successfully measure IT performance and not being able to bring everything together into a view that allows the IT management to take informed IT decisions. 22

23 What drives IT Performance Management Business Strategy drives IT strategy and results in defining, managing and optimizing IT processes and activities This needs to be balanced against the availability of quality data and processes that can provide information from the bottom up drives drives drives Business Strategy IT Strategy IT Processes IT Activities IT Balanced Scorecards CIO Dashboard / IT process health ITIL/COBIT Process Scorecards IT Performance Analysis IT Performance Metrics and Measures 23

24 Structured KPIs drive performance management for various users IT KPIs (that are aligned with the business strategy) play a role at each decision-making level in the organization and enable the organization to take informed IT decisions in line with business requirements Business Strategy Business Leadership IT Balanced Scorecard IT Strategy IT management drives IT process health dashboards drives Timely, accurate, and quality IT metrics and measures (KPIs) Processes and Activities IT departments Functional and process scorecards 24

25 From Measures to Decisions Operational Data Measures KPIs Scorecards Dashboards Decisions! IT Balanced Scorecard 25

26 Operational Data Source An Operational Data Source (ODS) represents data collected by a system or process associated with the delivery and support of IT services. A comprehensive understanding of ODS's within the context of the information ecosystem of an organization is essential to the structured definition and development of IT Service Metrics. ODS examples Service Desk Manual Custom Enterprise Systems Management Enterprise Resource Planning e.g. FrontRange HEAT, HP Service Manager, ServiceCenter, CA Unicenter, Service Plus, Servicedesk, Numara Footprint / Trackit, In house applications, Open Source applications Decentralized, disparate data related to specific users/processes Internal IT applications developed specifically for IT support/delivery functions e.g. IBM Tivoli, HP OpenView suite, CA Unicenter, Nagios, Open Source Applications e.g. SAP, Oracle Financials 26

27 Data Management Business objectives change, which has an impact on the requirements for data. Thus keep in mind that this process is dynamic and requires continual management Change IT processes and activities Data Integration IT strategy Data integration process Data Warehouse Insight? KPI based decision making information 27

28 Measures Measures are distinct sets of data derived from mathematical calculations. Measures are quantifiable, for example, size, volume, or percentage, and involve aggregation of data elements, for example, sum, average, min, max, or count. In and of themselves, measures may or may not be meaningful. However, measures represent building blocks for the metrics required to make business decisions. Examples: Calculation Sum (incidents) Sum (Incidents Resolved) where Met Target = Y Average (Database Availability) Sum (IT Resources) where Trained = Y Measure Volume of incidents Volume of incidents resolved within target resolution time % Database Availability Number of IT resources trained 28

29 Metrics Metrics consist of one or more measures combined with a mathematical calculation and a standard presentation (format) for the output. Dimensions Time Daily Categorization Priority = Critical Measures facilitates comparison functional categorization Volume of Incidents Resolved Within Target Resolution Time Volume of Incidents division Metric % Critical Incidents Resolved Within Target Metrics are associated with two dimensions: a time dimension and a functional categorization dimension. Metrics are used in the quantitative and periodic assessment of a process that is to be measured. Metrics should be associated with targets that are set based on specific business objectives. Metrics are associated with procedures to determine the measures required and procedures for the interpretation of metric results. 29

30 Acronyms that are used for metrics Numerous terms and acronyms exists for labeling metrics Impact related - Key Performance Indicator (KPI), Critical Success Factor (CSF), Outcome Measure, Performance Indicator Operational related IT Service Level Agreement SLA, IT Operational Level Agreement (OLA), and Service Level Objective (SLO) Term Metric often used interchangeably with Target, Benchmark and Goal Clear definitions are required in order to facilitate the necessary communications, and to set appropriate expectations, with users. CSF Outcome Measure IT Performance Metric KPI OLO SLA SLO 30

31 Characteristics of good metrics 31

32 Metric Presentation Concept Description Dashboard A dashboard is a graphical display of the status of a selected set of key metrics Scorecard Scorecards are the consolidated tabular and graphical display of sets of metrics related to particular business functions. Visualization Visualization reporting solutions show relationships between selected metrics and assist in performing impact analysis Interactive Data Analysis Online interactive analysis of data providing slice and dice and drill-down, drill-through capabilities. Known as OLAP (Online Analytical Processing). Scheduled Reports Individual reports related to specific data requirements. Operational Reports Reports available directly from operational systems used to support processes Monitoring Near real-time display of detailed data related to the delivery of a service. 32

33 Metric Presentation In theory various users in the organization have access to information that is timely and relevant only to them Concept Dashboard Scorecard Visualization Executive Managers Functional Managers Interactive Data Analysis Scheduled Reports Process Managers Operational Reports Monitoring Operations 33

34 Metric Presentation In practice many IT managers want to have access to very detailed reports. Also, scorecards or interactive data reports are only sporadically used Concept Dashboard Scorecard Executive Managers Visualization Functional Managers Interactive Data Analysis Scheduled Reports Process Managers Operational Reports Monitoring Operations 34

35 Dashboards A dashboard is a visual display of the most important information needed to achieve one or more objectives; consolidated and arranged on a single screen so the information can be monitored at a glance Intelligent Enterprise, 2004 Green IT dashboard Highly visual, data aggregated from various sources Provide a one glance view on the current state of the organization/process or activities. Typically used by CIOs, IT Management and business users. 35

36 Scorecards Scorecards focus on the consolidated presentation of metrics and presents an accurate view of the here and now compared with predefined goals Graphical representation of trends and alignment with defined targets is provided. They are supported by static and interactive reports, as well as diagrammatic representations of metric performance and linkages between metrics. Categorization of metrics within a scorecard is typically related to functional or process views. Service Desk Scorecard Metricus Scorecard from the Service Desk Module 36

37 Session 4 DEFINE REQUIREMENTS FOR IT PERFORMANCE MANAGEMENT 37

38 Method 1) GQM goal, question, metrics GQM is a technique to derive measures for project control starting from high-level goals, passing through the decomposition in several questions to answer. Author: Victor Basili in the early 80s (NASA Goddard Space Flight Center) Measurement object: software projects Application: GQM is one of the most well-known and used measurement approaches for establishing a measurement program Source: L.Buglione & A.Abran

39 Method 1) GQM goal, question, metrics Using the Goal Driven Method Business Objectives Increase Pace of Service Delivery Improve Service Availability & Stability Reduce outage minutes Reduce number of incidents Deliver Projects on Time / on budget Process Effectiveness Incident Problem Change Reduce MTTR Reduce MTBF Decrease days to root cause determination Decrease days to implement permanent fixes Decrease problems due to change Decrease issues during change implementation Process Governance Complete Correct & Valid Detailed Are mandatory fields filled out Are data entered according to process policies & standards Does the data reflect what actually happened Do text fields contain appropriate level of details Source: David Chiu of BMO Financial 39

40 Method 2a) Balanced Scorecard BSc is a multidimensional framework for translating (organizational) strategy into action at all levels of an enterprise, by linking objectives, initiatives and measures to an organisation s strategy Authors: Kaplan & Norton (HBS) in the early 90s, originating from a French method called Tableau du Bord (turn of 20th century) Main measurement object: whole organisation / a SBU Application: BSc is one of the most relevant management practices of last 75 years, according Harvard Business Review Journal Source: L.Buglione & A.Abran

41 IT strategy Business strategy

42 Method 2b) Balanced Scorecard and COBIT COBIT provides a bridge between business risks, control needs and technical by providing a comprehensive framework for delivering value while managing risk and control over data and information throughout the lifecycle. Authors: ISACA in the 90s, originating from being an auditors check list Main measurement object: The entire IT organization: IT resources need to be managed by a set of naturally grouped processes. COBIT provides a framework that achieves this objective. Source: COBIT 4.1, ISACA 42

43 Method 3) COBIT Aligning business goals with IT goals using COBIT, example for DS5 Source: COBIT 4.1, ISACA 43

44 Assignment We are going to use the following model from COBIT to define a process based KPI scorecard Business Goals Step 1 define business goals IT Goals Step 2 define IT goals IT management IT process health dashboards IT process goals Step 3 process goals IT departments drives Functional and process scorecards Step 5 Scorecard IT KPIs Step 4 KPIs 44

45 Use the following template for step 1, 2, 3 Business goal IT goals Processes 45

46 Case description At the Coffee Company, coffee is not merely a bean or a beverage. It is the furthest thing from a product or a commodity. Coffee involves relationships and responsibilities. It is a process involving high standards and tough decisions. More than anything, coffee has the connective force to enrich peoples lives. The Coffee Company is a gourmet coffee company, serving highquality coffee blends and a specialty selection of coffee by the cup. Espresso and related food and beverage products complement the offerings. Along with hot and cold liquid servings, the Coffee Company offers gourmet beans in retail bags through a retail channel as well as an online channel. All offerings are provided in an atmosphere that whispers of the warmth and convenience of your living room. From 1991 to date, the Coffee Company experienced and realized significant growth in size, revenue, and number of stores. From only one store in downtown Toronto, the company has grown to 12,000 retail stores and 1 online store. the current economic situation has made the Coffee Company shift its focus from expansion to consolidating business with its existing stores and focus on ensuring the uninterrupted availability of its coffee products and services.

47 The coffee production process Farming, drying, extracting Roasting Distribution Retail stores customers Coffee grows best in a warm, humid climate with a relatively stable temperature of about 27C all year round. The world's coffee plantations are therefore found in the so-called coffee-belt that straddles the equator between the tropics of Cancer and Capricorn. Un-roasted 'green' beans are shipped to Rotterdam in the Netherlands and Vancouver, Canada from locations as close as Hawaii and as far away as Indonesia. Varietals are then roasted or mixed with other beans to create blends, such as our popular VOC or Da Vinci blends. After roasting, coffee is packaged and sent to one of our five regional distribution centers in Vancouver, Rotterdam, Singapore, London and Houston. The distribution centers supply our retail stores manage the distribution from the regional internet sales. Online store Our outlets are company owned and franchise locations worldwide. The distribution centers supply our stores twice a week or more depending on demand. Each retailer is strongly connected to the community as well to ensure delivery of fresh products such as breads, bagels, etc.

48 Fragmented information from IT Retail store managers are getting tired of constantly having to wait more than 5 minutes on the telephone with the service desk. Before they get someone on the line..they have a shop to run! I am the incident manager, and I am proud of closing incidents within targets by my first level support team So as a result they don t call but bypass the service desk and go directly to the specialists Our CMDB is not very easy to use and as a result its not kept up to date always A change weekend is always exciting for the IT organization.. Will there by any store with problems with their online systems or does everything go well? In case of issues, the challenge is to find out what went wrong and how to revert. Testing plans are not maintained centrally and there is no review of implemented changes Change management is a rush job, and our change manager agrees to all requests from the business as they feel everything is important. It seems that we keep reopening incidents as our solutions don t seem to work, what s going on. We need a knowledge base..

49 Session 5 CREATE AN IT KPI BASED SCORECARD 49

50 Tips and Tricks for selecting KPIs When you are presented with a KPI wish list that looks too good to be true, it probably is. 1. Show me how well we are managing issues in IT 2a. Sure! Will this scorecard do? 2b. We don t know if it s possible but that s not our problem, we need the sale. 3. We don t have the processes or data in place CIO/Executive Management Consulting Middlemen IT Service Transition Team There is a gap between theory and practice, and there are not that many metric practitioners out there What looks and sounds nice is often not practical and possible. Don t be put in a situation with middlemen. Work direct with the users of KPIs to ensure that what is requested is actually feasible. 50

51 Tips and Tricks for selecting KPIs Use only use KPIs when you know the data is available and only use KPIs if the results can be tied to higher level operational, tactical or strategic objectives The horse and the cart: Only use metrics when you know the data is available. Confirm with all stakeholders the actual need and planned use of each of the reported metrics The requirement for metrics should not drive the requirement for data; process and strategy should. 51

52 Tips and Tricks for selecting KPIs Ensure that your effort-to-insight ratio remains positive Insight provided by the KPI Cost of data collection The cost for collecting data for a metric should weigh up against the insight that you are retrieving from the metric. The cost categories are data collection, Business Intelligence, and report development. Costs are predominantly qualitative whilst benefits are both qualitative and quantitative. 52

53 Tips and Tricks for selecting KPIs Garbage in = Garbage out: Poor data quality in IT services negatively affects the ability for informed decision making throughout the entire organization. IT metrics should be developed and presented with the same rigor as financial accounting metrics. Consider: Integrity of data, consistency with prior periods reporting, materiality (Value of data must exceed cost of reporting) Quality of IT Metrics High Low Low Data Quality High 53

54 Tips and Tricks for selecting KPIs Recognize that data integration will be the greatest challenge in developing KPIs, scorecards, and supporting reports. Focus on consolidating the data required to support metrics and scorecards. Don t assume one approach fits all for data integration. Build a 'data resource network, including developers, support and operations (DBAs), and application SMEs. Word Excel PowerPoint Face to Face Telephone Meetings Memos/Letters Help Desk Monitoring Systems Unstructured Structured 54

55 Tips and Tricks for selecting KPIs Less is more, and it is recommended to start with only a handful of KPIs rather than going with dozens of them KPIs are put in place to be used to manage and too many KPIs make managing to them difficult For every KPI processes need to be in place to collect data for every KPI and do something with the results of the KPI. Start small, keep it simple, and build upon achieved successes. Recognize that metrics will change over time based on the changing values required from the metrics and the growing maturity of the organization 55

56 Tips and Tricks for selecting KPIs Involve the right people when you are designing your IT Performance Management initiative Audience Executive Senior Management Rationale Implementing metrics and scorecards solutions is a strategic initiative that requires C-level sponsorship (CIO, CEO). This is generally recognized as the #1 Critical Success Factor to incorporate any Performance Management Initiative or implementing Business Intelligence software Identify Key People The people that really matter to a process must always be part of it. This is often a group so small that you can call the members out by name and count them on one hand. Key people need to be identified for requirements gathering, ownership of metrics and reports, assistance with developing metric/scorecard solutions, assistance in architecting and configuring the infrastructure required, and support for data collection and metrics presentation. Performance Management software solutions can provide guidance and templates for scorecards and reports for specific users in your organization Identify Metric Users Ask users what they need! Collect key information for users. Map metrics, scorecards, and reports to users. Develop an authorization and application security model. Keep it simple! Start on the premise of allowing users to see all information and selectively revoke rights where appropriate. 56

57 Metric Requirements - SMART - Now we have identified the Processes that have our key interest and that are important for us when we want to meet the objectives of the business. The next step is the identification of specific KPIs for the selected processes. First some guidelines around metrics: Metrics that are used need to be SMART: Specific Measurable Action oriented Realistic Time bound 57

58 Metric Requirements - SMART - The definition of an IT Performance Metric needs to be structured in alignment with defined attributes and valid measures This helps avoid the 100,000 feet syndrome of documenting wish lists of what sound like nice metrics to have, but in reality are not meaningful, not possible, too costly, and so on. Good Example Poor Example [% Calls Abandoned]. A fundamental Call Center/Service Desk metric, this divides the number of calls offered by the number of calls abandoned. Both measures are readily available and attributes such as time dimension, categorization fields, targets, user base, and so on are straightforward to obtain. (To be discussed more in the section IT Performance Metrics Attributes. Time used to resolve unavailable services. What services are included? What is the definition of unavailable? How is unavailable time equated with effort to resolve? Is it linked through incidents? If so, are the processes in place to record time? 58

59 Metric Requirements - SMART - IT Performance Metrics need to be quantitative, allowing users to actually measure progress and performance against the IT Performance Metric. The provision of an IT KPI should prompt the users to ask questions. Supporting data needs to be directly available and/or mechanisms available to obtain support data. Typically, quantitative metrics are supportable while qualitative metrics are hard to support. Good Example [SRs Open] is a valuable IT service metric but needs to be supported by details of open service requests. Poor Example % Projects with Predefined Benefit is often based on after the fact guestimates as to whether predefined benefits were available. For the metric to be of value, information on what predefined project benefits need to be stored in conjunction with baseline project details. 59

60 Metric Requirements - SMART - An IT Performance Metric should contain information that can be directly acted upon. This could involve questions being asked as to why a metric is a specific value, providing information on achieving defined levels of service, or automatically instigating remediation action within a particular IT service process. Good Example Poor Example [% Service Requests Resolved On Initial Contact] is often a metric contained within a Service Level Agreement (SLA). Failing to meet a defined contractual target, say 85% will require action to be taken in order to avoid possible penalties. Incidents Created While this is often interesting to look at, particularly across a period of time, and it is certainly easy to measure, it s hard to make an educated decision based solely on the results in isolation. This is actually a measure that needs to be categorized with IT performance reference data, such as service or classification, in order to be of value. 60

61 Metric Requirements - SMART - An IT Performance Metric should be realistic from a data perspective, that is, the data associated with underlying measures in the calculation of the metric needs to be available. An IT Performance Metric also has to be justifiable from the initial and ongoing costs. The effort in collecting the metric logically is lower than the value derived from decisions related to the metric. Good Example Poor Example [% Incidents Escalated] The volume of incidents and an indicator if an incident was escalated should be readily available from the operational data source associated with Incident Management. % of Problems Recorded and Tracked It is difficult to detect that a problem is not recorded or tracked unless it actually has been recorded and tracked! 61

62 Metric Requirements - SMART - An IT Performance Metric should be collected at regular time intervals, that is, have an associated time dimension. This may be hourly, weekly, monthly, quarterly, and so on. Metrics without a time dimension should be referred to as milestones. Good Example Milestones [% Incidents Caused by Changes] is a metric that should be analyzed over time to ensure a downward trend. Data should be available every day. % of Data Elements Contained within Enterprise Data Model. The creation of an Enterprise Data Model is a largely investigative activity aimed at understanding what data elements exist. Consequently, the denominator for this metric, the number of data elements, cannot be determined until an Enterprise Data Model actually exists. As a result, this should be an IT Performance Milestone Enterprise Data Model Complete.. 62

63 Assignment Based on identified processes, we are going to look at and select a number of IT KPIs and create a scorecard for this: Business Goals Step 1 define business goals IT Goals Step 2 define IT goals IT management IT process health dashboards IT process goals Step 3 process goals IT departments drives Functional and process scorecards Step 5 Designing a Scorecard IT KPIs Step 4 KPIs 63

64 Assignment Step 1: identify the most relevant KPIs for Change, Service desk / incident management Step 2: write down which KPIs are most relevant for your IT KPI based scorecard and why Process name: Metric Importance (1-10) Rationale (why) Note: keep the tips and tricks in mind! 64

65 Use the following template for step 4 Process name: Metric Importance (1-10) Rationale (why) 65

66 KPI Examples: Cost Control % IT Costs - Travel Reduce travel cost by providing viable alternatives, such as Web conferences or VOIP and e-learning education instead of classroom training. Energy Cost per IT Facility % Projects Related to Business % Inactive Application Accounts Energy cost per IT facility gives insight into the energy cost per facility. Monitoring this information periodically helps organizations in optimizing the energy consumption. Projects should not be created if they are not directly related to specific IT or business objectives. The combination of [% Projects Related to IT] and [% Projects Related to Business] should be 100%. Inactive accounts represent overhead on security systems, both from a performance and an administration perspective. Importantly, they make the task of auditing security more difficult due to unnecessary overhead. If effective security management processes are in place, the number of inactive accounts should be minimal. % IT Resources - Contractors IT departments will almost always have a requirement for contractors. The level of contractors needs to be balanced with the potential impact on overall IT resource moral as well as costs associated with projects. The % of IT costs that can be directly connected to business value drivers, for example, sales/services increase due to increased connectivity The average cost per user license for a particular application and types of applications A measure indicating how satisfied the business is with the allocation of costs from IT The % of IT staff that has participated in IT-related training over the last 12 months. This is a difficult metric to measure; however, the actual process of attempting to measure this metric will yield results through mapping IT costs to business value drivers. It is common for an IT organization to purchase far more licenses than required. For example, it is estimated that 80% of Business Intelligence software licenses are not used. Reducing the average cost per user license is a combination of ensuring that the correct number of users are determined and that costs are negotiated efficiently with IT vendors. This metric can be used to measure the effectiveness of the allocation of costs from IT to the business. While subjective in nature as the business will rarely be satisfied with IT costs, it can be used to gauge improvements in the distribution of costs. It is important within any organization that resources receive the necessary training to ensure that they are aware of current knowledge and to assist with career development. This metric provides details on what % of IT personnel have received training over the previous 12 months. The level of training is also significantly proportional to effectiveness and the costs associated with IT support.

67 KPI Examples: Ensure continuous service % Availability SLAs Met The % of services delivered for which the requirements in the Service Level Agreement for the availability of services is met % CIs Monitored The % of configuration items that are actively monitored by an infrastructure monitoring tool. Only configuration items that are candidates for monitoring i.e. infrastructure components such as servers, networks and databases are included in this metric. % Database Backup Failures The number of scheduled backups that failed in a selected time period % Database Restoration Failures % Databases Backup/Restore Verification % Databases Scheduled for Backup The % of database restoration attempts that failed during a selected time period. The % of databases that are scheduled for backup for which backup testing and verification of restoration has been performed The % of databases that required backup that have successfully been schedule for backup. % Server Backup Failures The % of total server backups that failed during a selected time period. Failure is defined by the non completion of a backup. This includes both scheduled backup's that did not start and backups that failed during the backup process. % Server Restoration Failures The % of server backup restoration attempts that failed during a selected time period % Servers Backup/Restore Verification % Servers Scheduled for Backup Average Training Days for IT Resources IT Continuity Plan Reviews The % of total servers that have had a full backup and restoration tested and validated. The % of services in an IT infrastructure that are scheduled for regular backups. The nature of backups will vary across servers based on the software and applications hosted on a server. This metric looks at all servers and provides an enterprise IT perspective. The average number of training days for IT resources over the previous 12 months Number of reviews per period of the IT continuity plan

68 KPI Examples: Project Management Projects Completed on Time The number of projects completed that were completed within the target end date. Projects Completed Within Budget The number of projects completed where the total actual costs were less than the total budgeted costs. Projects Completed on Time and Within Budget Projects Open with Milestones Missed Projects Scheduled Projects Open Projects Completed The number of projects that were completed by the target completion date and had total actual costs less than budgeted costs. The number of projects currently open where one or more milestones related to the project have not been met. The number of projects at a given point in time that are scheduled to be started in the future. Scheduled projects are those that have been created, subsequently approved, but not yet commenced. The number of projects that are currently open, that is, implementation or development has started but has not been completed. The number of projects marked as completed during a specified timeframe. The definition of completed is that the intended functionality to be provided by the project has been accepted by the users of the functionality. Projects Championed by Business The number of IT projects created were initiated from the business and that have funding and full support from the business. Projects Completed with Documentation and Testing Plans Projects Created A measure of the number of projects that have appropriate documentation associated with implementation tasks, such as testing and post-implementation tasks, such as support and training. The number of new projects created. % Projects Related to Business Projects should not be created if they are not directly related to specific IT or business objectives. The combination of [% Projects Related to IT] and [% Projects Related to Business] should be 100%. Projects - Predefined Benefit The number of projects created that have specified defined tangible benefits to the business or IT if successfully implemented.

69 KPI Examples: Manage Quality % IT Processes Meeting QA Objectives The % of IT processes that meet the QQ objectives that were defined % IT Processes Reviewed by QA The % of IT processes that have been reviewed by conducting a formal QA % IT Resources - Quality Training The % of IT resources who have received IT Quality related training within the previous 12 months. % Projects Meeting QA Objectives The % of IT projects that meet the QQ objectives that were defined % Projects Receiving QA Review The % of IT projects that have been reviewed by conducting a formal QA % Stakeholders Understanding IT Policy The % of stakeholders that understand the IT policy

70 KPI Examples: Enable Operation and use % CIs with Operations Documentation % Incidents Resulting from Inadequate Documentation The percentage of Configuration Items that have associated operations documentation The percentage of incidents that can be appointed to inadequacies in the documentation made available with new or changed applications % SRs Resulting from Inadequate Training % SRs Resulting from Lack of Documentation % Training Attendance New Applications Average Change Documentation Update Time Customer Satisfaction - Training/Documentation Incidents Resulting from Inadequate Documentation SRs Resulting from Inadequate Training The % of SRs created due to the requestor not having sufficient training in functionality related to an IT service e.g. configuration of new accounts in Outlook or accessing functionality within an ERP system or on the corporate portal. The % of service requests created as a result of lack of documentation related to an IT service. For example, a user calls the service desk because they cannot find information on the corporate portal on how to connect to the instant messaging system. The % of stakeholders that attended training for a new application after release Average time it takes for documentation to be updated after a change The average customer satisfaction score for IT Training/Documentation surveys sent in a selected period. This is representative of the overall satisfaction and confident that the business has in IT's ability to delivery training and IT related documentation required by the business to use IT services. A good range of values for measurement is 0 to 10 The volume of incidents that can be appointed to inadequacies in the documentation made available with new or changed applications The volume of Service Requests that can be appointed to inadequacies in the training that was presented to stakeholders after release of the new application or after a change made to the applicatoin

71 KPI Examples: Change Management Changes Created Changes Implemented Average Change implementation time The number of requests for change (RFC's) created. In IT Service Support, a change is the addition, modification, or removal of approved, supported or baseline hardware or software components. This can include network, application, environment, and system components, or other IT components, including documentation. All changes should relate to a configuration item. The % of critical priority changes implemented within an agreed target time. The average length of time required to implement requests for change. % Changes Implemented within Target - Critical The % of critical priority changes implemented within an agreed target time. % Changes Implemented within Target - High The % of high priority changes implemented within an agreed target time. % Changes Implemented within Target - Medium The % of medium priority changes implemented within an agreed target time. Average Cost per Change Average cost per change % changes failed The % of changes that failed during the implementation phase of Change Management. % emergency changes The % of changes implemented that classified as emergency changes. Emergency changes are those which require circumvention of routine change management processes due to the urgency of business requirements and changes to the IT infrastructure. A comprehensive Change Management process will include a process for handling Emergency Changes % high risk changes The % of changes created where the risk of incidents occurring within the IT infrastructure is high. A comprehensive Change Management process will include a process for handling High Risk Changes % changes rejected The % of change requests that are analyzed by IT and subsequently rejected. % changes due to CMBD issues The % of changes that were made as a result of incorrect information provided by the CMDB % changes post implementation feedback The % changes that received feedback after completion of the implementatoin % changes post implementation review The % of changes that were reviewed post implementatoin % changes process compliance The % of changes that correctly followed the defined change management process % changes audited The % of changes that were audited post implementation % changes causing incidents The % of changes implemented that resulted in one or more incidents occurring. % changes causing problems The % of changes implemented that resulted in one or more problems incurring % changes implemented without a back out plan The % of changes that were implemented without a defined back out plan

72 KPI Examples: Change Management % changes implemented without CI The % of changes that were implemented without proper CI testing testing % changes implemented without testing The % of changes implemented that did not go through the formal testing phase incorporated within the Change Management process. % changes requiring scheduled outages The% of changes that require a service outage to be scheduled in order for implementation of the change to occur. % changes specified inaccurately The % of changes that were not specified correctly % changes with incorrect data The % of changes that were closed but contained 1 or more incorrect data components e.g. wrong categorization fields entered, incorrect history, timestamps entered incorrectly, missing solution or closure description, etc. % Changes without sign off The % of changes that were incorporated without formal sign off

73 KPI Examples: Service Desk % IT Level 1 Support with Backup % SRs Resolved on Initial Contact IT Resources - Level 1 Support An important IT metric that is often used as a benchmark between IT organizations. The lack of backup personnel can directly effect the delivery and support of IT services, which in turn can effect the overall operations of an organization. This is a particularly important metric for the Service Desk because of the customer visibility involved. An important service desk metric as the ideal scenario is that service requests related to phone calls are resolved during the first call, minimizing the impact on the customer and the amount of process required by IT support. This metric is also a key driver of customer satisfaction because it is an 'interaction' metric - one whereby the service desk is exposed to the customer. The number of IT resources classified as Level 1 Support. Level 1 Support typically refers to agents at the Service Desk who are the first point of contact for service requests, incidents, and change requests from the business. They will try to resolve 'how-to' questions, routine simple changes, such as password resets, and ensure that more complex issues are assigned (dispatched) to the correct Level 2 and 3 resources. % Calls Abandoned Provides insight into how efficient the Service Desk is at answering IT support telephone calls from customers. A high abandonment rate may indicate inadequate staffing or inefficient processes in terms of the collection of information from the customer. Also, an important metric in terms of customer satisfaction because while small minorities of these abandoned calls are accidents or mistakes, the majority represent either dissatisfied customers (those who hung up) or customers who are about to become dissatisfied (those whose calls were terminated). Calls Answered Measures the number of calls that were answered by the Service Desk. This is related to <Calls - Offered> and <Calls - Abandoned> in that the difference between <Calls - Offered> and <Calls - Abandoned> are mostly the calls that were answered (the exception are the calls that were terminated during the initial pre-recordings). So, as the % of calls abandoned decreases, the % of calls answered should increase. Each call answered becomes a candidate for a service request, incident, or change request to be created. Call Answer Time A measure of how long it took to be answered from the time the customer was placed in a queue to the actual answer time. The time placed in a queue is normally immediately after the automated menu messages. Customer Satisfaction - Service Operations Avg Daily Incidents Handled per Service Desk Agent Average Cost per Call The results of survey responses for Service Delivery surveys sent. Relates to the calls that Service Desks can handle and is an important metric relevant to the planning of Service Desk resources. Average cost per call to the Service Desk. % IT Staff Turnover - Level 1 Support High Service Desk staff turnover will have a negative effect on key service request resolution metrics, such as [% SRs Resolved on Initial] and [% SRs Resolved within Target].

74 KPI Examples: Incident Management % Incidents Misrouted 'Misrouted incidents add to the overall resolution time and, as such, impact the customer and add cost to IT operations. A high number and % of misrouted service requests could indicate fundamental flaws and problems in the incident management process. % Incidents Reopened Primarily an operational metric that provides information on the effectiveness of the Service Desk and IT Operations resolution process. The ideal process is that an incident is resolved and then automatically closed within a set timeframe. If an issue recurs, a new incident is created. Incidents should not be reopened after initial resolution if the issue logged has been appropriately actioned and resolved. % Changes Causing Incidents If the change management process is functioning effectively and the CMDB is accurate, this metric should be very low because all changes should occur during scheduled outage times. Note: When analyzing this metric, care needs to be taken as if the incident root-cause analysis process is failing then this metric will be artificially low. % Incidents Dispatched to Level 2 Costs associated with level 2 are typically higher than for level 1 due to the additional expertise required. As IT operations becomes more efficient, this metric should fall, resulting in faster resolution times and reduced costs. % Incidents Resolved by Known Errors Known error information assists in the incident resolution by providing an IT operations resource with answers to questions previously asked. This assists in achieving target resolution rates and reducing overall costs. This is stated as an 'above target' metric, meaning that while the overall volume of incidents and incidents resolved by known errors should reduce, but the % of incidents solved by known errors should increase. % Incidents Dispatched to Level 3 Costs associated with level 3 are typically higher than for level 1 and 2 because of the additional expertise required. As IT operations become more efficient, this metric should fall, resulting in faster resolution times and reduced costs. % Incidents Resolved by Workaround While workaround solutions play an important part in maintaining the overall stability of IT services delivery, this metric should decrease as an IT environment matures. % Incidents Dispatched Provides an indication of IT operations efficiency in processing incidents. Ideally, over time, the volume dispatched decreases as a % of the total incidents created. % Incidents Void Void incidents should be monitored to ensure that upward deviations from longer term averages do not occur. Such deviations may represent data quality or data entry problems. They could also highlight issues with automated incident creation mechanisms. Average Incident Dispatch to Own Duration % Incidents Owned within Target - Critical This metric indicates how responsive IT support staff are at accepting ownership of incidents. Increasing trends can indicate issues with the process of service request ownership. Hourly or daily spikes can indicate staff rostering issues. Over time, the [Incidents - Dispatch to Own Duration] should decrease, but increase relative to the [Incidents - Create to Dispatch] duration. This will result in the [Incidents - Create to Own] duration decreasing. Prompt ownership of critical priority incidents indicates efficient service support processes and increases the probability that incidents will be resolved within the target time. Ownership of critical priority incidents within target has increased importance due to the customer visibility into processes within IT, that is, because of the potential business of critical priority service requests, the customer will have high expectations regarding ownership and subsequent resolution. The volume of incidents that are created Incidents created % incidents resolved by 1 st level The % of incidents that were resolved by the first level support team % incidents resolved within target critical The % of critical incidents resolved within the defined target time. Also referred to as 'Resolution Met' or TTR Met' (Time to Resolve Met). A typical target time for resolution of critical priority incidents is 2 hours.

75 KPI Examples: Incident Management % incidents resolved within target high The % of high priority incidents resolved within the defined target time. Also referred to as 'Resolution Met' or TTR Met' (Time to Resolve Met). A typical target time for resolution of critical priority incidents is 2 hours. % incidents resolved within target low The % of low priority incidents that were owned within the defined target time. Also referred to as 'Ownership Met, TTO Met' (Time to Own Met), or 'Accept Met'. A typical target time for ownership of low incidents is 4 hours % incidents caused by changes The % of incidents caused by the implementation of a change. Refer [Changes - Created] % incidents caused by CMDB issues The % of incidents that were caused by data errors within the configuration management database. % incidents linked to testing errors The % of incidents that can be linked to errors made during testing % incidents auto generated The % of incidents that were automatically generated by IT monitoring tools. % incidents resulting from inadequate documentation The % of incidents that are the outcome of inadequate documentation % incidents resolved by known errors The % of incidents that were resolved by access to information available regarding known errors. This information is typically stored in a 'Known Error' or Knowledge Management database. The source of the information should be Problem Management processes where known errors are identified.

76 KPI Examples: Problem Management % Problems with Incorrect Data Shows the % of problems that were identified as having one or more invalid data components. Poor data quality associated with operational IT systems adds significant cost to a business. Problems Open - Missed Target Used to monitor the volume of critical problems that are open and have missed target resolution. Increasing Resolution - Critical volumes highlight risk to the business and potential resource or IT infrastructure issues. % Incidents Linked to Problems In theory, all incidents are caused by problems. But, in the practical context of implementing IT service frameworks, not all incidents result in a problem record and resolution through the formal problem management process. This metric provides an indication of the % of incidents that are linked to the resolution of a problem. The goal of effective incident and problem management processes should be to increase the % over time while decreasing the overall volume. Average Problem Create to Close Duration Problems Created Can be used over time to track the efficiency of IT operations in resolving and closing problems. Increasing trends can indicate resource issues and instability of the IT infrastructure. However, care needs to be taken when analyzing this measure because some problems can be open for extended periods of time due to difficulties in capturing the data required for problem diagnosis Used to monitor the overall number of problems generated within an IT infrastructure. A high level of problems indicates inherit instability and potential problems with all IT service processes as problems are minimized via effective incident, configuration, and change management. % Proactive Problems Used to monitor how many problems are initiated by IT operations in a pro-active manner, that is, they start a new project because they believe that something could go wrong or they want to ensure that a given future problem does not happen. The presence of proactive problems indicates a mature, efficient problem management process. Problems Open Used to help identify resource issues within IT operations and assist with analyzing the overall stability of the IT infrastructure. % Problems Completed within Target - Critical % Problems Root Cause Identified Provides an indication of the effectiveness of Problem Management processes. Can also be used as an indicator of how many serious problem exists - those that require more time to resolve because of extensive analysis and diagnosis requirements. One of the fundamental goals of problem management is to identify the root cause of the related incidents and/or service requests. Efforts can be made to rectify the root causes, and information can then subsequently be fed into knowledge bases and known errors. % Problems RFC Created Provides information on how many problems require requests for change in order to be closed. The implementation of one of more changes associated with a problem can be a sign that tangible outcomes are occurring because of problem management processes or that the IT infrastructure requires significant change to correct issues. Given this, this metric needs to be analyzed with respect to the overall maturity of IT services delivery. Average Problems per Release Can be used to track the overall effectiveness with which IT implements releases. Problems related to releases can indicate serious flaws in the Release Management process.

77 Session 6 ANALYSIS OF A NUMBER OF SCORECARD AND DASHBOARD EXAMPLES 77

78 A number of example dashboards/scorecards for review View a number of examples from Metricus 78

79 Metricus IT Balanced Scorecard IT KPIs 25 practical, pragmatic, usable KPIs Trending Trending information to analyze KPIs in detail Settings Manage definition, set target, tolerance, data criteria Collaborate Provide comments with graphs, manage issues and documents 79

80 Metricus Service Desk IT KPIs 25 practical, pragmatic, usable KPIs for managing your service desk operations across the world Dashboards Call handling information, information on Service Requests and Incidents Drill down into the details Benchmark different service desk sites across the world and drill down into the details of available data 80

81 Metricus Service Request Management IT KPIs Every Module includes at least 25 KPIs with information on the practical use, adoption and benefits of KPIs. Add your own KPIs Simply add your own KPIs or modify KPIs to the Module and publish them to a dashboard Analyze to the greatest extent possible Dynamic charts allow you to analyze data and create charts that are most relevant to you 81

82 Metricus Change Management Six Sigma charts For selected KPIs you have the ability to create Six Sigma control charts or histograms. This functionality helps you reduce defects in your processes Comprehensive Dashboards A typical Metricus Dashboard Template that provides a comprehensive picture on your current IT performance and historical information. These two elements together allow you to take adequate decisions for the future 82

83 Metricus IT Balanced Scorecard For Visio experts Metricus allows you to create and upload Visio charts and link KPIs to Visio components. For every Module specific templates are available but you can also create your own Manage KPI categories For all areas in the IT balanced scorecard KPIs are available. We categorized them according to customer feedback, but you can create your own sub categories 83

84 IT Performance Management and best practices: ITIL IT Performance Management is strongly advocated in ITIL. Measuring, Managing, and Improving ITIL processes has its roots in the Deming Circle that is advocated in ITIL. The Plan, Do, Check, and Act (PDCA) Cycle requires to define a baseline and continuously measure progress against that baseline. ITIL addresses the use and benefits of KPIs and IT Performance Management in every process, as well as having an entire lifecycle phase fully dedicated to IT Performance Management, Continual Service Improvement (CSI). ITIL Service Lifecycle Crown Copyright 2007 Reproduced under license from OGC 84

85 IT Performance Management and best practices: ITIL The 7-Step Improvement Process is the most important element of Continual Service Improvement. This process follows the same logical steps as defined in the IT Performance Management lifecycle and can help you define what is important to you, measure progress against defined goals, and effectively utilize your resources to optimize your IT organization. The 7-Step Improvement Process Crown Copyright 2007 Reproduced under license from OGC 85

86 IT Performance Management and best practices: ITIL Strengths ITIL is strong in terms of processes surrounding service support. This has translated into broadly accepted sets of KPIs to measure Incident Management, Change Management, Problem Management, and so on. ITIL emphasizes effective Configuration Management. An accurate CMS is fundamental to the delivery of IT Performance Management. A process is not implemented until measured An improvement is not noted until measured against a baseline Justification for an ITIL project requires a measured baseline and an improvement target Weaknesses The ITIL publications are limited in the specification of KPIs to be used. There is not a lot of consistency in the definition of KPIs and information on KPI utilization between the various ITIL domains. The Continual Service Improvement is intended to be used throughout all phases of the service lifecycle, however in reality is only considered late in most implementation projects. Without metrics, an ITIL project will soon lose steam and eventually fail ITIL Service Lifecycle Crown Copyright 2007 Reproduced under license from OGC 86

87 IT Performance Management and best practices: ISO/IEC Capacity Management Service Continuity and Availability Management Release Processes Release Management Service Delivery Processes Service Level Management Service Reporting Information Security Management Budgeting and Accounting for IT Services Control Processes Configuration Management Change Management Relationship Processes Resolution Processes Business Relationship Incident Management Management Problem Management Supplier Management ISO/IEC provides an external quality mark for IT Service Management organizations ISO/IEC promotes the adoption of an integrated process approach to effectively deliver managed services to meet business and customer requirements. Adopting ISO/IEC formalizes the measurement component of IT processes for organizations because they have to demonstrate and attest control over IT processes. This includes requirements such as A process should be in place to identify, measure, report, and manage improvement activities, and Reports shall be produced to meet customer needs, including trend information, satisfaction analysis, and so on. 87

88 IT Performance Management and best practices: ISO/IEC Strengths Weaknesses ISO/IEC formalizes the need for well-defined and thought-through process adoption. There are many documentation requirements in ISO/IEC and, as a result, the chances of finding the data required for effective IT Performance Management are likely to be high. A specific process called service reporting dictates reporting requirements. There are no KPIs in ISO/IEC Even though ISO/IEC formalizes many of the operational IT processes, it does not provide guidance on good KPI selection and adoption. 88

89 Thank you insert photo Contact details: Arjan Woertman +31 (0)