EVALUATION FRAMEWORK FOR SERVICE CATALOG MATURITY IN INFORMATION TECHNOLOGY ORGANIZATIONS

Transcription

1 EVALUATION FRAMEWORK FOR SERVICE CATALOG MATURITY IN INFORMATION TECHNOLOGY ORGANIZATIONS Carlos Moreno Martínez Information Systems Department, Universidad Europea de Madrid Spain Abstract Today, Information Technology services are a fundamental part of the success of enterprises and allow the development of the Information Society. Every IT service provider should deliver services aligned with the changing needs of their customers to an agreed level of quality. The IT Service Catalog is a list of these services and represents the value proposition of the IT Service provider. Many Service providers have based their Catalog on industry and academic best practices such as ITIL V3. However these best practices do not include a specific maturity assessment model, do not advise about how to adapt the Service Catalog to organizational and business changes or how to make the best use of innovations in technology. This article proposes an evaluation framework (ECAT) to measure the maturity level of an IT Service Catalog, which should help service providers understand their value proposition and how it fits with the current situation and business needs of their customers. It also allows a periodic alignment review of the Service Catalog with the continual changes in the organization and business strategy. Keywords IT Service Catalog, IT maturity, Assessment framework, Business and IT alignment 1. INTRODUCTION The services supplied by every IT Service provider, either internal or external, are a fundamental part of the Business Operations. They support the evolution of the organization and, in some cases, even with the development and general growth of society and nations [1][2]. For this reason IT Service Management best practices (ITSM) highlights the need to manage the service lifecycle and the importance of having a Catalog that describes the relationship between the consumer and the supplier of these services. Multiple organizations have adopted these best practices, and have documented their provision of services in an IT Service Catalog. To ensure the organization gets the best out of these services and to add true value, these services must be integrated with the business processes they support and have the ability to adapt to changes in the organization [3]. Therefore the mutual relationship between the IT Service Provider and the consumers of these services must be described from the point of view of adaptation. Figure 1 represents this interdependence, and reflects how changes in the organization, in their products and services, or changes in the business model, affects or is affected by changes in IT, services or technology innovations 1. Business change Organization Products & Services Market Business model IT Change Provider type Products & Services Technology Innovation Figure 1: Relationship between business changes and IT changes The dynamics of this relationship requires a continuous revision of the IT Service Catalog, to ensure it adapts to these changes, and reflects the needs and vision of the consumer as well as of the supplier of the services. New 1 Examples of recent Technology innovations are Cloud services and Big Data 29

2 innovations in technology arise and it becomes necessary to have a standard framework to review the Service Catalog. This article proposes an evaluation framework to analyze the maturity level of an IT Service Catalog and the operational management required to maintain it up to date and valid. This model could be applied to any IT Service provider organization that has defined a Service Catalog based on IT Service Management best practice such as the Information Technology Infrastructure Library (ITIL ). The proposed evaluation framework: Allows a Service Provider to understand the current position of their Service Catalog and its alignment to the business s objectives. This is the primary goal of the proposed model. Serves as a reference point for the continuous improvement of the Service Catalog and IT Services Helps gather feedback after any IT Service management improvement plans. The proposed model is based on concepts of ITSM and extends into the best practices of Information technology management. 2. STANDARDS, MODELS AND RELATED FRAMEWORKS The proposed framework (ECAT) shares an approach used in various standards and best practices that are widely adopted and recognized in business, industry and academic environments: Control for Objects Information and related Technology (CobiT ) Information Technology Infrastructure Library (ITIL ) [2] CMMI for Services, Version 1.2 ISO/IEC 20000: Information Technology Service Management This section summarizes relevant aspects of those models in relationship to Service offerings Control Objects for Information and related Technology (CobiT ) The framework for the Control for Objects Information and related Technology and securing of IT Government (Cobit ) establishes best practices in five areas [4]: Strategic alignment is aimed at assuring the relationship between business and IT plans, defining, maintaining and validating the IT value proposition and aligning the business operations with IT operations. Value delivery is about executing the value proposition throughout the delivery cycle, ensuring that IT delivers the promised benefits against the strategy, concentrating on optimizing costs and proving the intrinsic value of IT. Resource management is about the optimal investment in, and the proper management of, critical IT resources: applications, information, infrastructure and people. Key issues relate to the optimization of knowledge and infrastructure. Risk management requires risk awareness by senior corporate officers, a clear understanding of the enterprise s appetite for risk, understanding of compliance requirements, transparency about the significant risks to the enterprise and embedding of risk management responsibilities into the organization. Performance measurement tracks and monitors strategy implementation, project completion, resource usage, process performance and service delivery, using, for example, balanced scorecards that translate strategy into action to achieve goals measurable beyond conventional accounting Information Technology Infrastructure Library (ITIL ) In its current version 3, ITIL ([5], [6], [7], [8], [9]), is the de facto standard for IT Service Management, covering processes, organization and technology. It is arranged around five Management areas, aligned with the lifecycle of a service: Service Strategy provides recommendations and defines concepts to translate IT Services into strategic business assets. Some of the concepts defined include business relationship management roles, value of a service from a 30

3 utility and warranty perspective, the IT portfolio and Service Catalog, as well as provider categorization and market segmentation in relation to service levels and business activities. Service Design defines recommendations for the design of new or enhanced services, along with policies and best practices for service implementation, to satisfy quality and customer expectations, as well as cost efficiency. Service Transition establishes guidelines and recommendations for the development of capacities that allow the release to production of services, assuring service design requirements, with risk control Service Operation defines best practices for the delivery and support of the services, assuring the value provided to the client by the service, utilizing the defined strategic targets. Continual Service Improvement proposes recommendations and techniques for optimizing Service Management through a cycle of continuous improvement that considers all stages of the service lifecycle CMMI for Services, Version 1.2 CMMI defines improvement practices for service delivery related processes. It is based on the Capability Maturity Model Integration for Software development as defined by the Software Engineering Institute. A fundamental part of the framework proposed by CMMI is the maturity and qualification levels that allow the evaluation of the maturity of an organization against a standard, in this case IT Service Management. The model associates process areas with categories and levels of maturity. Specific levels of maturity are reached if the area of processes is in place and fulfills all the objectives required for this level. The CMMI model proposes processes that are structured around a project phase and a delivery phase [10]: Project phase Capacity and Availability Management (CAM) Project Monitoring and Control (PMC) Project Planning (PP) Requirements Management (REQM) Service Continuity (SCON) Service Delivery phase Service Delivery (SD) Incident Resolution and Prevention (IRP) Service System Development (SSD) Service System Transition (SST) Strategic Service Management (STSM) In addition to these process areas there are other areas that are common to other models of the CMMI space, such as the definition of organizational processes and organizational training ISO/IEC 20000: Information Technology Service Management This international standard promotes the adoption of a process framework for IT Service management according to business needs and service consumer [11]. The standard defines the minimum requirements a service provider should provide to ensure acceptable quality for its customers. The provider organization can be certified under this standard, stating that all or parts of its services are managed through the levels proposed by the standard. The standard aims to improve the processes through a standard continuous improvement methodology that adopts the Plan, Conduct, Check, Act (PDCA) approach. The process framework proposed by the standard is organized into the following groups: Relationship processes, including Business relationship management and supplier management Service Delivery processes, including Capacity Management, Continuity Management, Security Management, Service Level Management, Budgeting and Accounting for IT Services and Service Reporting The Control Processes group includes Change Management and Configuration Management Release Processes includes Release Management Resolution processes includes Incident Management and Problem Management 31

4 3. THE IT SERVICE CATALOG MATURITY EVALUATION FRAMEWORK As discussed in the previous section on standards, models and related frameworks (hereinafter referred to as schemas for simplicity), all of them present the process as a key part of their approaches. The ECAT model also considers the IT management processes a fundamental aspect for assessing the quality of the Service Catalog, but it also includes other areas that should be measured before arriving at the overall maturity of an IT Service Catalog. These additional areas include communication, the consumer s knowledge of the Service Catalog, the relationship of the Catalog to the business processes and the ongoing Catalog management. The above schemas classify the IT management processes into categories which cover the entire service life cycle. The ECAT assessment model shares the common categories used in each schema but broadens the scope by adding additional management areas for the assessment. Table 1 shows the relationship of the schemas and the areas covered by ECAT. Scope of ECAT Schemas Cobit ITIL V3 CMMi- SVC ISO20000 Integration with the organization and the business mission Integration with business processes, products and services Communications strategy and publication of service offering Flexibility and adaptation to change Provider's operational efficiency Regulatory compliance Table 1: Scope of ECAT and relationship to other frameworks The areas listed in Table 1 are described below: a) Integration with the organization and the business mission. This area provides the analysis of the effectiveness of the IT Service Catalog as a relationship management tool between the business (service consumer organization) and the IT department (supplier), and to what extent the Catalog reflects the mission and business model. Another area to cover is the effectiveness of the Catalog as a tool to position the provider as the best alternative as technology supplier and strategic partner for the business, allowing among other things, to show the value of the service offerings. b) Integration with business processes, products and services. This area allows the analysis of the scope and accuracy of the Catalog with regards to business processes, and the relationship of IT Services to these processes. For example it should allow the measurement of which business processes are covered by the Service Catalog, or if the prioritization assigned to IT Services corresponds to the prioritization understood by the business. It should also measure the suitability and integration of the Catalog with the products and services of the organization. c) Communications strategy and publication of the service offering. This area allows the measurement of the focus and effectiveness of the Catalog, the distribution methods and the degree of awareness among the consumer organization of the Catalog contents. d) Flexibility and adaptation to change. This area assesses the ability of the Catalog to reflect changes in the business and IT organization. This covers the amount of integration between the Service Catalog and the strategic and operational demand processes as well as processes for technology and infrastructure change. Furthermore, this section should also measure the effectiveness of the processes in place for adapting the Catalog to business changes. 32

5 e) Provider's operational efficiency. This area is responsible for assessing the level of Catalog integration with the organization and the management frameworks of the IT supplier. It also assesses the Catalog s integration with the IT management systems including Finance. f) Regulatory compliance. This area assesses the compliance of the Catalog with corporate policies and regulations, as well as with external regulations The evaluation framework The ECAT model structure, components and relationships are depicted in Figure 2. Service management frameworks Business goals & Corporate strategy & IT Strategy Evaluation data Evaluation domains ECAT Evaluation areas Indicators Metrics and thresholds Plans Analysis & recomendations Maturity level Figure 2: IT Service Catalog Maturity evaluation framework (ECAT) The ECAT model (central area of Figure 2) assesses information at various levels to evaluate the Service Catalog maturity. The highest level, the domain level, provides assessment guidelines for each proposed IT Management area, in line with the principles and scope of the ECAT model. The first step therefore is to identify the management domains to be evaluated. This selection must be consistent with the current business goals and corporate strategy which are in place, and the strategy of the service provider organization (left block in Figure 2). Once decided which domains to evaluate, the model uses lower levels to add more detail for the evaluation. These levels are Evaluation areas, Indicators, and Metrics and Thresholds. Each level is based and dependent on the previous level. The Evaluation areas are used to structure and organize the domains. They facilitate the definition of improvement plans after the assessment has been carried out. The Indicators establish the specific objectives to be evaluated within each area. Indicators are the basis for further analysis and assessment. Finally the Metrics and thresholds define the data required for the maturity level measurement, which allows the quantitative calculation of the maturity. The input to the model is data needed to evaluate the concepts measured by the metrics. The output is the information to calculate the maturity levels of the IT Service Catalog. Plans, analysis and recommendations are beyond the scope of the ECAT model (right area of Figure 2) Evaluation areas and indicators The following table shows the summary of the evaluation areas and indicators proposed by ECAT for each domain (Table 2). This represents a comprehensive overview of ECAT s evaluation framework. 33

6 Domain Area Indicators 1. Integration with the organization and the business mission 2. Integration with business processes, products and services 3. Communications strategy and publication of the service offering Scope of the service offering and perception of the value of the Catalog Relationships of the Service Catalog with the Business mission and the current Business model Integration with the business organization and relationship with customers and users Level of integration with business processes Level of integration with business products and services Communication and publication of the Catalog Business units (BU) covered by the Catalog Level of understanding and opinion about the Catalog by the Board of Directors and Executive level management Existence of an agreed definition of IT Service and a glossary in the Catalog Existence and quality of a Strategic Service Map 2 Level of usage of the glossary by IT customers and users Existence of IT Services not based on Information Systems (such as professional services, etc.) Existence and quality of a Strategic Service Map Existence and quality of a Service Catalog Brochure Existence of an IT provider mission statement aligned with the corporate mission statement Existence of a detailed customer-centric service description (external vision of the Catalog) Inclusion of terms & conditions for IT Service users in the Service Catalog Inclusion of IT Service provider s responsibilities in the Service Catalog Identification of Services owners and Service managers for each service defined in the Catalog Business processes covered by the Service Catalog Does the Catalog reflect the IT Service importance agreed with service owners and in line with the importance of the related business processes? Scope of Products and Services included in the Catalog Direct relationship of IT Services with business customers Existence of an official Web publication of the Service Catalog (Intranet, Extranet, Internet) Existence of a corporate method to spread news related to the IT Service offering (newsletters, distribution lists, etc.) Existence and quality of the Service Catalog Brochure and leaflets. 4. Flexibility and adaptation to change Understanding of the Catalog by the service consumer community Relationship to business changes Catalog adaptation and agility to changes (internal, Level of involvement of the service consumer community in the review of the Service definitions and the Catalog Results of customer surveys i. Global value of the Catalog ii. Value of the accessibility to the Catalog contents iii. Quality of communication related to IT Service offering iv. Level of usage of the Catalog by users and external related entities Level of usage of the Catalog in the corporate Strategic and Operational demand management process Level of usage of the Catalog in Project Portfolio management and Technology Investment analysis Existence of corporate policies and procedures to manage new services and changed services and incorporate them into the Catalog Ratio of changes in Business organization vs. changes in Service Catalog Level of usage of the Catalog in Business technology change management process 2 The Strategic Service Map presents the relationship between the Business organization, the Business processes and the IT Services which supports and enables these processes. It represents the global overview of the Service Offering provided by the IT Organization. 34

7 Domain Area Indicators 5. Provider's operational efficiency external) Integration with the organization and with management frameworks of the service provider Integration with Quality management systems and policies Relationship with the technology asset lifecycle Relationship with IT providers Usage and acceptance of the Catalog by business relationship management roles within IT (Customer Managers, Service Managers, etc.) Efficiency of the Service Catalog Management process Existence of Service Catalog Management roles and responsibilities Level of Catalog quality measurement Existence of a service description including requirements and information intended for the service provider (internal service description) Relationship of the Catalog contents with Operational Management Systems (Event management, Incident and Problem management, inventory management systems, etc.) Existence of relationship with corporate Quality Management models and systems (Service Level Management process, EFQM, etc.) Usage of the Catalog as approved source for audit process Relationship with IT performance measurement Existence of the relationship and usage of the Catalog with service development frameworks and methodologies Relationship with IT Asset management process Level of integration of the Catalog with IT provider Management systems 6. Regulatory compliance Integration with external regulations Integration with corporate policies and regulations Table 2: Evaluation areas and indicators References in the Catalog of corporate regulations and policies Relationship with policies and regulations of the IT Service provider Usage of the Catalog as source of information in relation to corporate policies 4. PROPOSED EVALUATION METHODOLOGY The proposed evaluation Framework has been designed to be used in conjunction with an evaluation methodology based on a continual management cycle and periodic review of the maturity level. In general terms the evaluation methodology should include the following recommended activities: 4.1. Define the goals for the evaluation and selection of the evaluation domains The ECAT model has been designed to use all the proposed evaluation domains, although it is possible to limit the maturity assessment to specific management domains. The selected domains are aligned with the corporate and IT strategic priorities. For example, if an IT organization wants to focus on their relationship with the business, domains 1 and 2 could be assessed. Domain 4 should be included if a software quality improvement program is in place. If the focus is placed on IT management best practices for Service Operation, domain 5 could be used to evaluate the adequacy of the Catalog for these initiatives. The decision about which domain to assess should be taken by the CIO and IT Executives with the support of the Service Catalog Manager. Also the CIO should appoint an Evaluation Manager who should be in charge of leading and coordinating the complete evaluation of the maturity of the Catalog and to present the final results to the stakeholders Set up the evaluation plan and the evaluation resources This step is about defining the needed scheduling for the assessment, as well as scheduling the required tasks of the evaluation team. Other tasks are to identify the people involved from the IT provider and the business. This step should be carried out by the Evaluation Manager Obtain the data for metric calculation 35

8 Using the indicators from ECAT as guidelines, the data should be obtained using standard evaluation techniques, for example, interviews with stakeholders and a review of existing documentation Service Catalog maturity calculation The resulting evaluation data should be formulated according to a qualitative method, for example from level 1 (minimum) to 5 (maximum). If the metric is qualitative then a conversion algorithm should be used to convert the value to a numeric format. The following example contains a simple table to convert qualitative metrics into numerical format. Qualitative metric value Measured aspect Nonexistent and not planned to build in the Service Catalog Example: The following aspect is not in place and not planned to be included in the Service Catalog structure: Existence of a service description including requirements and information intended for the service provider (internal service description) Measured aspect Nonexistent but planned Measured aspect exists in the current Service Catalog Evaluation level 1 The final results obtained from each metric should be weighted with the rest of the evaluation area to get the final evaluation for the corresponding domain. 3 5 Once the maturity level has been determined for each assessed domain, improvement plans should be documented to improve the quality of the Service Catalog. The evaluation domains with lower levels of maturity could be used to prioritize the required improvement tasks. The improvement plan should have an owner responsible for the coordination and monitoring of the plan and it should involve all affected groups within the service provider and service consumer organizations. 5. CONCLUSIONS AND NEXT STEPS The proposed maturity evaluation framework for Service Catalogs can be a valuable tool for CIO, IT Executives and Service Managers in assessing the Catalog maturity. It could also be used as a framework where no Service Catalog exists and the design and development of the service offering is to be carried out. In addition, the ECAT model is proposed as a reference and guide related to general IT Service Management topics, including people, processes and technology. The proposed model could be evolved in relation to the metrics and their thresholds for each evaluation domain. Each indicator could be further developed defining specific metrics. Another interesting model evolution area is the definition of specific industry measurement frameworks to allow a more precise vision of the efficiency and correctness of the Service Catalog in relationship to industry and business sectors Definition and execution of improvement plans base on the evaluation result 6. REFERENCES [1] World Economic Forum, The Global Information Technology Report , ISBN-13: [2] R. Kaplan and D. Norton, "Alignment: Using the Balanced Scorecard to Create Corporate Synergies", Harvard Business School Press, 2006 [3] C. Moreno, "The IT Service Catalog as foundation for IT Governance", IV International Congress: "Evolving from IT Service Management to IT Governance", itsmf Spain, 2009 [4] IT Governance Institute, Cobit 4.1, ISBN [5] OGC, ITIL V3 Service Strategy, OGC, 2007, ISBN [6] OGC, ITIL V3 Service Design, OGC, 2007, ISBN [7] OGC, ITIL V3 Service Transition, OGC, 2007, ISBN [8] OGC, ITIL V3 Service Operation, OGC, 2007, ISBN

9 [9] OGC, ITIL V3 Continual Service Improvement, OGC, 2007, ISBN [10] CMMI for Services, Version 1.2, February 2009, Software Engineering Institute, Carnegie Mellon University [11] International Organization for Standardization, ISO/IEC Information technology Service management Part 1: Specification [12] Center for Information system Research MIT Sloan, Working Paper No , MIT Sloan School Don t Just Lead, Govern - Implementing Effective IT Governance [13] Sallé, M., Rosenthal, S., Formulating and Implementing an HP IT program strategy using CobiT and HP ITSM, IEEE Hawaii International Conference on System Sciences 2005 [14] Kris B., Curtis, D., The Fundamental Starter Elements for IT Service Portfolio and IT Service Catalog, Gartner Research ID Number: G , July 2010 [15] Johnson, Alice M., Lederer, Albert L., CEO/CIO mutual understanding, strategic alignment, and the contribution of IS to the organization, Journal of Information & Management 47 (2010) [16] McNaughton B., Ray O., Lewis L., Designing an evaluation framework for IT service management, Journal of Information & Management 47 (2010) [17] Sánchez, José I., Pueden las tecnologías de la información mejorar la productividad?, Universia Business Review, Primer trimestre 2004 [18] Abeck S., Link S., Mayerl C., Mehl O., Vogel T., A system supported method to design IT services, Cooperation & Management, Universität Karlsruhe (TH) [19] González Z., Calvo M., Gil A., Fundamentos científicos para el análisis de las complementariedades entre el binomio SI/TI y la calidad de servicio, DOCUMENTO DE TRABAJO , Facultad de Ciencias Económicas y Empresariales Universidad de La Laguna [20] Niessink F., Vliet H., Towards Mature IT Services, Software Process Improvement and Practice, volume 4, issue 2, June 1998, pp