Identifying Weaknesses for Chilean e-government Implementation in Public Agencies with Maturity Model

Transcription

1 Identifying Weaknesses for Chilean e-government Implementation in Public Agencies with Maturity Model Mauricio Solar, Hernán Astudillo, Gonzalo Valdés, Marcelo Iribarren and Gastón Concha Universidad Técnica Federico Santa María (UTFSM), Chile {msolar, hernan, gvaldes, miribarren, Abstract. Evaluating readiness of individual public agencies to execute specific e-government programs and directives is a key ingredient for wider e- Government deployment and success. This article describes how the egov-mm model was used to over 30 many Chilean public agencies and to identify specific areas in which each of them should focus improvement efforts. egov- MM is a capability maturity model, patterned on similar models from the IT and organizational domains; it recognizes 4 leverage domains, 17 key domain areas (KDA) and 54 critical variables, identifies capability levels for each critical variable and KDA and proposes a synthetic maturity level for institutions. The 9-agencies pilot study and the 30-agencies massive study indicate that, in Chile today, the best developed areas are IT-related and mirror exactly those that are pre-requisites for the goals of the 5-years Digital Strategy, and the least evaluated are related to internal processes and human capital management. Keywords: e-government, roadmap, capability, maturity model. 1 Introduction The egov-mm model (e-government Maturity Model) [1] resulted from a project to build and apply a model to measure public institutions readiness to manage and implement e-government. The project was executed by the Informatics Department of Universidad Técnica Federico Santa Maria (Chile) for the Executive Secretariat of Digital Strategy of the Ministry of Economy of the national Chilean government. egov-mm allows evaluating a public institution against the best international practices in the area of egov, including formulation of organizational strategies, management of Information and Communication Technologies (ICT), operative management, and capabilities of the organization and its human resources. It also proposes concrete roadmaps for capability improvement, i.e. directives about where the organization financial and human resources should be canalize to improve its capability to carry out egov initiatives.

2 2 Mauricio Solar et al. The egov-mm model [1] considered several information sources in its initial formulation: Models that provide the standard structure of CMM [2, 3]. Models that show scopes and levels for maturity and capability [4] directly related to egov (e.g., interoperability and financial analysis of ICT investments in the public sector) [5]. Models of specific intention that provide scopes and levels of maturity and capacity on subjects related to egov (e.g., enterprise IT governability and architectures) [6, 7]. Best egov practices (for implementation, measurement and evaluation) identified from the growing presence of national state-of-the-art experiences and international experiences (namely, from UK, the USA, Australia, Canada, Sweden, Denmark, Korea, Ireland, New Zealand and Brazil) [8]. The model was piloted with seven (initially 9) public agencies, and a tuned version was generated that incorporates the participants feedback and an egov implementation roadmap for each evaluated public agency. An associated selfassessment Web tool was also built and similarly validated. The refined model was then used to evaluate thirty public agencies; this sample was chosen with systematic random sampling modified to include public agencies of all maturity levels, sizes, and coverage. Section 2 describes the egov-mm and its main characteristics, objectives and benefits; Section 3 explains how an organization maturity is evaluated; Section 4 presents the pilot results; Section 5 presents the validation via massive application to 30 public agencies; and Section 6 summarizes and concludes. 2 General Structure of the egov-mm The egov-mm is structured around three main elements: Information Criteria, ICT Resources, and Leverage Domains. There are four leverage domains, which group 17 Key Domain Areas (KDA) and 54 Critical Variables (see Fig 1). 2.1 Capability and Maturity Levels Capability Level (CL): The capability is a measurement of the state of each KDA that contributes to support the organization development. The capability of a KDA is determined using the CL of each of its Critical Variables, i.e. what is really evaluated it is the capability of these variables to satisfy certain requirements. The capabilities of the critical variables are weighed according to their importance to give a final KDA CL. Maturity Levels (ML): The CL of a KDA and its critical variables determine the organizational ML. The ML is a property of the organization as a whole; each ML corresponds to a predetermined configuration of KDAs in predefined CLs. The model allows, once the current ML is assessed, to identify the states required to advance to a higher level and propose a roadmap to improve the organization.

3 Identifying Weaknesses for Chilean e-government Implementation in Public Agencies 3 Figure 1. Hierarchical structure of domains, areas and variables. 2.2 Model Contributions and Benefits The model is a diagnosis element for public agencies, with several purposes: Identify the fundamental areas in the development of an egov strategy. Establish the technological CL in scopes relevant to the delivery of services to citizens. Determine the ML of the organization s technology management. Articulate a development strategy of its capabilities for continuous improvement of its maturity degree (roadmap).

4 4 Mauricio Solar et al. The main benefits already identified in the egov-mm model application are: It is a reference framework to identify the areas that support an egov strategy, based on internationally recognized best practices for IT management (as summarized in a state-of-the-art [8] executed as part of the same project). Additionally, the model incorporates locally arisen best practices, and combines ideas and knowledge of egov development specialists. It can be applied to improve both interoperability practices and technology management capabilities. Having an ordered set of good practices and an evaluation methodology yield a support tool for strategic and tactical management. The egov-mm incorporates the business requirements and the information to support, which themselves demand associated IT resources that, when applied to the Leverage Domains, should give necessary criteria to fulfill the business goals. It has a methodology to determine an improvement roadmap; besides the 4 leverage domains, 17 KDA and 54 critical variables, the model incorporates organizational capabilities and maturity. Each KDA critical variable has minimum required values for higher maturity levels, marking improvement roadmaps for the organization. It is operationalized with a Web tool for self-assessment of critical variables, regarding the capabilities of each institution to carry out egov initiatives. The tool and the information it requires are available for public agencies, and can be evaluated periodically to monitor improvement of capabilities and maturity levels. It allows integrating the key issues for efficient technology management at each institution, combining the organizational, process, human capital, and technology domains, to get a global view of egov development for all government agencies. It provides data to formulate technological projects and to base budgetary requests on an institutional perspective. 3 Model of Capabilities and Maturity One goal of the egov-mm is to generate roadmaps for progressive evolution of capacities and maturity. To this end, first we describe the generic model used to define the capacity levels for each model variable; then we describe the relation between variables capabilities and their respective KDAs; and finally we describe how the organization maturity is determined from its KDA capabilities. 3.1 Generic Capabilities Model For each KDA there is a measurement scale from 1 to 5, associated with a generic qualitative capabilities model that ranges from initial capabilities to integrated capabilities ; the values are:

5 Identifying Weaknesses for Chilean e-government Implementation in Public Agencies 5 Level 1: Initial capacities The KDA is approached ad-hoc and reactively, and individually case by case. There is evidence that KDA has been recognized and needs to be approached. Level 2: Developing capacities (repeatable but intuitive) A regular intuitive pattern is followed itself to approach the KDA; different people use similar procedures for the same task. There is no formal training or procedures spreading, and the responsibility to follow them falls to each individual. Level 3: Defined capacities Procedures related to the KDA are defined, documented and communicated. There is informal training to support particular initiatives related to the KDA. The procedures are not sophisticated, but are the formalization of existing practices. Level 4: Managed capacities It is possible to monitor and measure fulfillment of procedures, and to take action when the KDA apparently is not working properly. Established standards and norms are applied through the organization. Tools are mainly automated. Level 5: Integrated capacities (Optimized) Procedures have become best practices, and continuous improvement is applied. ICT is used in an integrated manner to optimize the KDA. Use of standard or world-wide tools helps to optimize the KDA. The capacity levels of each KDA are built from the levels of their variables. For each level, several aspects are considered (incrementally in each level): awareness; human capital training; communication; procedures and practices; compliance of standards and norms; tools and automation; and involvement and responsibility. 3.2 Relationship between Capability Variables and KDAs The capability level (CL) of a KDA is generally the average of the CLs of its critical variables. To accommodate egov strategies or country development levels with different variables relevance, weights are used for each variable group. Thus, the CL of a KDA is the weighted average of the CLs of its variables Vi (see Eq. 1). CL KDA = Average[CAP(V 1 )*P 1, CAP(V 2 ) *P 2,,CAP(V n ) *P n ] (Eq. 1) The weights P i used in the first model applications (pilot and initial massive application) are shown in Table Organizational Maturity Model Maturity is a property of the organizational unit as a whole, and the maturity level (ML) is obtained from the KDA capacity levels that the unit has. Thus, each ML:

6 6 Mauricio Solar et al. Frames a set of KDAs for a given CL. Establishes equivalence among egov implementation maturity of units. There are several options to determine an organization maturity, namely: 1. Minimum CL among all KDAs 2. Average CL of all KDAs 3. Predetermined KDA configuration, using a set of values for all KDAs in model. 4. Configuration of high-priority KDAs, using a set of minimum values for all KDAs in the model. Table 1. Variable weights of the egov Strategy KDA KDA Variable Weight Vision, Strategies Strategy Alignment 15 and Policies High Level Commitment 30 Communication to involved 25 Resource allocation to egov Enterprise Architecture Strategy IT Management and Organization Definition of Implementation Strategy 30 Alignment with Reference Models 20 Strategy of Services Reuse 20 Business Architecture IT Organization Planning 15 IT Infrastructure Planning 25 Organizational Structure Definition 30 IT Processes Map The last criterion (Configuration of high-priority KDAs) was adopted in egov- MM. The organization ML is determined (Eq. 2) by a set of values for all KDAs in the model (see Table 2). ML1 = Conf1(CAP(KDA1),,CAP(KDAi)) (Eq. 2) ML5 = Conf5(CAP(KDA1),,CAP(KDAk)) This mechanism was selected for egov-mm for its flexibility to allow graduating progress according to specific egov strategies, since it only requires to fix a minimum set of KDAs that are important for a given ML; development criteria and rates for other KDAs are left to the organization. The actual criteria to use can be extracted from domain specialists or agencies leaderships; e.g. from phrases such as: Maintaining enterprise architecture is an advanced issue, which allows aligning business objectives and computer networks and thus should not be requiring nor evaluating for lower MLs It is very important to start by aligning the IT, egov and of business strategies this should be required even for lower MLs.

7 Identifying Weaknesses for Chilean e-government Implementation in Public Agencies 7 4 egov-mm Pilot Study This section presents the results of applying the egov-mm and its self-evaluation tool to a small set of public agencies as a validation mechanism; it describes the main sample characteristics, the pilot methodology used, the results of capacity measurement for the sample, and the relevance of the variables. egov Strategy Table 2. Example of organizational ML with a set of high-priority KDAs Maturity Level (ML) KDA Variable ML1 ML2 ML3 ML4 ML5 IT Governance Processes Management Organization and People Capabilities Vision, Strategies and Policies Enterprise Architecture Strategy 2 3 IT Management and Organization IT Architecture 2 3 Portfolio and Risk Management Change Management 3 3 Asset Utilization Business Processes Management Performance Management Services to Citizens and Business IT Service Delivery Norms Compliance Quality Assurance Interoperability Practices Infrastructure and e-gov Tools Knowledge 3 3 Human Capital Sample and Pilot Study Methodology Nine public agencies were initially selected for the pilot study, using three criteria: 1. It should include institutions at maturity level 3, as estimated by their capabilities in two easily-verifiable areas: basic infrastructure (connectivity, processing infrastructure, security), and online services implementation. 2. It should have at least 3 public agencies per level. 3. A maturity level should be chosen that allowed to validate that the model can predicted the real situation, even if it included additional areas and indicators. Of the nine selected agencies, one declined to participate, and another one accepted but did not send any representatives to the assessment workshops. To maximize participation and yet simplify execution, the assessment was supported with upper management involvement, conducted as a guided selfassessment, and prepared with qualification workshops segmented by knowledge area, to limit KDA subgroups for each audience. Three profiles were defined for the institutions representatives: (1) Operational Management (Director or Assistant director); (2) IT Management (Assistant director or IT Unit Head); and (3) Human Resources Management (HR Head).

8 8 Mauricio Solar et al. 4.2 Pilot Study Results At the workshops guided self-assessment, participants responded questionnaires for their specific knowledge areas, and gave the results shown in Tables 3 and 4. Capabilities: Table 3 shows the KDA capability values for each participant institution. Six of the participants evaluated all 3 profiles (but one gave incomplete answers). Two issues deserve comment: The highest average capability values (shaded soft) are for IT Architecture and Interoperability Practices. Since this contradicts our (admittedly impressionistic) field observations, we surmise that the capability concept was not fully understood and/or the questions did not cover well the capability level. The lower capability levels (shaded dark) are Performance Management, Services to Citizens and Business, Norms Compliance and Human Capital. These values seem realistic, again based on our field observations. Table 3. Capabilities of Key Areas for the pilot public agencies Areas MIN TGR INP SEN SERC SAG ISP Average VU CE OTEC egov Strategy Vision, Strategies and Policies Enterprise Architecture Strategy IT Management and Organization IT Governance IT Architecture Portfolio and Risk Management Change Management Asset Utilization Processes Management Business Processes Management Performance Management Services to Citizens and Business IT Service Delivery Norms Compliance Quality Assurance Organization and People Capabilities Interoperability Practices Infrastructure and e-gov Tools Knowledge Human Capital Average per Organization Relevance of Variables: Besides the guided self-assessment, a survey was conducted to determine the relevance level of the variables included in the model. The results (see Table 4) show that, on average, 80% of the variables were considered as of high or very high relevance (levels 3 and 4). If a weighted average is used that considers

9 Identifying Weaknesses for Chilean e-government Implementation in Public Agencies 9 the percentage of variable presents in each of the 3 scopes, this value grows to 89%. Most remarkable is the case of the IT Management scope, where only IT Unit Heads participated and who gave a relevance perception of 96%. Table 4. Relevance of variables per area according to pilot-participating experts Frequency Relevance Level Operational Management IT Management Human Resources Total Percentage 4% 14% 29% 53% % answer 3 & 4 82% Total Percentage 1% 3% 22% 74% % answer 3 & 4 96% Total Percentage 7% 12% 15% 66% % answer 3 & 4 80% Weighted Total 89% 5 Larger Study Results The egov-mm was refined with the pilot study results, and was validated with a larger set of 30 public agencies. Table 5 ranks the evaluated institutions. Fully half of them are at maturity level 2, and only a single leading agency is at level Ranking of Public Agencies An organization maturity level is determined by the KDA with smaller capacity; e.g. an institution with 16 of 17 KDAs with a good capacity level but one KDA with level 1, has a maturity level of 1. To allow readers to identify these situations, Table 5 has an extra column with the percentage maturity, the sum of all its KDA capabilities divided by its total capability. Table 5 also shows that the groups with maturity 1 and 2 are institutions that with little additional work can increase their organizational maturity; e.g. INP and Customs are about to reach level 2, and Housing Sub-ministry is close to level Analysis of Key Areas Table 6 indicates the average capacity value of each KDA in the sample of public agencies that participated in the larger study. The mean level of organizational maturity of the evaluated public agencies was 2 (of a maximum of 5). A key finding was that the average value in 16 of the 17 key domain areas is below level 3.

10 10 Mauricio Solar et al. Table 5. Maturity ranking for the sample M a t u r i t y Institution Percentage Absolute SII (Tax service) 74% 3 Housing 58% 2 TGR 55% 2 AGCI 54% 2 SAG (Agriculture) 53% 2 SERNAC 53% 2 Navy Ministry 53% 2 FONASA 52% 2 SSMSO 52% 2 CSE 51% 2 ISP 49% 2 SML 49% 2 JUNAEB 48% 2 SUBTEL 47% 2 SEC 45% 2 SUSESO 42% 2 SENCE 54% 1 INP 51% 1 Customs 49% 1 DGAC 48% 1 SERCOTEC 46% 1 ONEMI 44% 1 CNE 41% 1 Sernapesca (fishing) 40% 1 Airports 39% 1 Labor Ministry 39% 1 Gendarmería 38% 1 Subs. RR.EE. 38% 1 Planning Ministry 36% 1 SUBDERE 33% 1 Average 48% The best developed areas are: Interoperability Practices and IT Architecture, which are explained by the emphasis of the Chilean Digital Strategy in the last few years, whose goals are to advance towards integrated egov and to generate norms for its implementation. Services to Citizens and Business, which are explained because egov strategy has privileged putting public agencies and transactions online, and hence the development of technological channels to serve citizens services requests. The least developed areas are: Business Process Management, a subject that has been insufficient and unequally developed, depending on each agency modernization strategy. Only in the last few

11 Identifying Weaknesses for Chilean e-government Implementation in Public Agencies 11 years has been emphasis on improvement of the management processes themselves, via legal initiatives about IT incorporation in administrative processes. Knowledge, which is an important challenge egov development field. Currently there are only embryonic initiatives to use systems that empower collaborative work and knowledge transfer; major advances are required. Human Capital, with a need to incorporate, train and formally develop both people and competences, required for sustainable systemic, processes and technological management of the complexity level required for integrated egov. Table 6. Average capability of Key Areas KDA Average capability Interoperability Practices 3.10 Services to Citizens and Business 2.70 IT Architecture 2.67 IT Service Delivery 2.63 Infrastructure and egov Tools 2.60 IT Management and Organization 2.47 Performance Management 2.43 Asset Utilization 2.37 Norms Compliance 2.33 Quality Assurance 2.30 Change Management 2.30 Vision, Strategies and Policies 2.23 Enterprise Architecture Strategy 2.20 Portfolio and Risk Management 2.20 Knowledge 2.10 Business Process Management 2.07 Human Capital Conclusions The egov-mm was initially deployed with a pilot study, and its results were used to refine the model and associated assessment Web tool. An associated survey found that 80% of the used variables were considered highly or very highly relevant, a level that rose to 96% for IT Management. The refined model was applied to 30 public agencies. It found that all but one of the 30 agencies are at level 1 or 2, and that the most developed KDAs are Interoperability Practices, Services to Citizens and Business, and IT Architecture ; the less developed are Knowledge, Business Process Management, and Human Capital. An explanation for these findings is that, for the last 5 years, the Chilean Digital Strategy has emphasized Web presence and online services, both issues that demand improvement in IT infrastructure but not necessarily in underlying processes. Clearly, efforts to introduce IT in business processes have not been coupled with efforts to formalize internal processes and to develop the required human capital. This result is

12 12 Mauricio Solar et al. perhaps country-specific, but nonetheless illuminating on the model measurement power and of the real effects of government policies and priorities. Another finding was that several agencies (e.g. INP, Customs, and Housing) are quite close to reach a higher maturity level, and are being held down only by isolated and (now) well identified areas, which can (now) be targeted for improvement. Thus, egov-mm is not only a diagnostic tool, but also a generator of improvement roadmaps. Government has a methodological and technological tool to measure status and improvement of egov implementation by specific public agencies. References 1. Iribarren, M., Concha, G., Valdes, G., Solar, M., Villarroel, M., Gutiérrez, P., and Vásquez, A. Capability Maturity Framework for e-government: A Multi-dimensional Model and Assessing Tool. in M.A. Wimmer, H.J Scholl and E. Ferro (Eds.): EGOV 2008, LNCS vol. 5184, , Springer-Verlag Berlin Heidelberg, (2008). 2. Software Engineering Institute: CMMI for Development v1.2. USA (2006). 3. Andersen, K.V., Henriksen, Z.H.: E-government Maturity Models: Extension of the Layne and Lee Model. Government Information Quarterly 23, , (2006). 4. Cresswell, A., Pardo, T., Canestarro, D.: Digital Capability Assessment for e- Government: A Multidimensional Approach. LNCS vol. 4084, pp , (2006). 5. Layne, K., Lee J.: Developing Fully Functional e-government: A Four Stage Model. Government Information Quarterly 18, , (2001). 6. Wimmer, M.A., Tambouris, E.: Online One-Stop Government: A Working Framework and Requirements. Proc. of the 17 th IFIP World Computer Congress, Kluwer Academic Publishers, Boston (2002). 7. Esteves, J., Joseph, R.: A Comprehensive Framework for the Assessment of e- Government Projects. Government Information Quarterly 25, , (2008). 8. Valdes, G., Iribarren, M., Concha, G., Solar, M., Visconti, M., Astudillo, H., Villarroel, M., Gutiérrez, P. and Vásquez, A. Identifying relevant National e-government Implementations for an Emerging Country: A Selective Survey, in E. Ferro, H.J. Scholl, M.A. Wimmer (Eds.): Proc. of the 7th Int. Conf. on Electronic Government, EGOV 2008, Trauner Verlag Druck Publisher, Torino, Italy, , (2008).