Using Agent-based Simulation for IT Governance Evaluation

Transcription

1 Using Agent-based Simulation for IT Governance Evaluation José Tomaz Doctoral Program on Complexity Sciences ISCTE-IUL Abstract. IT governance, i.e. the way organizations manage IT resources, has became a key factor for enterprise success due to the increasing enterprise dependency on IT solutions. In fact, empirical research has shown a strong link between effective IT governance and organizational performance: organizations may increase their return on IT investments by as much as 40 % with the help of well-organized IT governance. Actual IT governance practices are strongly influenced by empirical knowledge and professional experience. One well accepted framework is COBIT, although there is no deep reflexion on its own ontology and application. This paper describes an agent-based simulation model for IT-Governance analysis. The model provides a convenient characterization of a given organization (e.g. socio- technical structures and culture, qualified agents), IT resources and specific governance practices (e.g. accountability, responsibility, delegation) and may be used to study relationships between governance practices and organizational goals and sustainability even in risky situations and scarcity of resources. The model is being developed in the course of ongoing PhD research. We finalize the paper by referring different possible ways of validating the proposed agent-based simulation model and how we intend to analyse and detect emergence within this context. Keywords: Agent, Alignment, Emergence, IT Governance, Management, Maturity, Model, Process, Risk, Social Simulation. Introduction In recent years IT governance knowledge area, has been the focus of an increasing interest from both professionals and researchers []. Several issues made its contribution to explain this phenomenon: () Business and support activities are dependent and embedded in IT systems. Frequently those systems are automated and working continuously in different geographical locations. (2) Consequently IT investment costs had soared and business failure and success are more dependent on IT. (3) Following various scandals lawmakers and regulators are demanding more accurate and timely information reporting to assure corporate governance (i.e., Sarbanes-Oaxley, Patriot Act, Anti Money Laundering). (4) Business and organization s professionals wish to improve the accountability of IT resource usage. (5) Corporate governance needs an integrated evaluation performance view among different enterprise functions and IT. Then IT should deliver value to business and be aligned with the achievement of organization s goals. (6) Response to fast changes in business environment from product lifecycle to mergers and acquisitions. (7) Ensure business continuity and resilience to face natural disasters and hacker attacks. To

2 2 Using Agent-based Simulation for IT Governance Evaluation address such bold challenges IT must be managed based on accurate measures of its own processes regarding subjects like risks, requirements, service levels or costs involving many technical and management areas. According to previous research [2], organizations may increase their return on IT investments by as much as 40 % with the help of well-organized IT governance. Recent studies demonstrate that IT governance adoption improved organizational performance, regarding profitability measures. The effect is more significant one year after the adoption of the framework, and as long as the control mechanisms get more mature, more expressive are their benefits [3]. A lot of research has been done related to IT governance issues, but it is mainly normative, interested in theoretical and methodological issues or empirical and dedicated to collecting enterprise data to be analyzed later on. This paper is part of a PhD research thesis to present a novel approach to IT governance evaluation using an agent-based modeling approach, not only to explain organizations case studies behavior, but also to be used in forecasting its future results, providing us some insights into the emergence of macro level phenomena from micro level actions [4]. In the following sections of this paper it will be described the IT governance foundations and the COBIT framework (section 2) and the agent based simulation modeling (section 3). Finally conclusions and future work of this PhD research project will be presented (section 4). 2 IT Governance A definition by the Information Systems Audit and Control Association (ISACA) states that IT governance is the responsibility of executives and the board of directors, and consists of the leadership, organizational structures and processes that ensure that the enterprise s IT sustains and extends the organizations strategies and objectives [5]. Following Jacobson, IT governance is broadly understood to represent how organizations structure and manage IT resources [6]. Independently of the definition it is commonly understood that it is an important area of research in many disciplines, and some empirical research has shown a link between effective IT governance and organizational performance [2]. Several studies provided useful examples of what effective IT governance should look like. Some frameworks emerged, and are being used in organizations, like IT Infrastructure Library (ITIL) [7], Capability Maturity Model Integration (CMMI) [8] and Control Objectives for IT and related Technology (COBIT) 2. These framework models can be seen as an interesting object of research, because they are widely spread and also incorporate a huge amount of consolidated knowledge [9]. However they had different genesis and purposes. ITIL was originally developed by a British Government agency 3 to establish best practices and a standard of IT service quality that customers should demand from suppliers and as evolved to delivery of IT CMMI is a Trademark of Carnegie Mellon University and maintained by the Software Engineering Institute (SEI) 2 COBIT is a Trademark of IT Governance Institute (ITGI) 3 Central Computer and Telecommunications Agency (CCTA)

3 Using Agent-based Simulation for IT Governance Evaluation 3 services. In the last version it also addresses strategy within the enterprise. CMMI has its roots on software development lifecycle (CMM), as a process improvement approach providing essential elements to organization s processes, promoting process improvement goals, priorities, quality and evaluation. ITIL tends to be holistic, but lacks an internal structure. CMMI also has a coherent structure, but it mainly focuses on development. Since COBIT is holistic and represents almost all tasks and processes needed by an IT organization, it was chosen as a basic framework model to support this work on IT governance. Recently, some scholars began to examine the COBIT framework from an academic perspective as it relates to auditing and found it valid and useful [9]. This framework is structured in domains, processes and other components, closed in itself and self-contained [0]. The four domains are Plan and Organize (PO), Acquire and Implement (AI), Deliver and Support (DS) and Monitor and Evaluate (ME), in a total of 34 processes and 20 control objectives, following a sequence known as Deming cycle (Plan, Do, Check, Act) []. For each process various components are considered, such as business requirements, inputs/outputs, IT goals, risk drivers, controls, activities, resources, responsibilities, metrics and a maturity model. Enterprise contingencies can be modelled using an IT risk management framework [2]. Metrics and measurements can be related to the maturity model, supported by organizations strategy execution and balanced scorecard (BSC) [3] reporting. When evaluating an enterprise using this governance framework, it is possible to establish relations among business strategy alignment and operational IT alignment, its impact in process performance and value creation [4]. To better understand the basics of the COBIT framework, we can use a simple example to describe a possible process sequence to deliver an IT solution to a business requirement. Considering an existing enterprise, with IT governance principles and an IT organization already defined. Usually, when a new business area or product is created it is necessary to develop a new IT solution. Then business requirements are presented to the IT organization. The need for an IT solution is integrated in a plan and organize COBIT domain in order to decide the best way to implement a solution. A define a strategic plan (PO) process would address business needs and requirements integrating them in tactical plans and IT portfolio management granting an initial business-it alignment. Other processes in this domain would be used like manage IT investments (PO5), resources (PO7), compliance (PO8), risks (PO9) and finally projects (PO0). As soon as a project starts acquisition and implement domain processes would select and acquire software (AI2) and infrastructure (AI3), manage changes (AI6) and install an accredited solution (AI7). Then deliver and support processes would be involved to define and manage SLA (DS), ensure continuous service (DS4), system security (DS5) and manage problems (DS0). Finally monitor and evaluate domain processes would evaluate IT performance (ME) and provide IT governance (ME4) to top management.

4 4 Using Agent-based Simulation for IT Governance Evaluation IT Governance Principles PO Define Strategic IT Plan PO5 Manage IT Investment PO7 Manage Resources AI2 Acquire Maintain SW Applications AI6 Manage Changes AI3 Acquire Maintain Tech. Infrastructure AI7 Install Accredit Solutions Changes Business Requirements PO8 Compliance External Requirements PO9 Assess Manage IT Risks ME Monitor Evaluate IT Performance ME4 Provide IT Governance DS Define Manage SLA DS4 Ensure Continuous Service DS5 Ensure System Security DS0 Manage Problems PO0 Manage Projects Fig.. COBIT block process diagram representing the example. This cycle works continuously ensuring future project changes to the solution, and in a similar way for other IT projects. Each process has input and output to other processes, its own human resources, with defined responsibilities and accountabilities, targets and indicators, consolidated in a maturity level. If the project clearly finishes on time and on budget, accomplishing the pre-defined business goals, it delivers value, a positive return on investment (ROI) and reflects an adequate business-it alignment. Probably the maturity levels are appropriate to achieve these results. COBIT framework is adequate to describe the processes, input/output information flows, identifying IT and business goals and responsibilities, adjustable to different enterprise structures. It works even better in a posteriori analysis. Also, this short example follows a linear sequence description, but in real life this is quite different, since contingencies, risks and unexpected events would create disruptions, affecting negatively an organization. The responses to fix those disruptions can be previously defined within the organization, but its use can be different depending on many variables. Moreover, the project development has to deal with different skill levels and heterogeneous behavior from staff members, which also contributes to

5 Using Agent-based Simulation for IT Governance Evaluation 5 unexpected results that can t be predicted using a framework like COBIT and linear simulation techniques. 3 Agent-Based Simulation Modeling It is considered that social environments like enterprises or other organizations, largely dependent on human resources are complex with almost infinite variables, parameters and contingencies to be considered. For example, accumulated experience in organizations and staff members, contingencies, failures or success stories, and its resilience will shape eventually the enterprise future. It shows, that human organization s are complex systems, using complex in the sense that the behavior of the system as a whole cannot be determined by partitioning it and understanding the behavior of each of the parts separately, which is the classic strategy of the reductionist physical sciences. One reason why human organizations, and IT governance in particular are complex is that there are many, non-linear interactions between their units, that is among people. The interactions involve the transmission of knowledge, information and materials in the IT governance environment that often affect the behavior of the agents. Then it becomes impossible to analyze a society as a whole by studying the individuals, or the agents within it, one at a time. The behavior of the enterprise environment is said to emerge from the actions of its units. There are many examples of emergence in social systems; indeed, it may be that almost all significant attributes of social systems are emergent [5]. Business Goals Information Criteria COBIT Domain -PO-Plan and Organize -AI-Acquire & Implementation -DS-Deliver & Support -ME-Monitor & Evaluate Governance Focus Areas -Strategic Aligment -Value Delivery -Risk Management -Resource Management -Performance Management IT Goals Process Maturity Model Input Output Metrics AssociationClass RACI Chart Activities * - * - Function Resources Results Control Objective -Application -Information -Infrastructure -People Business Requirements Control Practice Fig. 2. COBIT UML Class Model.

6 6 Using Agent-based Simulation for IT Governance Evaluation Furthermore when we try to evaluate, or test in the real-world the origins and impacts of contingencies, in social simulation, some ethical issues could be raised, since it is not acceptable to replicate human social events using real individuals as guinea pigs [6]. In order to represent processes in an IT organization, the COBIT was chosen as a base model, since it provides a holistic and comprehensive approach and integrates professional experience in the area, along with scientific knowledge. Using a multi agent based approach seems to be adequate to model the problem, where programmed software agents, will interact in a virtual enterprise IT governance environment [5]. As shown in figure 2 main classes of COBIT framework are represented using Unified Modeling Language (UML) [7]. The agents will have a controlled autonomy degree to interact with each other, following collaboration, delegation, competition or responsibility patterns and goals to achieve. Norms and rules will be included to regulate agent s behavior. The interaction amidst agent s involving the transmission of information and data will affect its behavior. Actually, each relationship to be modeled has to be accurately specified, and each parameter has to be assigned a value, to be used in simulation. Then agents will represent individuals, organizations or structures, like in the real world that is being modeled, and agent s interactions will represent interactions between real world individuals. System AI6 Manage Changes -Origin * -Req * Change Request Form request «extends» Standard Procedure Client-Req originator «uses» -Coord * -Eval Evaluates «extends» Cost-Benefit of Change * Expected ROI ROI Criteria Change Coordinator «uses» -Owner-Analysis * * Project Owner Analysis Request Change Project Plan «uses» «extends» Project Baseline Implementation Fig. 3. Use case diagram of agent behaviour in Manage Changes Process (AI6). Figure 3 shows agents use case diagram in manages changes process (AI6), from COBIT Acquire and Implementation (AI) domain, to be implemented in the ABMS. At this point social simulation using agents programmed by researchers seems to be a good option to override such limitations, as we can perform reruns over and over again, fine-tuning parameters, testing attributes, different communications flows or organizational structures in order to get some insight to be applied in the real-world.

7 Using Agent-based Simulation for IT Governance Evaluation 7 Thus, agent-based modeling and simulation of human social behavior is very important to study IT governance area problems, its contingencies and risks involved using human-like autonomous agents that can be changed, adapted and customized to interact with environmental conditions on simulation environments. Organizations using IT governance and management principles try no standardize behaviors and practices applying norms and regulations, but intrinsic human behavior, or technical and environmental contingencies, some internal to its structure or coming from the external environment would impact the outcomes of each particular enterprise. Attributes can be used as variables, and are characteristic of each object. Message buffers will be used to hold temporarily messages sent among agents in the environment. Besides a description to a static model for the environment, some dynamic aspects are considered to describe the sequence of actions performed by agents. Since simulation of complex social processes like these involve the estimation of many parameters that can impact negatively in the simulation output results, parameters will be limited to the essential ones, avoiding oversimplifying to the point that the system no longer produces meaningful results. Agent-based modeling and simulation (ABMS) is a new approach to modeling systems comprised of interacting autonomous agents [6], promising far-reaching effects on the way that businesses use computers to support decision-making and researchers use electronic laboratories to do research. The proposed ABMS environment to simulate IT governance in organizations is represented in figure 4, using a UML component model. The main components are related to business requirements and governance as an input area. COBIT domains and processes represent a core processing area, reacting to contingencies and risk events. An output area, represented by monitor and evaluate processes, and a maturity model associated to these COBIT processes. Input Core Processes Contingencies Plan & Organize Governance Strategy Risk Management Engine Acquire & Implement Business Case Risk Event Handling Deliver & Support Project Output Monitor & Evaluate Maturity Model Fig. 4. UML Component Model representing an IT governance ABMS environment.

8 8 Using Agent-based Simulation for IT Governance Evaluation Each component module contains its own agents that can be diverse, heterogeneous, and dynamic in their attributes and behavioral rules [6]. Agents are also considered to be autonomous and capable to interact with other agents and the environment [8]. Constraints and contingencies will be simulated using specific mechanisms provided by the simulation environment as shown in figure 4. Measurement and metrics will be included in the model, in order to evaluate the progress and completion using balanced scorecards and maturity. To complete the model the environment in which these objects are located is specified as an object with its attributes, including the current simulated time. A hierarchical IT governance balanced scorecard (BSC) is being used for IT performance measurement [9]. The system is based in the Kaplan-Norton Balanced Scored Card (BSC), with four perspectives, the user, operational excellence, business contribution and future orientation perspectives [20]. The use of a BSC methodology provides more detailed operational level assessment information to manager s responsibility areas and a holistic overview of IT governance whereas those professionals have their responsibility areas. For instance, qualitative measurements in the category of project visibility and control include a higher capability maturity level (CMMi) [2] and increased project sponsor satisfaction. Quantitative measurements should evaluate healthy projects, their on-budget and on-time performance, and reduced project management costs. Qualitative results of IT services automation are associated with improvements in the quality of audit and regulatory compliance [22] and the reduction of overall operational IT spending. Higher productivity and lower IT personnel costs are directly linked to working time reductions though faster incident handling while compliance with architectural standards is enforced [23]. The COBIT IT governance framework provides both a measurement methodology and IT process structure that provides a foundation for the measurement of process capability maturity across the lifecycle of IT investment [24]. In COBIT the process maturity has six levels, allowing a detailed profiling for each process. In this ABMS, processes and maturity levels are modeled and managed in the IT governance environment for each simulation run. However, maturity levels to assure completion of goals and objectives may differ among processes in the same organization. As already stated, it is not mandatory that every single governance process should be in the highest maturity level, as for some processes that can be too costly or present a lower performance. Actually we ll demonstrate that regarding best results, value creation, return on investment, efficient enterprise IT governance could be achieved with balanced maturity levels for different processes. To validate the simulation model, data collected from real world enterprises and already present in surveys or case studies will be used in comparison tests to evaluate this ABMS performance and accuracy results. 4 Conclusions Definitely, IT Governance presents an important role in modern organizations since every business or technical support process depends on it, and its management involves many disciplines and social interactions, pointed out by complexity.

9 Using Agent-based Simulation for IT Governance Evaluation 9 Traditional tools used to model, quantify, calculate and evaluate data are not enough to explain or find accurate results. Moreover, through this analysis new relations and highlights could emerge, to better understand the phenomena in the governance area. In this paper we propose a novel approach to IT Governance evaluation, using agentbased modeling and simulation. Several quantitative models, defining critical success factors and using key performance indicators, mapped into a balanced score card are considered to characterize an associated maturity model. Economic and financial indicators like value creation, process maturity and ROI will be considered as evaluation variables. 4. Future Work In order to achieve a more detailed environment, modeling goes on detailing attributes, properties and relations amongst agents, processes and typical structures in organizations. Further research is ongoing to include goals, risks, critical success factors, key performance and financial indicators. A selected set of indicators and parameters will allow the establishment of an organizational environment simulating a real world enterprise. Furthermore, registered data from surveys and case studies will be selected to be used in setup environment parameters and compare results obtained with the agent based modeling simulator. A detailed Java environment will be setup and programmed following the requirements of the present model. In the next phase, implementation will start with a simplified prototype model, and its features and functionalities will increase accordingly to the experience achieved and system dynamics, keeping our focus on what we are trying to obtain from the model. These results are expected to show some light on IT value management [25], business and IT alignment and its contribution towards IT governance. References. Dahlberg,T. and Kivijärvi, H.: An Integrated Framework for IT Governance and the Development and Validation of an Assessment Instrument, Proceedings of the 39th HICSS (2006) 2. Weill, P. & Ross, J.: IT Governance: How Top Managers Manage IT Decision Rights for Superior Results. Boston: Harvard Business School Press (2004) 3. Lunardi, G.L., Becker, J.L., Maçada, A.C.G.: The Financial Impact of IT Governance Mechanisms Adoption: An Empirical Analysis with Brazilian Firms, Proceedings of the 42th HICSS (2009) 4. Gilbert, N.: The simulation of social processes. In N. Ferrand (Ed.), Modèles et Systèmes Multi-Agents pour la Gestion de l'environment et des Territoires (pp. 2-37). Clermont- Ferrand: Cemagref (2000) 5. IT Governance Institute.: COBIT 4.-Framework, Control Objectives, Management Guidelines, maturity Models. (2007) 6. Jacobson, D. Revisiting IT Governance in the Light of Institutional Theory, Proceedings of the 42nd HICSS (2009) 7. Bon, J., Jong, A.: Foundations of IT Service Management Based on ITIL V3. Van Haren Publishing (2007)

10 0 Using Agent-based Simulation for IT Governance Evaluation 8. Ahern, D.M., Clouse, A., and Turner, R.: CMMI distilled: A practical introduction to integrated process improvement. Addison-Wesley, 2nd ed. Publications (2004) 9. Tuttle, B., & Vandervelde, S. D.: An Empirical Examination of COBIT as an Internal Control Framework for Information Technology. International Journal of Accounting Information Systems, 8, (2007) 0. Goeken, M.; Alter, S.: Representing IT Governance Frameworks as Metamodels, in: Proceedings of the 2008 International Conference on e-learning, e-business, Enterprise Information Systems, and e-government (EEE 08), World Congress in Computer Science (Worldcomp 08), July 4-7, Las Vegas Nevada (2008). Deming, W.E Elementary Principles of the Statistical Control of Quality, JUSE. 2. IT Governance Institute.: The Risk IT Framework. (2009) 3. Kaplan, R. S., Norton, D. P.: The balanced scorecard: measures that drive performance, Harvard Business Review, vol. 70, pp (992) 4. Beimborn, D., Schlosser, F., Weitzel, T.: Proposing a Theoretical Model for IT Governance and IT Business Alignment. Proceedings of the 42 nd HICSS (2009) 5. Gilbert, N.: Agent-based social simulation: dealing with Complexity. (2004) 6. Macal, Charles M., North, Michael J.: Tutorial on agent-based modelling and simulation, Proceedings of the 37th conference on Winter simulation, December 04-07, Orlando, Florida (2005) 7. Booch, G., Rumbaugh, J. And Jacobson, I.: The Unified Modeling Language User Guide. 6 th edition. Addison-Wesley, Reading, MA (2000) 8. Wooldridge, M.: An Introduction to Multi Agent Systems. 2 nd Ed. Wiley (2009) 9. Grembergen, W.: Introduction to the Minitrack IT Governance and its Mechanisms. Proceedings of the 36th HICSS (2003) 20. Ribbers, P., Peterson, R. and Parker, M.: Designing Information Technology governance processes: diagnosing contemporary practices and competing theories, Proceedings of the 35 th HICSS (2002) 2. Gerrard, M.: Creating an effective IT governance process, Gartner, Stamford COM (2003). 22. Eisenhardt, K.M.: Building theories from case study research, Academy of Management Review, 4(4), (989) 23. Heier, H., H., Borgman, H., Mileos, C.: Examining the Relationship between IT Governance Software, Processes, and Business Value: A Quantitative Research Approach, Proceedings of the 42 nd HICSS (2009) 24. Debreceny, R., Gray, G.: IT Governance and Process Maturity: A Field Study. Proceedings of the 42 nd HICSS (2009) 25. Maes, K,. Haes, S., Grembergen, W.: How IT Enabled Investments Bring Value to the Business : A Literature Review. Proceedings of the 45 th HICSS (20)