A METHOD FOR CREATING ENTERPRISE ARCHITECTURE METAMODELS - APPLIED TO SYSTEMS MODIFIABILITY ANALYSIS

Transcription

1 International Journal of Computer Science and Applications c Technomathematics Research Foundation Vol. 6, No. 5, pp , 2009 A METHOD FOR CREATING ENTERPRISE ARCHITECTURE METAMODELS - APPLIED TO SYSTEMS MODIFIABILITY ANALYSIS ROBERT LAGERSTRÖM ULRIK FRANKE PONTUS JOHNSON JOHAN ULLBERG Industrial Information and Control Systems, Royal Institute of Technology SE Stockholm, Sweden {robertl, ulrikf, pj101, johanu}@ics.kth.se Enterprise architecture models can be used in order to increase the general understanding of enterprise systems and specifically to perform various kinds of analysis. It is generally understood that such modeling encompasses general scientific issues, but the monetary aspects of the modeling of software systems and their environment are not equally well acknowledged. Even more so, creating a good metamodel for enterprise software systems analysis is an important but challenging task. The present paper describes a method for creating metamodels for such analysis. The enterprise architecture models are formalized using probabilistic relational models, which enables the combination of regular entityrelationship modeling aspects with means to perform enterprise architecture analysis. The proposed method for creating metamodels is general, however this paper presents the method by creating a metamodel for systems modifiability, i.e. the cost of making changes to enterprise-wide systems. The method and the method outcome, i.e. the metamodel, is validated based on survey and workshop data and the applicability of the metamodel is illustrated with an instantiated architectural model based on a software change project at a large Nordic software and hardware vendor. Keywords: Enterprise architecture; Metamodeling; Probabilistic relational models; Software modifiability. 1. Introduction Managing software systems today is a complex business. In order to achieve effective and efficient management of the software system landscape it is essential to be able to assess the current status of system properties such as availability, performance, security, and modifiability, as well as predict their values in different future scenarios. Estimation of these properties is however a great challenge that to a large extent can be addressed by introducing relevant models as a means of abstraction, which can be achieved with enterprise architecture modeling. The purpose of this paper is to propose a method for creating enterprise architecture metamodels that 89

2 90 R Lagerström, U Franke, P Johnson, J Ullberg support analysis of various system properties Enterprise architecture In recent years, Enterprise Architecture (EA) has become an established discipline for business and software system management [Ross et al. 2006]. Architecture models constitute the core of the approach and serve the purpose of making the complexities of the real world understandable and manageable to humans [Winter and Fischer 2007]. Enterprise architecture ideally aid the stakeholders of the enterprise to effectively plan, design, document, and communicate IT and business related issues, i.e. they provide decision support for the stakeholders [Kurpjuweit and Winter 2007]. A key underlying assumption of the EA models is that they should provide some more aggregated knowledge than what was merely put into the model in the first place. Software system architecture, for instance, does not only keep track of the set of systems in an enterprise and their internal relationships, it also provide information about the dependencies between the systems. Conclusions can be drawn about the consequences in the enterprise given that one specific system is unavailable. The enabling of this type of analysis is extremely important in providing value of EA for its stakeholders. Unfortunately however, EA frameworks rarely explicitly state neither what kinds of analyses that can be performed given a certain model nor the details on how the analysis should be performed [Johnson et al. 2007; Franke et al. 2009b]. Another permeating problem in EA modeling is the uncertainty that is related to the model [Johnson et al. 2007]. For instance, whether a model is the result of a very thorough and recent investigation or a quick read-through of somewhat old documents will impact the quality of the decision support that the model offers to its stakeholders. Are all the software systems in the model still in use, is the data flow still as depicted, and does the process structure really illustrate what is actually happening? This kind of uncertainty is not addressed by EA frameworks of today. Again the user of the models is simply left to her best knowledge or gut feeling when estimating to what extent the EA model, and the analyses based on it, can and should be trusted. This paper presents a formalized approach to enable analysis of EA models by connecting the analysis to the language used to express the models, the metamodel. In other words, the approach devises analysis frameworks in terms of metamodels suitable for analysis of different properties such as modifiability and security. Specifically, the focus of the approach is to identify and manage the problems that arise when such metamodels are created. The approach also copes with empirical uncertainties in the application of the devised analysis framework by not considering the information in EA models as fixed constants but rather as probabilities. The underlying fundamental formalism in the approach is called Probabilistic Relational Models (PRMs) [Friedman et al. 1999].

3 A method for creating enterprise architecture metamodels applied to systems modifiability analysis Enterprise system modifiability As discussed in the previous subsection, enterprise architecture models can be used to analyze different system properties and provide information for the decision maker regarding different scenarios. In this paper enterprise software system modifiability, i.e. the cost of making changes to enterprise-wide software systems, will be considered as a running case to illustrate the proposed metamodel creation method. Business environments today progress and change rapidly to keep up with evolving markets. Most business processes are supported by software systems and as the business processes change, the systems need to be modified in order to continue supporting the processes. Modifications include extending, deleting, adapting, and restructuring the enterprise systems [Bass et al. 1998]. The modification effort range from adding a functional requirement in a single system to implementing a service oriented architecture for the whole enterprise. An essential issue with today s software systems is that many of them are interconnected, thus a modification to one system may cause a ripple effect among other systems. Also, numerous systems have been developed and modified during many years. Making further changes to these systems might require a lot of effort from the organization, for example due to a large amount of previous modifications implemented ad hoc. Problems like these pose questions for IT decision makers such as: Is there enough documentation describing the systems, and has the documentation been updated correctly after each modification? Is the source code easy to understand? Which systems are interconnected? Several studies show that the modification work is the phase that consumes the greatest portion of resources; Harrison and Cook reports that over 70 % of the software budget is spent on maintenance [Harrison and Cook 1990], Pigoski refers to studies stating that the maintenance cost, relative to the total life cycle cost of a software system, has been increasing from 40 % in the early 1970s up to 90 % in the early 1990s [Pigoski 1997], and Jarzabek states that the cost of maintenance, rather than dropping, is on the increase [Jarzabek 2007]. The activities of modifying enterprise systems are typically executed in projects, and IT decision makers often find it difficult to predict and plan their change projects. Thus, a large proportion of the projects aiming to modify a software system environment fail. That is, the projects tend to take longer time and cost more than expected. Laird and Brennan states that 23 % of the software projects are cancelled before completion, whereas of those completed only 28 % were delivered on time, and the average software project overran the budget by 45 % [Laird and Brennan 2006]. This can often occur due to lack of information about the systems being changed. According to Laird and Brennan, software engineers must be able to understand and predict the activities, as well as manage the risks, through estimation and measurement [Laird and Brennan 2006]. Therefore, it would be useful for IT decision makers to gather more information in a structured manner and use this information to analyze how much effort a certain modification to an enterprise

4 92 R Lagerström, U Franke, P Johnson, J Ullberg software system would require. This paper will address these issues of software change by emplying enterprise systems modifiability analysis as a running case, thus providing a metamodel for assessment of software change project cost and manage the problems that occur while devising such a metamodel Creating enterprise architecture metamodels Metamodels are the core of enterprise architecture. They describe the fundamental artifacts of business and IT. Such high level models provide a common language and a clear view on the structure of and dependencies between relevant parts of the organization [Winter and Fischer 2007]. However, when creating a metamodel for enterprise architecture analysis there are some aspects that need to be considered. First, a great number of factors influence system properties. Second, the factors are intertwined in a complex manner. The researcher or practitioner who sets out to model these interdependencies thus inevitably faces a discomforting amount of modeling choices, all of which to some extent influence the ability of the final assessment framework to provide accurate decision support for management decisions. Furthermore, all modeling choices represent a cost in terms of collecting the information needed for actually using the model. This cost, whether expressed in money, effort, or time, must be kept under control, lest the entire modeling effort be misguided. This paper delineates a method for creating enterprise architecture metamodels for analysis. While the focus is assessment of software modification projects, the method itself aims to be general. The method and the resulting metamodels address the concerns mentioned above. As all EA frameworks, ours aim to clarify complex interdependencies, but is at the same time subject to uncertainties. As with all estimates, ours are built upon basic scientific questions of measurement, scales and precision. As with all metamodels, ours requires careful consideration of the classes and relationships included. A number of general problems, outlined in section 2, serve as a roadmap for the rest of this paper. The theoretical considerations necessary for creating a metamodel presented can all be related to this roadmap. In that sense, it provides a unifying framework, explaining how the different parts together constitute a single whole. In a previous paper, a formal argument was given [Franke et al. 2009c] for the possibility to treat all the roadmap problems jointly. In this paper, we expand the treatment slightly, putting it into the broader context of an entire EA approach, rather than single software measures considered by themselves Outline The remainder of the paper is structured as follows: Section 2 introduces general modeling problems for enterprise architecture analysis. Section 3 presents the probabilistic relational models which serve as the underlying formalism for the enterprise architecture metamodels proposed for analysis. The following section proposes the

5 A method for creating enterprise architecture metamodels applied to systems modifiability analysis 93 enterprise architecture metamodel creation method which includes a development process and guidelines supporting the creation of metamodels for analysis. Next, in section 5 an instantiated model for a software change project is described in order to show the applicability of the modifiability metamodel. Section 6 contains data for validation of the method by considering the correctness of the qualitative structure of the metamodel and by discussing the usability of the creation method. In section 7 related work is presented and finally, section 8 summarizes the paper with conclusions. 2. General modeling problems for enterprise architecture analysis a priori Complexity 2 1 a posteriori Cost of Change Halstead Lines of Code Cyclomatic 3 Fig. 1. A schematic overview of the modeling process when change cost analysis is the main purpose, displaying the five problems listed in the text. Many system properties availability, performance, security, and modifiability, to name a few share the elusive feature that while they are easy to define a posteriori, i.e. after system implementation, such definitions give precious little guidance on how to ensure them a priori, i.e. before system implementation. For example, measuring the cost of change of a system a posteriori is mere book-keeping. But assessing it beforehand is a formidable task. Such assessment must be carried out by measuring variables available prior to the modification. The literature, provides a wealth of different methods to make such cost of change assessments. The modeler however cannot afford to employ them all, she requires accurate and cost-efficient decision support. The same holds true for the assessment of other system properties, even though we shall use cost of change (modifiability) as a running case for the remainder of the present paper. Fig. 1 is a generic depiction of a modeling process, aiming to create a framework for prediction of costs associated with software modification projects. Five key

6 94 R Lagerström, U Franke, P Johnson, J Ullberg problems, numbered in the figure, need to be addressed by such a framework: (1) Correlation and causality. (a) Correlation. The choice of an a priori measurement quantity is the problem of finding a measure that correlates accurately with the sought a posteriori quantity. A typical example of this question is given by the cost of change, i.e. the cost for modifying or amending a software program, which often is estimated by software complexity. Software complexity in itself is just a measure of certain properties of the program code. No complexity measures make any mention of the cost or time spent creating or modifying this code. Nevertheless, it is generally assumed that complex computer programs are difficult to maintain and modify. This is the correlation (and causal relation) that, supposedly, binds together the a priori metric of complexity with the a posteriori system property of cost of change. (b) Causality. Causality is closely linked to correlation, but they are not the same, and there is an ever-present need to separate the one from the other. In the present context, we can discern two different decision-support aspects: First there is a prediction aspect, reflecting the need for a priori accurate measures of how things turn out a posteriori. To achieve this, correlation is sufficient. Second, however, there is an action guiding aspect, reflecting the need for control. Here, we take another step: seeing an a priori estimate of high project costs, how can we act so as to lower these costs? To achieve this, mere correlation is insufficient: we need causal relations, so that our actions causally impact and change the cost. In the rest of this article, we try to separate correlation from causality, usually aiming to create models that are indeed causal, not merely exhibit correlations. (2) Uncertainties. Uncertainty is inherent in any EA analysis. However, there are many kinds of uncertainties, and to better understand and alleviate them, they can be broken down into sub-categories. The following list is based on [Johnson et al. 2007]: (a) Definitional uncertainty. Most concepts can be interpreted in many different ways. One example is the notion of software complexity, that might be interpreted as for instance Halstead complexity, cyclomatic complexity, or simply lines of code. Since there is no general agreement on the meaning of all concepts, it is imperative that assessment frameworks are explicit about the intended meaning of the terms used. (b) Theoretical heterogeneity. There is no general agreement on a single framework within which all EA knowledge can be expressed. Despite attempts to rectify this, such as [Franke et al. 2009b; International Standardization Organization/International Electrotechnical Committee 2007], this is nevertheless likely to remain the case for the foreseeable future.

7 A method for creating enterprise architecture metamodels applied to systems modifiability analysis 95 (c) Causal uncertainty. It is rare that empirical phenomena are fully understood. For example, it is unlikely that an IT decision maker knows for certain to what extent the introduction of a new tool, a new training procedure, or the adoption of a more mature change management process will increase modifiability. (d) Empirical uncertainty. Whenever architecture models of an enterprise are created, there is a risk of erroneous modeling. Incorrect modeling information can easily arise due to lack of time or manpower, immature processes, or simple mistakes. Furthermore, the information might have been correct when collected, but has become obsolete by the time of its use. As is concluded in [Johnson et al. 2007], these uncertainties call for a probabilistic analysis framework. Section 3 describes the formalism used throughout this paper. (3) Measurement devices. Representational theory of measurement is about mapping an empirical relational system, i.e. observable reality, into a mathematical model, a numerical relational system [Hand 1996]. As put in [Nagel 1931], measurement is the correlation with numbers of entities which are not numbers. This correlation aspect of measurement is one of our foremost interests for the purpose of this paper. To the modeler, the use of a particular measuring device will affect the model s theoretical performance by its inherent accuracy limits. One of the most convenient methods for collecting information about certain characteristics of a system is to consult an expert in the field. In this paper, we consider the use of expert estimation to be a kind of measurement, on a par with other methods, such as using measurement software. (4) The selection of appropriate scales. That the choice of scales has non-trivial repercussions on scientific models has been known at least since Stevens seminal 1946 paper [Stevens 1946]. The fact that different scales permit different statistical operations creates constraints on the permissible structure of assessment frameworks. Following our example, we note that interval scale concepts such as LoC, Halstead and cyclomatic complexity might be transformed onto ordinal scales when discretized. One such possible ordinal scale is High, M edium, Low. The loss of information is evident, but often unavoidable for instance for expert estimates. (5) Discretization of measurement variables. A special case of the preceding problem is discretization of properties that are continuous. In the context of measurement, discretization not only approximates a continuous phenomenon, but also enables mapping onto a predetermined scale. In general, of course, such mappings entail a loss of information. The general trade-off thus amounts to achieving the simplifications of discretization while retaining as much information as is necessary in a given context. Throughout this paper, we will address the problems above, using them as start-

8 96 R Lagerström, U Franke, P Johnson, J Ullberg ing points in the process of creating the enterprise architecture metamodels for analysis. As it turns out, the correlation, causality, and uncertainty problems lend themselves to a more general analysis, whereas the measurement devices, scales and discretization need to be addressed more on a case by case basis. In the following, thorough examples from both categories are given. 3. Probabilistic relational models As stated in the introduction Probabilistic Relational Models (PRMs) serve as the underlying formalism for the enterprise architecture metamodels proposed for analysis. A PRM [Friedman et al. 1999] specifies a template for a probability distribution over an architecture model. The template describes the metamodel for the architecture model, and the probabilistic dependencies between attributes of the architecture objects. A PRM, together with an instantiated architecture model of specific objects and relations, defines a probability distribution over the attributes of the objects. The probability distribution can be used to infer the values of unknown attributes, given evidence of the values of a set of known attributes. An architecture metamodel M describes a set of classes, X = X 1,..., X n. Each class is associated with a set of descriptive attributes and a set of reference slots. The set of descriptive attributes of a class X is denoted A(X). Attribute A of class X is denoted X.A and its domain of values is denoted V (X.A). For example, a class System might have the descriptive attribute Size, with domain {large, medium, small}. The set of reference slots of a class X is denoted R(X). We use X.ρ to denote the reference slot ρ of X. Each reference slot ρ is typed with the domain type Dom[ρ] = X and the range type Range[ρ] = Y, where Y X. A slot ρ denotes a relation from X to Y in a similar way as Entity-Relationship diagrams. For example, we might have a class Documentation with the reference slot Describes whose range is the class System. An architecture instantiation I (or an architecture model) specifies the set of objects in each class X, the values for the attributes, and the reference slots of the objects. For example, Fig. 8 presents an instantiation of the change project metamodel of Fig. 6. It specifies a particular set of changes, systems, documents, etc., along with values for each of their attributes and references. For future use, we also define a relational skeleton σ r as a partial instantiation which specifies the set of objects in all classes as well as all the reference slot values, but not the attribute values. A probabilistic relational model Π specifies a probability distribution over all instantiations I of the metamodel M. This probability distribution is specified as a Bayesian network [Jensen 2001], which consists of a qualitative dependency structure and associated quantitative parameters. The qualitative dependency structure is defined by associating with each attribute X.A a set of parents P a(x.a). Each parent of X.A has the form X.τ.B where

9 A method for creating enterprise architecture metamodels applied to systems modifiability analysis 97 B A(X.τ) and τ is either empty, a single slot ρ or a sequence of slots ρ 1,..., ρ k such that for all i, Range[ρ i ] = Dom[ρ i+1 ]. For example, the attribute Cost of class ChangeP roject may have as parent ChangeOrganization.Developer.Expertise, thus indicating that the cost of a prospective software modification project depends on the expertise of the developers employed in the organisation. Note that X.τ.B may reference a set of attributes rather than a single one. In these cases, we let x.a depend probabilistically on some aggregate property over those attributes, such as the logical operations AND, OR, and NOR. In this paper we use the arithmetic operations SUM and MEAN as aggregate functions. For instance, if there are several developers engaged in a modification project, we might aggregate the individual developers expertise into a mean expertise of the whole development team. Considering the quantitative part of the PRM, given a set of parents for an attribute, we can define a local probability model by associating a conditional probability distribution (CPD) with the attribute, P (X.A P a(x.a)). For instance, P (ChangeP roject.cost = high ChangeOrganization.Developer.Expertise = low) = 90% specifies the probability that the project cost will be high, given the expertise of the developers involved. We can now define a PRM Π for a metamodel M as follows. For each class X X and each descriptive attribute A A(X), we have a set of parents P a(x.a), and a conditional probability distribution (CPD) that represents P Π (X.A P a(x.a)). Given a relational skeleton σ r (i.e. a metamodel instantiated to all but the attribute values), a PRM Π specifies a probability distribution over a set of instantiations I consistent with σ r : P (I σ r, Π) = P (x.a P a(x.a)) x σ r(x) A A(x) where σ r (X) are the objects of each class as specified by the relational skeleton σ r. A PRM thus constitutes a formal machinery for calculating the probabilities of various architecture instantiations. This allows us to infer the probability that a certain attribute assumes a specific value, given some (possibly incomplete) evidence of the rest of the architecture instantiation. In addition to expressing and inferring uncertainty about attribute values as specified above, PRMs also provide support for specifying uncertainty about the structure of the instantiations. A more detailed description of this topic is, however, beyond the scope of the paper. 4. The enterprise architecture metamodel creation method Creating a good metamodel (probabilistic relational model) is not trivial. Obviously, it is important that the metamodel is tailored for the management tasks it should support, i.e. what kind of analysis the metamodel is intended to support. For instance, if one seeks to employ an enterprise architecture model for evaluating business process efficiency, the information required from the model differs radically from the case when the model is used to evaluate the modifiability of an enterprise software system.

10 98 R Lagerström, U Franke, P Johnson, J Ullberg Addressing this task, the general problems presented in section 2 are used as an underlying framework. In this section, our main focus will be on the first problems; correlation, causality and uncertainties. The issues of measurement devices, scales, and discretization will be addressed subsequently, both in section and in section 5. All the problems, of course, need to be considered when creating and using a metamodel for enterprise architecture analysis Methods for creating the qualitative part of a metamodel This subsection presents the method steps for creating the qualitative part of a metamodel, i.e. the classes, reference slots, attributes and their parents. The basic modeling aim is to ensure the correlation between a priori measurements and a posteriori properties of interest. For any such property, there are numerous candidates for a priori measurements, each with its own set of advantages and disadvantages. Our running case makes this even clearer: the cost of change, i.e. the cost for modifying or amending a software system, can be and is often estimated by the software complexity, as discussed in section 2. However, taking complexity alone as a sufficient indicator of the cost of change is usually not enough. Most estimation methods available indeed use multiple a priori measures. The first step of the method for creating decision support metamodels is thus to find a set of appropriate a priori measures for change cost estimation and decision making, i.e. finding measures with high correlation and causal influence on change cost. This can be done in several ways, for instance by studying research literature, doing experiments and case studies, or using expert opinions. In subsequent iterations, variables causally affecting the variables found in the former iterations are identified. This iterative process continues until all paths of variables, and causal relations between them, have been broken down into variables that are directly controllable by the decision maker, see part 1 of Fig. 2. For instance, it might be identified through literature study that the goal decrease change project cost is affected by enterprise system understandability and change management process maturity. In the next iteration the enterprise system understandability might be affected by the, to the IT decision maker, directly controllable variables system size and system internal coupling. This iterative goal break down structure approach has also been presented in [Lagerström et al. 2009e] as a part of a method to create stakeholder-oriented metamodels Eliciting variables for goal break-down fragments The goal to be broken down in our running case is enterprise system modifiability, i.e. the cost of making changes to enterprise wide systems. The metamodel for modifiability analysis needs to have a high degree of construct validity in the sense that it actually provides good estimates and decision support for the goal variable under analysis. A convenient way of ascertaining a sufficient degree of construct

11 A method for creating enterprise architecture metamodels applied to systems modifiability analysis 99 Goal selection Goal break-down iteration Goal break-down fragment 1 Goal Goal Goal Indirectly controllable Directly controllable 2 Select source and identify start variable Identify and extract variable evidence Translate extracted evidence Select new variable Perform sanity check and control For each variable in the goal break-down fragment 3 Is the variable defined? Yes Is the variable uncontrollable? No Is the variable semicontrollable? No Is the variable directly controllable? Yes No Yes Yes No Introduce definitional variables Delete variable Introduce causal variables Introduce causal variables Fig. 2. Overview of the iterative goal break-down method. validity is to base the information in the goal break-down structure on scientific theory. Such theory is readily available in the plethora of scientific papers and articles published in scientific journals or presented at academic conferences. As we shall see, the use of scientific publications ensures that the models built reflect causal relations, i.e. handles the first problem in section 2. However, re-use of such previously published information also poses some challenges when it comes to the uncertainty problem. This is addressed in section and [Lagerström et al. 2007] presents a method focusing on knowledge elicitation from scientific texts, i.e. finding the appropriate measures for a certain analysis, see part 2 of Fig. 2. In the example case it means breaking down the goal enterprise systems modifiability by studying research on this particular topic. (1) Step one: select scientific text to use as a source for the goal break-down and identify which variable to break-down. Once the variable has been identified,

12 100 R Lagerström, U Franke, P Johnson, J Ullberg this is used as the variable under consideration in the first iteration of the process. (2) Step two: Read the scientific text and identify all occurrences of the variable under consideration where a causal or definitional relation to other variables is implied. Extract these occurrences into an evidence database, a piece of evidence could be a sentence, a figure, a table, or combinations thereof. (3) Step three: For each piece of evidence in the evidence database, identify if it is a causal relation or a definitional relation and between which variables this relation acts. Document this with an appropriate goal break-down syntax, e.g. extended influence diagrams as proposed in [Johnson et al. 2007]. (4) Step four: Use the variable fragments to identify variables related to the variable under consideration. Use each one of the identified variables as the new variable under consideration and iterate starting with step 2 in the process. Perform this iteration until no more variables related to the variable under consideration are found in the selected source. (5) Step five: Perform sanity check of the variable fragments. Before this step, the created fragments give a fairly good representation of what the text actually states. However, there is always a chance that this representation does not accurately reflect the semantics of the chosen scientific text. In order to reflect the semantics, the user of the process is allowed to make some interpretations, provided that these interpretations are carefully documented. During this step, relations and variables that are implied in the source may be introduced. Moreover, relations and variables that are explicitly stated in the text, but that make no sense may be removed. During this step the variables in the goal break-down fragments need to be tested regarding controllability and if these are well defined or not. This process is further elaborated in subsection The guidelines presented above can be slightly altered and for instance be used when interviewing experts instead of or as an alternative to written scientific sources Expanding goal break-down fragments In [Johnson et al. 2007] a process for managing extended influence diagrams, i.e. formalized causal goal break-down structures, is proposed cf. part 3 of Fig. 2. The presented process contains steps which together with the text elicitation guidelines provides a structured foundation when constructing a goal break-down fragment. The process presented aims to find variables for the goal break-down fragment which are well defined and controllable for the decision maker. This use of controllability constitutes an important way of ensuring appropriate causality for the action guiding aspect of the model, and of removing unnecessary correlations that do not possess this quality. (1) The first step is to determine whether a variable in the goal break-down struc-

13 A method for creating enterprise architecture metamodels applied to systems modifiability analysis 101 ture is lexically defined, stipulatively defined or undefined. A variable is lexically defined if we can assume that there is common agreement on its definition. Although almost all definitions can be challenged, concepts that would normally qualify as lexically defined include weight in kilograms, number of processors or number of users. For many concepts, however, such as information security, architecture quality and competence there is no universal definition even by very pragmatic standards. If a variable is not lexically defined, it needs to be stipulatively defined. As an example, the variable documentation quality might be defined by the two variables documentation availability and documentation accuracy. The stipulative definition of a variable results in the introduction of new variables into the goal break-down fragment. When new variables are introduced, this affects the conditional probability distributions of the child variables. These must therefore also be specified, thereby detailing the dependencies between the parents and the child. (2) For each defined variable, the second step considers whether the variable is uncontrollable. An uncontrollable variable cannot at all be affected by the decision maker; this means that potential variations in the value of the uncontrollable variable are unrelated to the choices of the decision maker. If the variable is uncontrollable, it will not provide decision supporting information, so there is no need to continue exploring this branch of the goal break-down fragment. The variable can therefore be deleted. (3) Step three queries whether the variable under consideration is semi-controllable. Semi-controllable variables are affected both by the decision maker s choices and other, uncontrollable, phenomena. If a variable is semi-controllable, it is important to separate those aspects which are controllable from those which are not. Therefore, new variables are introduced into the goal break-down structure with causal relations to the semi-controllable variable under consideration. As an example, we might believe that the semi-controllable variable mean time to repair is causally affected by both the maintainability of the system and flexibility of working hour regulations. Of these, the maintainability might be controllable, while the working hour regulations might be beyond the decision maker s domain of control. (4) In the final step, the process considers whether the variables are directly or indirectly controllable. A directly controllable phenomenon is an immediate consequence of the decision maker s choice. For instance, if a scenario is chosen where one system is replaced by another, this may directly entail that the CPU speed is increased. For variables that do not seem to be directly controllable, one or several new variables are introduced into the goal break-down structure, the focus shifts to the one of these, and the process returns to the beginning. When the whole process is finished, the result is a goal break-down structure where the variables can be controlled by the decision maker. No variables are left undefined, and uncontrollable variables have no causal parents.

14 102 R Lagerström, U Franke, P Johnson, J Ullberg By using the goal break-down approach, including the knowledge elicitation guidelines and the control process, a causal goal break-down fragment is produced. Such a fragment is based on scientific knowledge, and exhibits variables that are all causally linked to a well-defined goal, controllable by the decision maker. Fig. 3 presents a goal break-down fragment derived when decomposing the modifiability goal. Change Project Change Cost Project Cost Component Understandability Component Understandability Component Documentation Component Documentation Quality Quality Component Size Component Size Component Internal Component Coupling Internal Coupling Component Complexity Component Complexity Fig. 3. The outcome of the goal break-down method is a goal break-down fragment. This example fragment for modifiability is focusing on component understandability and variables causally related to this understandability Extracting metamodel elements The goal break-down fragment with its well defined and controllable variables can now be translated to a metamodel fragment, cf. Fig. 4. The set of variables and their relations have been identified in the goal breakdown fragment, e.g. as in Fig. 3. Thus it is time to translate these into metamodel classes, attributes, and reference slots. A metamodel class can either represent physical artifacts, such as computer and person, or more conceptual ones such as data flow or process, depending on what the fragments of the goal break-down states. The attributes of the metamodel classes correspond to the variables found in the goal break-down part of the method, as presented in previous subsections. Recall the goal break-down fragment presented in Fig. 3. This corresponds to the metamodel fragment presented in Fig. 5. The method steps described so far will result in a number of goal break-down fragments and accompanying metamodel fragments, all based on the selected source or set of sources. However, depending on the granularity of the sources, these fragments are usually very small and local models, i.e. models describing the relations between a few select elements in great detail, but without a sense of a bigger picture. Furthermore, the fragments are sometimes completely disjoint, sometimes completely overlapping, and usually somewhere in between all depending on the scope of the original sources. To make full use of the knowledge elicited, however, we need to merge the fragments into one metamodel. The challenge is to make sure that this metamodel remains coherent and non-ambiguous, despite its diverse

15 A method for creating enterprise architecture metamodels applied to systems modifiability analysis 103 Goal break-down fragment Goal Metamodel fragment Class -Attribute Class -Attribute -Attribute Class -Attribute -Attribute -Attribute Fig. 4. Translating the goal break-down to a metamodel. Change Project Component Documentation 0...* Supports -Cost -Quality * Is changed in 1 Component -Understandability -Size -Complexity -Internal coupling Fig. 5. The resulting metamodel fragment based on the modifiability goal break-down fragment example. origins. This integration challenge is addressed in [Lagerström et al. 2008], which gives some guidelines for merging. A metamodel fragment merge can be performed on several levels. First and foremost, classes can be merged. However, once classes have been merged, it is still an open question whether the attributes of the original classes are suitable for merging, or not. Therefore, the fragments at hand need to be examined in a number of dimensions. Some suitable criteria are those listed in Table 1. Class merge Similarity of names Similarity of sources Similarity of reference slots Similarity of attributes Attribute merge Similarity of names Similarity of sources Similarity of attribute relations Table 1. Criteria for merging of metamodel fragment classes and attributes.

16 104 R Lagerström, U Franke, P Johnson, J Ullberg A few words of explanation are called for. The matter of name similarity is rather straightforward. To make assessment easier, the criterion can be categorized into identical, similar, or dissimilar names. Concerning sources, the question is whether the concepts come from the same place. If the authors, the interviewees, the companies, the scientific journals etc. are the same, this indicates that the classes or attributes are suitable for merging. The similarity of reference slots can be assessed according to whether the classes or attributes under consideration have all, some, or none of their reference slots to others in common. Finally, the similarity of the attributes of a class can be used as an additional heuristic for assessing whether the classes are suitable for merging. This method can be implemented formally for machine execution, as described in [Lagerström et al. 2008], or used more informally for manual fragment integration. By using the guidelines for metamodel fragment merge, the metamodel depicted in Fig. 6 was finally obtained for modifiability analysis. Change project Is divided into -Cost Is divided into 14 Architect team -Expertise -Time on project Is a resource of Change organization -No of architects -No of developers Is a resource of Developer team -Expertise -Time on project Change difficulty -Change size Architectural change activities -Cost -Synchronization need Performs Technical changes to architecture Executes -Quality 3 Supports 1 16 Change management process -Maturity Architectural documentation 2 Is written according to Component documentation -Quality Is written according to 19 Supports 17 Executes Component change activities -Cost -Synchronization need Performs Technical changes to components -Change difficulty -Change size Works in System change environment -Quality of tools -Quality of infrastructure 7 Implemented in Systems Describes -Understandability -Internal coupling -Size -Complexity -External coupling 26 Describes Components -Understandability 27 -Internal coupling 28 -Size 29 -Complexity Is a part of -External coupling Implemented in Works in Component change environment -Quality of tools -Quality of infrastructure Communicates with Communicates with Fig. 6. The resulting modifiability metamodel after fragment merge Documenting metamodel definitions and scales So far, our attention has been directed to the identification and break-down of relevant measures to support our decision-support endeavor. However, settling for a set of theoretical a priori measures is only a first metamodeling step. Data collection often requires concerted efforts from many investigators, who need a clear common picture of the concepts at hand. Similarly, the literature is often less than clear

17 A method for creating enterprise architecture metamodels applied to systems modifiability analysis 105 on the exact nature of the concepts described or used in research. Both of these phenomena contribute to the importance of definitional uncertainty, as described in section 2. Definitional uncertainty must be at a minimum before precise analysis can be carried out. At a first glance, the problem of clear definitions seems shallow. Having based the classes and attributes upon the scientific literature, it might be assumed that the precise meaning of the concepts used would be clear. This is, however, not necessarily the case. For instance, the complexity measure is commonly defined in several different ways, such as: (1) The Halstead measure which is based on the numbers of operators and operands, distinct as well as total, found in the code [Grubb and Takang 2003]. (2) The number of lines of code (LoC), i.e. the software size, which is a crude estimate of complexity but nevertheless a possible choice. LoC is usually defined excluding comments and blank lines [Grubb and Takang 2003]. (3) Cyclomatic complexity which was introduced by McCabe and is based on graph theory. It measures the number of linearly independent paths through a source code [Grubb and Takang 2003]. Even if every individual source is clear about its usage of the terms, it is important to keep in mind that the metamodel constructed in the previous steps represent the merging of several theoretical fragments. Such a merging is necessary to make the best use of the available knowledge, but for this reason alone it is clear that the final setting of definitions and scales is a task that necessarily falls on the shoulders of the metamodeler. The metamodel creation method proposed here takes definitional uncertainty into consideration in several steps, as described in subsection and This subsection provides an example, based on the modifiability case, on how to document and describe the definitions found and used in the metamodel. The metamodel presented in Fig. 6 contains a number of classes with accompanying attributes, these are all described in [Lagerström et al. 2009a;c]. In this paper the organizational view of the metamodel is more thoroughly presented, cf. Fig. 7. The organizational view focus on presenting the classes; change organization, architect team, and developer team. This means that the aggregating classes of architects and developers with their accompanying attributes also are described, as well as their relation to the architectural and component change activities. The following paragraphs present the organizational view of the modifiability metamodel in text and Table 2 contains the selected metamodel definitions and scales. The change organization refers to the organization implementing the software system modifications, i.e., the parties involved in the project, such as the customer, the vendors, consultants etc. Attributes in the metamodel related to the change organization are number of architects and number of developers involved in the project, affecting one of the two types of activity costs each. [Grubb and Takang

18 106 R Lagerström, U Franke, P Johnson, J Ullberg Change organization Is a resource of 1 -No of architects -No of developers 1 Is a resource of Architect team -Expertise MEAN -Time on project MEAN Architect -Expertise -Change project experience -Language experience -System experience -Time on project MEAN Developer MEAN -Expertise -Change project experience -Language experience -System experience -Time on project MEAN MEAN Developer team -Expertise -Time on project Is a part of 1 * 1 * Is a part of Executes Architectural change activities Component change activities 1 1 -Cost -Synchronization need -Cost -Synchronization need Executes Fig. 7. The organizational view of the metamodel. 2003; Pigoski 1997] Architects are the people in the change project who design and modify the architecture of the enterprise systems. Developers, on the other hand, are the ones working on the source code of the different components. The architects and developers both have the attributes expertise and time on project related to them. Expertise is stipulatively defined in terms of change project experience, source code / design language experience, and system experience. Time on project refers to the amount of time a person spend in the project compared to other parallel work. The team expertise is affected by the amount of time the team members spend on the project and the team expertise affect the activities cost. [Grubb and Takang 2003; Pigoski 1997; Boehm 1981] Change projects can conceptually be divided into architectural change activities and component change activities. Architectural change activities are the activities concerning modifications on an architecture level, i.e. involving several systems or components, while component change activities concern modifications to a single component. Both types of change activities have the attributes cost and synchronization need, the former measured as number of man-hours and the latter as the amount of time spent on synchronization. The more systems, components, people involved, and the higher the coupling between them, the higher the need of synchronization among the different activities will be. [Boehm 1981; Grubb and Takang 2003] 4.2. Methods for creating the quantitative part of a metamodel The former subsection presented the method steps for creating the qualitative part of an EA metamodel for analysis, i.e. the classes, attributes, and reference slots, cf. Fig. 6 for the resulting modifiability metamodel. This subsection presents the next step of the method, creating the quantitative part of such a metamodel. That is, to define the conditional probability distributions (CPDs) related to each attribute, as