Synopsis: Title: Software Quality. Theme: Information Systems. Project Term: 9th semester, fall Project Group: sw907e13

Transcription

1 SOFTWARE QUAL I TY WHATCODEMETRI CSCANTELLUS

2

3 Title: Software Quality Theme: Information Systems Project Term: 9th semester, fall 2013 Project Group: sw907e13 Students: Kristian Kolding Foged-Ladefoged Kasper Møller Andersen Rasmus Hoppe Nesgaard Aaen Supervisor: John Stouby Persson Copies: 5 Pages: 43 Finished: December 27, 2013 Aalborg University Department of Computer Science Selma Lagerlöfs Vej Aalborg East Synopsis: This project studies how code quality is understood and communicated in software development. We look at various definitions of quality, culminating in the ISO/IEC standard, and use this as a basis for exploring code quality in the research and development community. Our investigation is twofold, with code quality in the research community explored through a literature review, and code quality in the development community explored through a review of available code analysis tools. Our results are assessed using the theories of quality we present, and we rely on the ISO/IEC standard for categorizing the different metrics and tools we review to gain a good overview of the field. Finally, we present a decision model that can aid developers choose the proper software analysis tools and code metrics according to their desired quality characteristics. The project concludes that code metrics and software analysis tools can aid developers in increasing the quality of their products with respect to certain quality characteristics. This report and its content is freely available, but publication (with source) may only be made by agreement with the authors.

4

5 Kristian Kolding Foged-Ladefoged Kasper Møller Andersen Rasmus Hoppe Nesgaard Aaen

6

7 Table of contents 1 Introduction Measuring Quality Quality Product Quality Model Measurement Our Goal With Quality Measurement Research Questions Literature Review on Code Metrics Search Strategy Code Metrics Conclusion on RQ Code Analysis Tools Search Strategy Quality in Code Analysis Tools Conclusion on RQ Communication of Quality Bringing it Together Conclusion on RQ Discussion The Problem Formulation Process The Problem Exploration Process Limitations Future Work Conclusion 42 7 Appendix 44 2

8 Table of contents 7.1 Quality Characteristics Model Quality characteristics Articles of the Code Metric Literature Review Tools of the Code Analysis Tools Search Tool Quality characteristics Top 100 Journals Decision Model Entrepreneurship Project

9 Chapter 1 Introduction The purpose of this project, as defined by our study regulation, is for us to gain an insight into the field of computer science, as well as an ability to communicate a contemporary research problem[94]. The problem of determining the quality of a piece of software has been topical for many years, but the software engineering community has yet to agree on what software quality is [87]. This speaks to the complexity of the subject, as there are a lot of different ways to look at quality. The applicability of each of them depends on the context; examples of these contexts could be the requirements of the customer, the budget of the project, or what the intended use of the software is. This makes it complex to say just what the level of quality for a given piece of software is, which is why we will seek to investigate this problem such that we can gain the desired insight and ability to communicate about this problem. Our focus in this project is to give developers a better understanding of the quality of the software they are creating, so they can make more informed decisions about it. This has led us to conduct a literature review on code metrics metrics that show to what degree code possesses given characteristics allowing us to see just what qualities can be derived from such measures. We have also conducted an investigation into code analysis tools to see how they define and communicate software quality. We have chosen to investigate the subject bottom up, starting from the code metrics and tools and seeing what results they can bring. This approach however carries the risk that we become too focused on measuring for the sake of measuring, if we do not identify metrics and tools with tangible benefits. By contrast, a top down approach, where we had started with tangible and beneficial software qualities and tried to explore ways of measuring them, could have mitigated this, but would carry the opposite risk, that we do not explore measurements that are viable for use in the industry 4

10 Chapter 1. Introduction at the code level. We specifically chose to work with code metrics because they are forms of static code analysis analysis that does not require the code to be executed which means they do not depend on the context of the code, and many of them can be automated. This makes the process of measuring the code fairly unobtrusive for developers, and also keeps the measurements simple and reliable. But while we want to find out exactly what qualities code metrics can actually expose for a piece of software, it is important to emphasize that we do not believe code metrics to be the end-all and be-all of quality measurement, nor that they are always relevant. There are many other kinds of measurements one can take, such as performance or usability measurements, which will expose very different qualities compared to code metrics, and depending on the context, they may also be far more relevant. Before we delve into our research questions, we will present some background information on quality and code measurement, including details on the ISO/IEC standard, which we will use for giving the metrics and tools context. Then we present our research questions, followed by, first our literature review, then our investigation into code analysis tools, and then we tie the two together. Finally, we discuss our findings and conclude on our results. 1.1 Measuring Quality Quality is a broad term that can be applied to many different things or aspects; from the abstract, such as ideas, to the concrete, such as a product. Measurement is an activity that can be applied in many different ways to many different things, from time, to velocity, and mass. This section explores the combination of the two, starting by looking at quality Quality Quality can be viewed in a variety of ways; we present three different takes on quality, starting with an overall view of quality and making it more software specific with each subsequent take. 5

11 Chapter 1. Introduction Quality according to Reeves and Bednar Reeves and Bednar look at quality throughout history, and how it has evolved to also accommodate services and not just products [72]. They identify four different measures of quality, each with distinct nuances, described below: 1. Excellence 2. Value 3. Conformance to Specifications 4. Meeting and/or Exceeding Expectations Excellence is about building or doing something to the highest of standards, regardless of price. Many luxury products fall into this category, e.g. a Rolls Royce car or an Armani suit. Value differs from excellence, by taking price into consideration. Here it is not in itself important to use high end materials, but rather to deliver a certain excellence at a certain price, such that consumers will get good value for their money, without necessarily paying a very high price. Quality as conformance to specifications can be seen as different things, as there are many different specifications that can be conformed to. These can be the requirements put forth by the customer, some international standard to adhere to, or maybe just an internal plan at a company to make different components of a product fit together. These specifications likely all have different forms and are elicited in different ways. Finally, meeting and/or exceeding expectations of a consumer puts the end user at the center of quality. This can be very beneficial, as the end user experience should arguably be a central part of many products and services. However, this is also a very complex definition, that can make quality difficult to quantify as consumer expectations shift. Quality according to Kitchenham and Pfleeger The previous section describes a general definition of quality, so this section explores a more software oriented approach. Such an approach is provided by Kitchenham and Pfleeger [44], where they try to adapt the views of product quality put forth by David Garvin [22] to software. Kitchenham and Pfleeger presents five views on quality, which are as follows: 1. Transcendental view 2. User view 3. Manufacturing view 6

12 Chapter 1. Introduction 4. Product view 5. Value-based view The transcendental view says that quality can only be recognized, and not defined. Kitchenham and Pfleeger equate this to user delight; the user will recognize quality software and be delighted by it. As this view does not define quality, but states that it can only be recognized, it connects with all the takes on quality presented by Reeves and Bednar. While it does not agree with any definition of quality, users can still be delighted by recognizing quality, regardless of whether the quality is that of excellence, value, etc. In the user view, quality is defined as fitness to purpose, and the authors suggest several different ways this can be measured for software. There are reliability and performance modeling, as these will usually be compared against expected functionality and usage patterns, and usability testing, when it measures how a user does at a relevant task. Quality in the manufacturing view refers to conformance to specifications. This is reminiscent of what was described before by Reeves and Bednar, and Kitchenham and Pfleeger give a very concise description of the view as an effort to avoid the costs associated with rework during development and after delivery [44, p. 15]. The product view shares similarities with the quality as excellence approach described earlier. Kitchenham and Pfleeger link this view in software to the various metrics that can be used to describe a piece of software, such as lines of code or line coverage in testing, as these metrics are used for measuring internal qualities of the software (qualities that the users are not faced with directly). And finally, the value-based view is also quite similar to the aforementioned value approach to quality. As Kitchenham and Pfleeger describe it, the best way to utilize this view is by asking how much the customer is willing to pay for something, and whether the customer thinks it represents good value. This view can pull the other views together, as they may pull in their different directions but still end up with resulting products or services that the customers are not willing to pay for. Quality according to ISO/IEC Finally, we look at the view on software quality that is presented in the ISO/IEC [37] standard, which is meant as a quality assessment tool that can be used in the industry. The ISO/IEC standard is a collaborative work by the International Organization for Standardization (ISO) and the International Electrotech- 7

13 Chapter 1. Introduction nical Commission (IEC), and concerns quality in software products and software-intensive computer systems. Specifically, it incorporates many of the perspectives on quality described by Reeves and Bednar, and Kitchenham and Pfleeger. The only perspectives not taken into account is that of value and the transcendental view presented by Kitchenham and Pfleeger. As such, high quality per ISO/IEC is essential for both business and personal software products as well as software-intensive computer systems, as it decreases potential negative consequences for developers, product leads, end-users, and shareholders, all referred to as stakeholders. An essential factor for stakeholders is comprehensive specification and evaluation of software quality, and it is possible to define the necessary and desired quality characteristics of a product based on stakeholders goals [37]. It is also important to measure and evaluate these characteristics by using the proper measurement methods. ISO/IEC provides three quality models for identifying relevant quality characteristics: 1. The quality in use model is used to characterize the impact a product has on the stakeholders. The model contains five characteristics, where each characteristic is divided into sub-characteristics, and it measures how well a finished system performs in its real use domain. The five top characteristics are: Effectiveness, efficiency, satisfaction, freedom and risk, and context coverage. 2. The product quality model is used to determine the internal quality of a product and trying to predict the quality of the final product based on that. The model provides eight characteristics, where each characteristic is divided into sub-characteristics. The eight characteristics are: Function suitability, performance efficiency, compatibility, usability, reliability, security, maintainability, and portability. 3. The data quality model provides six characteristics, which are divided into sub-characteristics. The characteristics aid database developers in evaluating the quality of data, reduce ambiguity, avoid redundancy, ease data maintenance, increase reliability, and improve database security. The characteristics of data quality are: Functionality, reliability, usability, efficiency, maintainability, and portability. General quality model For a system to be qualified as being of high quality per ISO/IEC 25010, it must fulfill the various needs of the stakeholders. These needs can be represented by one of the three ISO/IEC quality models, that categorize the different characteristics of software quality. It can be necessary to explore some of the characteristics in depth by dividing them into subcharacteristics, as this provides a more detailed overview of all the qualities 8

14 Chapter 1. Introduction which need to be fulfilled. Each characteristic or sub-characteristic results in a number of measurable properties, referred to as quality properties. To perform a comprehensive measurement of the characteristics or sub-characteristics, it is important to use a collection of quality properties that is adequate for the situation. This collection of quality properties contains measures for each characteristics or sub-characteristics, and these measures are combined to form a quality measure which describes the degree to which a system satisfies the stakeholders needs. Figure 1.1 illustrates the breakdown of characteristics into quality properties. Figure 1.1: This figure illustrates how the characteristics are divided into sub-characteristics and quality properties [37] Product Quality Model ISO/IEC provides a model, displayed in figure 1.2, which describes the targets for each of its quality models. As code metrics are inherently for describing internal qualities [44], this means they fall under the product quality model. For this reason, only this model will be described in further detail. Model Breakdown To discuss the product quality model, we introduce another ISO/IEC standard: the standard. ISO/IEC provides a set of system and software measures for internal and external quality [36]. The standard measures can be modified and new measures can be introduced to provide better measurement of a software system. If the standard measures or new 9

15 Chapter 1. Introduction Figure 1.2: This figure displays the target groups of each quality model[37]. measures are introduced into a project, it must be specified how these measures relate to the product quality model. ISO/IEC does not require that all eight product quality characteristics be used, and it is encouraged to only pick the characteristics that are suitable for a given software system. Internal quality measures are usually applied to source code during the development of a system. The internal quality measures provide the developers with information about the current state of quality for the source code; this makes it possible to identify some early implementation or design errors in the source code. Consequently, checking the source code quality can help the developers and stakeholders better predict the quality of the final product [37]. External quality measures are only used in conjunction with the testing stage. The software system is required to be executable, and the goal is to perform the external quality measurement in the system use domain. External quality identifies how well the system works in the intended use domain. We are mainly interested in the internal quality; however ISO/IEC recommends picking internal qualities that have a strong relationship with the desired external qualities to help predict the external quality values, in turn increasing the probabilities of correctly predicting the final product s quality, according to figure 1.3 [36]. Designing a model with a strong relationship between internal and external qualities is often difficult, and can 10

16 Chapter 1. Introduction also turn out to be ambiguous. Figure 1.3: How different qualities of a project are connected [37]. The predefined characteristics and sub-characteristics of the product quality model are displayed in figure 1.4 and a definition of each sub-characteristics are listed in appendix 7.1. ISO/IEC contains a list of internal quality metrics which cover all the characteristics and sub-characteristics of the model; however, not all of these metrics can be collected automatically. Figure 1.4: This figure displays the eight characteristics and their subcharacteristics of the product quality model; see appendix 7.1 for a bigger image [37] Measurement Measurement in software engineering is about understanding, controlling, and improving entities, where a software entity can be e.g. processes or 11

17 Chapter 1. Introduction resources according to Kitchenham et al. [45] That is, in order to best control and improve an entity, understanding and measuring it is a key part of gaining that understanding. The difficult part is to agree on what is important to measure and how to measure it. The IEEE Standard Glossary of Software Engineering defines a metric as A quantitative measure of the degree to which a system, component, or process possesses a given attribute [1, p. 47]. By interpreting systems, components, and processes as entities, we see that in software engineering, metrics allow us to better understand, control, and improve software entities. The definition above has however been criticized by Gray [25] for not accounting for elements of existing metrology theory. She argues that the definition of a metric does not cover the concepts of units, scales, and expression of uncertainty, and though Kitchenham et al. do not refer to this definition specifically, their work on validating software metrics state that a metric must include these concepts to be valid [45]. Ideally then, each of the metrics identified in the literature review would be verified as metrologically valid. Even though metric validity is also an important topic, the main focus of this project is to determine quality by code analysis, not the validation of metrics. Each metric is therefore assumed valid in accordance with the established metrology theory, and the usefulness of the metric is judged by the empirical evidence provided in the papers describing the metric Our Goal With Quality Measurement Our goal in this project is not to advocate that a given software project reaches some quality level as measured using code metrics; rather, the goal is to give feedback to developers about the software they are developing. However, Kitchenham and Pfleeger [44] rightfully point out that it is not possible to say that internal quality ensures external quality, though code metrics can still give useful information about a piece of software. For example where bugs are most likely to occur [47] or how, and where, a design might have been weakened over years of maintenance [90]. These are very much internal qualities that an end user may never experience, but they allow developers to better understand and reason about their software and the challenges and tasks they face with it. And even if internal quality does not ensure external quality, the ISO/IEC standard points out that the two are still related, as described in figure

18 Chapter 1. Introduction 1.2 Research Questions As described in the introduction of this chapter, does this project addresses the concern about qualities of a software product that can be measured using code metrics. Because many code metrics have desirable properties such as being viable for automation and being unambiguous, can they serve as tools for giving developers feedback on and a better understanding of the code they are writing, in a rapid and unobtrusive manner. This will allow the developers to better reason about their projects, and thus allow them to better control it. Our reasoning behind this is based both in the software engineering literature, which outs measurement at the heart of understanding, controlling, and improving processes and products [45], but also in the practice of reflective learning. Specifically, we use the work of Donald Schön as a basis for our reasoning. In his article Designing as reflective conversation with the materials of a design situation [76], he dissects the design process, and how designers learn about what they can actually create with the resources available to them. As software developers, we also design our products based on what we believe we can do, and that overall process of designing something is equally as valid for software developers as it is for architects (which Schön uses as examples). The process of design as outlined by Schön as consisting of move experiments. These are the parts of an iterative process, where the designer first sees something and evaluates it, then makes a move to improve it or explore his options for improving it, and then sees and evaluates the results of his move. This process allows the designer to get a better understanding of what it is he is designing and what he can do with it, but it is very important that the designer can also properly evaluate their designs to make the right moves. By providing developers with more feedback about the work they are doing, we wish to enable them to better evaluate their work, and thus gain a better understanding of what is possible for them to do. The analysis will be done based on the information presented in section 1.1. Different takes on what quality is have been presented, ending with the very software specialized ISO/IEC and standards. This made it clear, that while we will be measuring internal qualities of the software with code metrics. This makes it important for us to do research that is viable for practical use, and is also the reason we have chosen to not just investigate the literature, but also investigate code analysis tools that utilize code metrics. As a result, we address defined the following research questions (RQ): 1. How does the literature characterize and measure software quality 13

19 Chapter 1. Introduction using code metrics? 2. How do existing code analysis tools characterize and measure software quality using code metrics? 3. How can these measures of software quality support the communication of quality in software development? RQ1 will be answered with a literature review. This is described in chapter 2 which has a detailed description of what qualities the different code metrics have been found to expose. It also includes the search strategy of the review and more details on the code metrics deemed the most relevant. RQ2 will be answered in a similar fashion to RQ1, focusing on identifying code analysis tools that can be taken into use in the industry here and now. This is detailed in chapter 3, which contains more information on the search strategy behind the investigation and the results of the investigation into code analysis tools. As RQ1 and RQ2 deal with the literature and code analysis tools respectively, answering these two questions will allow RQ3 to be answered. This is done in chapter 4, which is then followed by the discussion in chapter 5 where the future implications of the work, reflection on the work and its strengths and weaknesses, as well as the validity of the results are discussed. The answer to RQ3 will provide a picture of how better feedback can be given to developers on the code they are writing. This should make it easier for developers to control their projects. 14

20 Chapter 2 Literature Review on Code Metrics This chapter is dedicated to answer the first research question: How does the literature characterize and measure software quality using code metrics? There are two aspects to the question: characterization of qualities and measurement of qualities; each based on code metrics. We conduct a literature review to identify the state of the art on code metrics, to discover a broad swathe of code metrics and the qualities they can be used to measure. They will then be characterize by describing each of them and their fit with the various characteristics of the ISO/IEC product quality model. Section 2.1 describes the search strategy for the literature review, and section 2.2 describes the results, with a conclusion to RQ1 in section Search Strategy Webster and Watson [96] describe a high quality review as one which is a complete review focusing on a concept. Complete reviews are defined as reviews that cover more than one research methodology, one journal, or one geographic location. To address this issue Webster and Watson recommend an approach, that focuses on structuring the review to make the research more complete. The first part of the recommended structure is identification of relevant and impacting articles, previously important articles, and new research based on those articles. To identify relevant and impacting articles, Webster and Watson suggests finding leading journals on the subject and search for rele- 15

21 Chapter 2. Literature Review on Code Metrics vant articles in these journals. Then the review should include a backward citation search based on the articles identified from the leading journals. Finally, the review should have a forward citation search based on the key articles found previously. The research question we wish to answer with our literature review is as follows: How does the literature characterize and measure software quality using code metrics? This suggests that we should find articles related to measurement of quality in software using code metrics. To search for scientific work, we used the search engine Web of Knowledge (previously Web of Science), as we find it best satisfies the requirements put forth by Webster and Watson [73] Web of Knowledge lets the user construct queries and limit the search to specific authors or publications to find specific scientific articles. With respect to the review approach suggested in Webster and Watson, the literature review search strategy has been performed as follows. We first constructed a search query that included all articles which describe code metrics, quality metrics, software metrics or attributes. ((cod* OR qualit* OR software) near/1 (metric* OR attribute*) AND software*) This query found 3,244 articles in total. To reduce the search to articles describing quality metrics in either software communication or software frameworks, we included the words communication and framework. ((cod* OR qualit* OR software) near/1 (metric* OR attribute*) AND software* AND (communicat* OR framework*)) The query returned 515 articles; to reduce the amount of material even more, the query was changed to only include all articles from the top 100 most influential journals in software engineering and information systems. The top 100 most influential journals can be seen in appendix 7.6, and were identified by using the impact factor tool created by Web of Knowledge at Reuters, sorted by the 5-year-impact attribute. By issuing the same query and excluding all less influential journals, the amount of identified articles was reduced to 85. During our first search (returning 3,244 articles) we discovered that many articles containing the words code metrics describe research concerned with measuring results of a study (typically only in the conclusion). Because of this, when we had narrowed our result to 85 articles, we went through the articles, looking at their title, abstract, and conclusion, to de- 16

22 Chapter 2. Literature Review on Code Metrics termine whether the given article is relevant to our project or not. We determined an article to be related to our research, if it contained a description of the use of code metrics or quality metrics and preferably in combination with a project s characteristics. We narrowed our results down to 59 articles from the 85 we identified through Web of Knowledge, by going through title, abstract and conclusions. As Webster and Watson describes, we made a backwards citation search to identify the important articles that our current findings cite. The five most cited articles were then selected as a part of the literature review. We did not do a forwards citation search, even though Webster and Watson suggests it, because many of the articles identified in the initial Web of Knowledge search are of recent date. Finally, during the reading of the articles, we found out that we had been overly optimistic with our last refinement of the articles, as many articles dealt with aspects of code metrics without going into detail with specific metrics. This led us to a final count of 23 relevant articles from our initial search, and 27 with the relevant articles from the backwards citation search. A shortened version of this process can be seen in table 2.1, and an overview of all the final articles can be found in appendix 7.3. Step Articles Description 1. Query (515) 2. Top Journals (85) 3. Manual reduction (59) 4. Backwards Citation (5) 5. Initial Result (64) 6. Final Result (27) Search query described above executed at www. webofknowledge.com October 18th Reduction by only including articles from top 100 journals. Narrowing results to articles that contain descriptions on the use of code metrics or quality metrics and preferably in combination with a project s characteristics. From the citations from the articles in step 3, find the most cited article not already included. Combining articles identified in step 3 with articles identified in step 4. Removing irrelevant articles from step 5 upon reading. Figure 2.1: Short description of the literature search. 17

23 Chapter 2. Literature Review on Code Metrics 2.2 Code Metrics We have found 27 articles that describe and utilize code metrics. Our original final search had a notably higher number of articles, but we had to remove many of them for not fitting our purpose once we actually read them. To best deal with this and the fact that not all articles contained exactly what we were looking for, but still felt relevant, we created a ranking system to judge the articles by the metrics they were describing. 1. Metrics that can be used in evaluating code quality, are measured directly on source code, and are viable for automation. 2. Metrics used to estimate overall or specific qualities of the code, and are not viable for automation. The articles also had to provide data to show the usefulness of their metric(s), though this was generally not an issue. To identify the code metrics described in the articles with ranking #1, each article that details its use of code metrics (how they utilize metrics, their reason for choosing these metrics) will be described. The identified metrics can be seen in table 2.2, which contains a description of the use of the metric. A short presentation of the #2 ranked articles comes last. Rank #1 Articles Metric Definition Desired value CBO Coupling between classes: Low Computes the number of couplings between two classes. A coupling exists when two classes share at least one of the following: Method calls, field access, inheritance, arguments, return types, object declaration, exceptions. CDBC Change dependency between classes: Low Computes the number of methods in a client class that depend on a server class. CTA Coupling through abstract data type: Low Computes the number of classes that are used for attribute declarations in a given class. CTM Coupling through message passing: Low Computes the number of messages passed from one class to another. 18

24 Chapter 2. Literature Review on Code Metrics Cyclomatic Computes the number of independent paths in a software program using a control flow graph.[54] The complexity is defined as: Cyclomatic value = E N +2P. N is the number of nodes in the control flow graph. P is the number of exit nodes. Low DAC Data abstract coupling Low Computes the number of abstract data types defined in a class. DIT Depth of inheritance tree: Low Computes the total number of superclasses for a given class using a tree structure LCOM Lack of cohesion of methods: High Measures whether methods in a given class actually belong to that class. It does so by looking at the number of pairs of methods in the class which share one or more instance variables, versus the number of pairs of methods that do not. LOC1 Lines of code: N/A Computes the number of lines of code. Implementation of this metric can vary depending on the line types which are counted. Some line types could be: Black lines, comment lines, executable lines, combination of comment and executable lines. LOC2 Lack of cohesion: Low Computes the number of methods in a class that are not sharing a common instance variable. NLM Number of local methods: Low Computes the number of locally declared non-inherited methods that are accessible outside the class. NMA Computes the number of non-inherited methods in a class Low NMO Computes the number of overriden inherited methods. Low NOC Number of children: Low Computes the number of direct descendants for a given class. NOM Number of methods: Low Computes the number of methods declared in a class. NPAVG Number of parameters on average: Low Computes the average number of parameters per noninherited method. RFC Response for class: Low Computes the total number of method declarations and method calls in a class. 19

25 Chapter 2. Literature Review on Code Metrics SIX Specialization Index: High Computes the product of NMO and DIT normalized by NOM. TCC Tight class cohesion: High Computes the number of directly connected methods in a class. Methods are connected if they share one or more instance variables. WMC Weighted methods per class: Low Computes the sum of each method complexity in a class, each method complexity is calculated by another complexity metric, such as the cyclomatic metric. Table 2.2: Description of the code metrics identified in the 27 articles. All the articles use at least one of the following four categories for metric categorization: Complexity, coupling, cohesion, and inheritance. Therefore we categorize the metrics according to their stated focus on the four categories, presented in table 2.3. Coupling metrics describe the dependencies among classes in an objectoriented system [19]. Low coupling often relates to good design and well structured software and supports high readability and maintainability. Cohesion metrics describe the degree to which methods and attributes of a class belong together [19]. High cohesion is often related to high reliability and reusability and low cohesion is a sign of low maintainability, reusability, and testability. Inheritance metrics describe the structure of the inheritance hierarchy [19]. High inheritance promotes reusability and maintainability. Complexity metrics describe interaction between entities of a software product [19]. Complexity is often related to low maintainability, modifiability, analysability, and testability The goal of the article Measuring software evolution at a nuclear fusion experiment site: a test case for the applicability of oo and reuse metrics in software characterization [50] is to measure how code architecture changes when a software product changes. For this purpose, the article uses a metric suite by Chidamber and Kemerer (referred to as C&K)[11]. The C&K metric suite is a collection of metrics focused on measuring the complexity of an object oriented software project. The C&K metric suite contains the following metrics: WMC, DIT, NOC, RFC, CBO, and LCOM, and as stated in the introduction, code metric descriptions can be found in table 2.2. The article does not consider other metrics than the C&K metric suite and yet the article only briefly explains that the C&K metric suite offers great metrics 20

26 Chapter 2. Literature Review on Code Metrics # Name Complexity Coupling Cohesion Inheritance a CBO [50, 38, 19, 67] [41] b CDBC [90] c CDE [90] d CTA [3] e CTM [3] f Cyclomatic [90, 19] g DAC [90] h DIT [19, 3, 41] [90] [50, 38, 90, 67] [3, 41] [19] i LCOM [50, 38, 90, 67] [19, 41] j LOC1 k LOC2 [90] l NLM m NMA [19] n NMO [19] o NOC [41] [90] [50, 38, 90, 67] [41, 19] p NOM [90] [90] q NPAVG [19] r RFC [50, 38, 90, 67] [90] [19, 41] s SIX [19] t TCC [90] u WMC [50, 90, 19, 67] [3, 41] Table 2.3: This table shows how the different metrics can be used for measuring complexity, coupling, cohesion, and inheritance. Note that LOC1 is lines of code and LOC2 is lack of cohesion. for measuring code organization. The article is however using a simplified version of WMC, where the complexity is the number of methods and therefore there is no reason to use an extra complexity metric. The article concludes that by using the mentioned metrics, the original class structure was maintained and new features kept in new classes that complement the old structure. The goal of the article Identification of defect-prone classes in telecommunication software systems using design metrics [38], is to study the relation between object-oriented design choices and defects in software systems. It 21

27 Chapter 2. Literature Review on Code Metrics compares the relationship in the early stages of the development process and the late stages of the development process. Again the C&K metrics suite is used, because it is relatively simple and well understood metrics suite. Janes et al. use the following metrics from C&K: WMC, DIT, NOC, RFC, CBO, and LCOM, as well as some metrics that is not part of the C&K metrics suite: LOC1, NOM. The article mentions a variety of other metrics but does not explain why or how they work, the metrics are never mentioned again in the article and are therefore not included in this review. The C&K metrics are as explained in table 2.2. NOM is used to calculate WMC, and LOC1 is used to verify the state of the development process, where a low LOC1 value indicates an early position in the development process and a high LOC1 value indicates a late position. The article concludes that the mentioned code metrics can be used to identify the most defect prone classes early in the development process. This finding can help project managers identify classes that require careful design, refactoring, and extensive testing. The goal of Improving design quality using meta-pattern transformations - a metric-based approach [90] is to improve the design quality of large object oriented systems. Thavildar et al. propose a framework whereby metrics can be applied to automatically identify some design flaws. They suggest the following metrics: WMC, NOM, RFC, LOC2, Cyclomatic, CDE, DAC, CDBC, TCC, LCOM, DIT, and NOC, again explained in table 2.2. Thavildar et al. believe these metrics will detect design flaws and will measure complexity, coupling, cohesion, and inheritance. The article mentions that RFC and NOM can be used to calculate WMC, but it does not specify which one they use. The article suggest a three step refactoring strategy that utilizes the mentioned metrics, described below, and the article concludes that, using the suggested metrics and refactoring strategy, it is possible to prevent loss of maintainability. 1. Detect classes with reduction in quality by measuring and recording code metrics. The article mentions that there are several reasons for quality reduction in a class but focuses on detecting classes with high complexity and coupling. 2. Refactoring the detected design flaws such that they satisfy the desired values of complexity, coupling, cohesion, and inheritance. 3. Measure and record the refactored classes with the same code metrics that were used in step one. It is then possible to compare the recorded metric results from step one and three. This comparison forms the basis for evaluating the new design improvements to check whether or not step two was successful. 22

28 Chapter 2. Literature Review on Code Metrics The confounding effect of class size on the validity of object-oriented metrics [19] examines if class size has an effect on object-oriented metrics. The article uses a combined subset of the C&K and L&K (metric suite by Lorenze and Kidd) metrics suites which consists of the following ten metrics: WMC, Cyclomatic, DIT, NOC, CBO, RFC, LCOM, NMO, NMA, SIX, and NPAVG. El Emam et al. believe these metrics adequately cover the metric categorizations; complexity, coupling, cohesion, and inheritance The article states that software with high cohesiveness, sparsely coupled classes, and low inheritance are less likely to contain faults. It concludes that the empirical investigation shows that class size has a notable effect on object-oriented metrics. The article An empirical validation of object-oriented metrics in two different iterative software processes [3] determines the effectiveness of objectoriented metrics in short-cycled and long-cycled iterative software development processes. It utilize the C&K metric suite because of its wide acceptance in the software engineering community and the capability to be measured automatically; the following metrics are used: WMC, DIT, LCOM, NLM, CTA, and CTM. The article does not describe how WMC is calculated nor which of the four categories, represented in table 2.3, that NLM measures. Alshayeb et al. concludes that object-oriented metrics have high effectiveness in short-cycled interactive processes for predicting evolutionary changes to system design. But they find that object-oriented metrics are largely ineffective in long-cycled interactive processes however. It is mentioned that object-oriented metrics in short-cycled interactive processes are only effective if the design accumulates to a certain critical mass, though critical mass is not elaborated in the article. Empirical evaluation of reuse sensitiveness of complexity metrics [41] proposes a revision of C&K metrics suite that can be applied to constructed software by reusing software components, such as class libraries. The hypothesis is that the C&K metrics suite treats reused and newly developed classes the same way and gives the same results for them. The article disagrees with C&K on the quality of reused components, because they tend to have a higher quality than newly developed ones. They also argue that there are different complexity factors for reused and newly developed classes. Kamiya et al. uses a C&K metrics suite that contains WMC, DIT, NOC, CBO, RFC, and LCOM. They use a simplified version of WMC such that all methods are equally complex, that way WMC only counts the total number of methods per class. They also propose a modification of DIT, NOC, CBO and RFC, which should be more suitable for evaluating software constructed from existing software components. Kamiya et al. choose to modify DIT, NOC, CBO, and RFC because these 23

29 Chapter 2. Literature Review on Code Metrics metrics are used to measure external complexity, which is the complexity between classes. They do not modify WMC nor LCOM because these metrics measure the internal quality, such as complexity or cohesion per class. Figure 2.2 describes the modifications Kamiya et al. has made to the metrics to improve measures on reused software components. DIT is seperated into DITN and DITR. DITN is the number of newly developed classes that is part of the inheritance tree. DITR is the number of reused classes that is part of the inheritance tree. NOC is seperated into NOCN and NOCR. NOCN is the number of newly developed direct descendant classes of a given class. NOCR is the number of reused classes direct descendant classes of a given class. CBO is seperated into CBON and CBOR. CBON is the number of couplings between a class and a newly developed class. CBOR is the number of of coupling between a class and a reused class. RFC is seperated into RFCN and RFCR. RFCN is the total number of method declarations and method calls in a newly developed class. RFCR is the total number of method declaration and method calls in a reused class Figure 2.2: Metric modifications by Kamiya et al. [41] The article does not explicitly state why these eight new metrics are better measurements for software constructed by reusing software components that the existing C&K metrics suite. They compare the number of faults and the effort to fix these faults of both the C&K metrics suite and the revised version of C&K metrics suite, based on this do they conclude that CBOR and RFCN are useful but the experimental project was too small to prove the hypothesis. The goal of Empirical analysis of software fault content and fault proneness using bayesian method [67] is to create a Bayesian network model which relates object-oriented metrics to software fault content and fault proneness. A Bayesian network is a graph which represents a set of data values and how the sets depend on each other. The metrics of the C&K suite are used, consisting of: WMC, DIT, RFC, NOC, CBO, LCOM, a simplified version of WMC, which only counts the number of methods per class, and LOC1 for measuring class sizes, is used. The C&K metrics suite is used because of it is widely adopted in industrial software development, such as NASA, whom is sponsoring the work of the article. Ganesh et al. successfully creates a Bayesian network which relates objectoriented metrics to software fault content and fault proneness. 24

30 Chapter 2. Literature Review on Code Metrics Rank #2 Articles An interesting #2 article is Using structural and textual information to capture feature coupling in object-oriented software [74] which seeks to measure coupling between features, in contrast with many other coupling measures which focuses on classes. They do this by looking at both the structure of the code and checking whether a method is related to a feature, but also if the documentation of the code (e.g. comments) points to two features being coupled. This leads them to create two metrics: Structural Feature Coupling (SFC) and Textual Feature Coupling (TFC). The authors do note that each of these metrics are only as strong as the structure or text they are respectively measuring, and thus they create a hybrid of the two (HFC), where SFC and TFC are combined with weights to allow for adjustment, though they get good results from a 50/50 split between the two as default. The authors successfully link their metrics to feature coupling, allowing for coupling analysis that crosses over many classes at once. However, these metrics also require significant work to be usable, as they require that code be linked to specific features, which is an arduous task that is to be done manually. The other articles found here generally fall into two categories: those that attempt to estimate properties of a project ahead of time, and those that attempt to evaluate the quality of the project with more than just code metrics. The articles focusing on estimating ahead of time are about trying to estimate project size to better estimate the effort it requires [27], estimating project size to know how much memory the code will require in embedded systems [49], and estimating the maintainability of communication protocols ahead of time [32]. The articles that seek to evaluate some aspects of a project seek to evaluate quality on either FLOSS projects [5], distributed development projects [42], or projects in general [97]. One article also seeks to evaluate reusability of assets in product line engineering, which focuses on putting together existing assets to form a product [29], while one attempts to estimate the effect of object-oriented code on productivity [59]. We wanted to look out for this category of articles when reading to find potential outliers that may still have been relevant for the main reading, but we did not find anything substantial in these articles. 2.3 Conclusion on RQ1 This chapter has sought to answer RQ1, which is as follows: How does the literature characterize and measure software quality using code metrics? The literature characterizes code metrics by four distinct characteristics: 25

31 Chapter 2. Literature Review on Code Metrics complexity, coupling, cohesion, and inheritance. Complexity describes the interaction between entities of a software product. Low complexity supports good maintainability, modifiability, analysability, and testability. Coupling describes the dependencies among the classes in an object-oriented system. Low coupling support good readability and maintainability. Cohesion describes the degree to which methods and attributes of a class belong together. High cohesion supports good reliability and reuseability. Inheritance describes the inheritance structure of a software product. High inheritance supports good reusability and maintainability. The examined metrics analyze code according to these widely accepted design patterns going for low coupling and high cohesion. The metrics do not provide any insight into whether or not the software complies with the system requirements however. By using metrics which cover all the four characteristics, metrics can however help developers be better aware of the maintainability, modifiability, analyzability, testability, readability, reliability, and reuseability of their software. According to figure 1.4, the metrics partly cover the maintainability characteristics and the overall characteristic of reliability. This however does not imply that early code analysis does not increase the quality but rather support better external quality, this relation can be seen at figure 1.3. Our literature review indicates that researchers tend to pick the C&K metrics suite. Even though the C&K metrics suite is popular, there are some disagreements about what the metrics actually measure. According to table 2.3, DIT, NOC, and RFC are used for multiple types of measurements, which may indicate that there are no clear guidelines on how to use the metrics nor what the metrics measure. And without clear guidelines, using these metrics only becomes more difficult. 26

32 Chapter 3 Code Analysis Tools This chapter is dedicated to answering RQ2: How do existing code analysis tools characterize and measure software quality using code metrics? Like RQ1, it contains two aspects: characterization of quality and measurement of quality, but from the perspective of code analysis tools rather than specific metrics. To see how measurements of quality are done using code analysis tools available to the industry today, we conduct a search similar to that of our literature review. The search is however done using a different search engine, leading to a number of differences that will be outlined in the dedicated section. Characterization of the qualities will be done using a simple scheme, that divides the tools into their stated goals based on their own description of purpose. As part of the discussion of the different tools, the ISO/IEC standard will also be used for characterization of the tools before we conclude. The goal of the code analysis review, is to determine how well the used tools are in compliance with the quality definitions described in section This is done, by identifying the most common code analysis tools and determine the definition of quality the tool is assessing. 3.1 Search Strategy To identify code analysis tools, we first tried executing a search strategy for scientific literature similar to the one described in section 2.1. But the resulting articles were either describing specific code analysis methods in combination or methods to do static code analysis. The search query used on was 27

33 Chapter 3. Code Analysis Tools (static near/5 ("program analy*" OR "code analy*") AND tool*) The articles and research found in this initial search was mostly concerned with the state of the art of the subject. There are therefore very few articles describing consumer related static code analysis tools; some explore the impacts of given tools, but they are few. Since we are interested in getting a broad range of available tools, we had to use a strategy other than searching for scientific articles. Staying true to Webster and Watson [96] we conducted the search by using the Google search engine, because it helps identify the most relevant websites [23, 9]. A problem with Google s indexing algorithm, is that it includes the user s previous interests in search results. Because of this, search queries will often return different results from user to user. This can be solved by using the search engine which allows anyone to do anonymous searches on multiple search engines [17]. Adding the keyword!g tells DuckDuckGo to only include results from Google; thus we modified the original search query slightly, to cope with the query syntax of Google:!g tool AND ("static program analysis" OR "static code analysis") As of december 2nd, Google returns about 182,000 results from the search query, so we have to limit our results. As described on Google s How it works page [23], all results are sorted by relevance, so we can expect the most relevant articles to be present first in the search. Keeping this in mind, we went through every link on the first two pages of results returned by Google, and found 31 tools which can be seen in table 7.4. We did not include results from websites that contain a lot of information that is not relevant for static software analysis. This includes websites like Wikipedia, and any tools described on these websites are not present unless they have been identified from another source in the first five pages of results from Google. This is because sites like Wikipedia, SourceForge, and others like them, are being deemed relevant by Google, because there exists a lot of links to and from these sites. A shortened version of this process can be seen in table

34 Chapter 3. Code Analysis Tools Short Description Google Search (182,000 websites) Search query described above (!g tool AND ( static program analysis OR static code analysis )) executed at Result Reduction (20 websites) Reduction by only including the 20 most relevant websites according to Google [23]. Excluding Websites (17 websites) Excluding all websites that focus on having a big variety of data (e.g. Wikipedia). Result (31 tools) Combining all tools identified on the 17 websites. Figure 3.1: Short description of the code analysis tools search. 3.2 Quality in Code Analysis Tools To get a general understanding of the code analysis tools, each of the identified tools have been categorized according to the quality model seen in figure 1.4. A given tool is characterized based on its goal and the code metrics it uses, as described on the website of the tool, with only characteristics that is seen as a direct purpose for the tool being included in the figure. This can of course be somewhat misleading, as the sites for each tool will likely be biased in the information it presents. Because of this, we choose three tools with different stated goals to go into a deeper analysis of. The complete list of tools and their characteristics can be found in figure 3.2 below and in appendix 7.4. Further analysis Three tools have been selected for further analysis; these tools have been chosen based on the amount of information available for them and their stated goals. Having the right information is important, as e.g. SQuORE is an interesting case, because they state that they determine quality based on ISO/IEC 25010, but it cannot be verified because there is no other information on their product. The three tools that will be analyzed in detail are NetBeans Code Inspector, HP Fortify, and Scrutinizer. NetBeans Code Inspector is interesting because it is built into the NetBeans IDE to be used directly by the developer during development. This makes for an interesting case as the NetBeans IDE is used by a large community of developers around the world [61]. Fortify is designed by Hewlett Packard to help software companies develop more secure software products. We believe Fortify is an interesting 29

35 Chapter 3. Code Analysis Tools Figure 3.2: This figure shows eight product quality characteristics with the matching tools, determined according to their descriptions; bigger image in appendix 7.2. case as it seems to have some popularity in the security community; e.g. it is award winning in the category of application security [84]. And finally, Scrutinizer is a tool that focuses on determining the state of a software project, from the source code alone. As opposed to Code Inspector, Fortify, and many of the other tools found, Scrutinizer focuses on presenting the result in a simplified and easy to read manner [83]. This makes for an interesting case; not only is it related to the work we have done here, as will be discussed in section 5.4, but the way the tool is designed to communicate quality means there is a separate quality aspect in the presentation of the results. It would be interesting to explore whether Scrutinizerci, the company behind the tool, has evaluated this aspect higher than the static code analysis itself in their tool. Code Inspector NetBeans describes Code Inspector as a tool for finding potential problems and detecting inconsistencies in your source code [60]. But it actually consists of two tools; a bug detection tool called FindBugs developed by a Ph.D student at the University of Maryland, and a tool called Java hints by Oracle and the NetBeans developers to detect common programming flaws. Both of these tools analyze Java code to find specific patterns that give rise to known or potential issues, and there is some crossover in their functionality. The biggest difference between the two is that FindBugs is more comprehensive and checks the bytecode for patterns, while the Java hints check 30

36 Chapter 3. Code Analysis Tools against the source code. Common issues these tools look for are issues like bad coding practices or code that is likely to fail; examples include use of deprecated APIs and constructs, or checking if some boolean expression always evaluates to false. The idea of quality in these two fits very well with the product quality model of the ISO/IEC standard, as can be seen in figure 1.4 of section 1.1. The performance and security characteristics are dealt with directly, as both tools recognize patterns for some known correctness, performance, and security issues. Compatibility, funcyional suitability, reliability, maintainability, and portability are dealt with in more implicit manners, as the tools make developers aware of patterns associated with bugs or bad practices. This leaves only usability as an aspect which these tools do not deal with at all, but despite this, they are still very much in accordance with the idea of quality that is put forth in the standard. Fortify Fortify is part of the security software HP Fortify Software Security Center. HP Fortify Software Security Center is a solution that Hewlett Packard provides for software companies, to identify and resolve security defects in software products [65]. The static code analyzer is usable during code development and notifies the developer of potential security concerned defects in the code [31]. The tool is based on knowledge gained by the researchers of the HP 2012 Cyber Risk Report, which is a report from Hewlett Packard that identifies the current state of software vulnerabilities [64]. The HP 2012 Cyber Risk Report is developed by multiple units in Hewlett Packard and open source communities [64, p. 3]. There is very little information about exactly what vulnerabilities and metrics Fortify is analyzing for. However, parts of the HP Cyber Risk Report is based on vulnerabilities detected through HP Fortify Software Security Center; thus we can assume that HP Fortify Software Security Center is able to at least detect some of the vulnerabilities described in the Cyber Risk Report. Fortify is concerned with unauthorized access to data and unauthorized manipulation of data, but their annual report on cyber risks also gives a clear picture of how much work they put into identifying new software threats and vulnerabilities. In comparison to the security quality attribute of the ISO standard in section 1.1.1, Fortify detects confidentiality and integrity issues to protect system information. But it does not seem like Fortify is concerned with non-repudiation, accountability, or authenticity, according to the information available on their website [64, 66, 65, 63, 31]. 31

37 Chapter 3. Code Analysis Tools In conclusion, the HP Fortify Software Security Center s is a security concerned code analysis tool, that detects flaws and vulnerabilities based on patterns. HP Fortify Software Security Center s view on high quality security seems similar to the ISO standard characterization of security quality, even though it is not itself as comprehensive as the ISO. Scrutinizer Scrutinizer supports PHP developers by combining a variety of open source PHP and JavaScript analysis tools and presenting their results in a simple overview. Scrutinizer is dependant on a lot of third party software analysis tools, listed on their website [81, 79], in combination with their own PHP Analyzer [80]. The PHP tools used are a broad range of various PHP analyzers, namely PHP Depend, PHP Mass Detector, PHP Analyzer, and PHP Code Sniffer. Scrutinizer is interesting because their software analysis depends on these many third party projects, their interpretation of quality, and how this quality is communicated to developers using it. Besides being focused on providing a simple presentation of the analysis results, Scrutinizer is covering a lot of different aspects of code analysis. The collection of tools included in Scrutinizer, informs the user of the functional suitability, reliability, maintainability, and security of a PHP project. This is accomplished by finding the test coverage (PHPUnit [6]), identifying common programming defects (PHP CodeSniffer [26], PHP Coding Standard Fixer [85], PHP Mass Detector [69], and PHP Depend [15]), testing processing time (PHP HHVM [20]), determine the functional suitability (PHP Analyzer [80]), and checking dependency security vulnerabilities (SensioLabs Security Checker [86]). This means that Scrutinizer can be used to get a very broad overview of the quality of a piece of software, by bringing together a variety of PHP analyzers to get a deeper knowledge of the reliability, maintainability, and to some extend security (because SensioLabs Security Checker only tells about dependency vulnerabilities) of a PHP project. 3.3 Conclusion on RQ2 The research question to be answered in this chapter was How do existing code analysis tools characterize and measure software quality using code metrics? Code analysis tools characterize and measure software quality, similarly to the metrics described in section 2.2, based on code metrics and design patterns to get an overview of a software project. A number of tools use the 32

38 Chapter 3. Code Analysis Tools same kind of code metrics as presented in chapter 2. The tools differentiate from the code metrics, by combining measurements with a pattern detection mechanism as well as the presentation of the results. By identifying defects and vulnerabilities by pattern detection, code analysis tools can span a wider range of quality characteristics. The tools identified have been categorized by the quality attribute and stated goal described on their websites, which can be seen in figure 3.2. The quality attributes covered by the code analysis tools identified are: functional suitability, performance efficiency, compatability, reliability, security, and maintainability. There are plenty of tools available, that can give an estimate of defects and of various static code analysis metrics on a software project. Most of the tools that were identified during this project are focused on identifying defects or common programming flaws and few tools describe the quality of a software project directly. The developer should be aware of the stated goal of the software analysis tool and assess if the focus of the stated goal is suitable for the specification of the project. Some tools, e.g. Scrutinizer [82], gives a score to the software project which is an estimate of the quality of the project, according to the stated goal of the tool. Other tools that use static code analysis to identify defects can be used to determine quality of a software project by comparing the result to the projects specification requirements. To conclude, software analysis tools can be used to determine the quality of a software project to some degree, if the tools are well matched with the project specification. 33

39 Chapter 4 Communication of Quality This chapter is dedicated to answer the last research question: How can these measures of software quality support the communication of quality in software development? To answer this, the results from research question 1 in chapter 2 and research question 2 in chapter 3 will be briefly summarized, and related back to the theory on quality we presented in chapter 1. This will help give a broader perspective on what they do show, what they do not show, and how they can help communication of quality in software development. 4.1 Bringing it Together In chapter 1, quality is described according to Reeves and Bednar s definitions [72]. It is further elaborated with the view provided by Kitchenham and Pfleeger [44], who adapt the product quality definition of David Garvin to the quality of software products. The software product quality is then elaborated further using the ISO/IEC standard, which divides quality into three quality models: quality in use, product quality, and data quality. Product quality was explained in detail based on the assumption that code analysis is product focused, leading to figure 1.4 describing the product quality model. Chapter 2 describes how the literature characterizes and measures software quality using code metrics. According to the literature, code metrics describe to what degree a piece of code possesses a given attribute [1, p. 47]. The literature roughly divides the code metrics into four categories based on what can be derived from the metric: complexity, coupling, cohesion, 34

40 Chapter 4. Communication of Quality and inheritance. These code metrics are connected to the product quality model seen in figure 1.4 mainly through the maintainability category, and bit through the reliability category. Chapter 3 describes how quality is characterized and measured through code analysis tools. A noteworthy difference compared to our code metric findings is the diversity in purpose, which is notably greater for the analysis tools. All of the metrics could be fitted fairly well within the eight properties of the maintainability characteristic of the ISO/IEC standard, but the tools span more characteristics, making them harder to categorize. The tools often use code metrics with code pattern detection to expose more information about the software product than a single code metric can achieve. The code analysis tools were thus characterized according to the entire product quality model of the ISO/IEC Though none of the tools were a complete match for the model, they did cover most of the characteristics of the model in combination. The code analysis tools covered the characteristics: functional suitability, performance efficiency, compatibility, reliability, security, and maintainability. This means that code metrics by themselves tell us primarily about the maintainability of a software project, with different metrics focusing on the different sub-characteristics of maintainability in the product quality model. The code analysis tools are able to take a broader perspective on the quality of software projects, by combining multiple code metrics with other detection mechanisms. Since the code analysis tools often measure a number of metrics and detect a number of code patterns, they can also go into more details with the the maintainability characteristic. These results make it clear that the tools and metrics we have found are a good fit for the type of quality described in the product quality model of the ISO/IEC standard. But if we look at the broader perspectives of Reeves and Bednar as well as Kitchenham and Pfleeger, the qualities of the tools and metrics are more open to interpretation. Tools looking at code correctness for example, fall in line with the quality as conformance to specifications, as they make it easier for developers to spot certain faults in the code. But having fewer faults can also be classified as a quality as excellence view, as faults are generally detrimental to overall excellence. As for the metrics and maintainability, an obvious fit is with the manufacturing view on quality, as per Kitchenham and Pfleeger describing it as an effort to avoid the costs associated with rework during development and after delivery [44, p. 15]. The value-based view is also very relevant here, as lower maintenance requirements also help keep costs down. Decreasing the number of faults and decreasing maintenance requirements can also be seen as ways of meeting and/or exceeding expectations, unless the customer did not have any expectations for this particular aspect. 35

41 Chapter 4. Communication of Quality 4.2 Conclusion on RQ3 Code metrics and code analysis tools can be used to determine the degree of certain quality characteristics in a software project, based on the quality characteristics defined in section When using code metrics or code analysis tools is it important to determine the quality characteristics being important for the specific software project, because this influences the choice of which metric(s) or tool(s) to use. By specifying the important quality characteristics of a software project, code metrics and tools can help determine the degree of quality of the given project. The process of selecting a code analysis tool or code metric can be defined as seen in figure 4.1 below. The code metrics have been referenced according to the corresponding letter at table 2.3, the code analysis tools have been referenced according to appendix 7.4. Figure 4.1: Decision process when determining code metric or code analysis tool for a project. Since code metrics and tools can help stakeholders determine the level of certain qualities in a software project automatically, it can also help track whether the project is improving or worsening and thus give the stakeholders information to help discuss the state of the project. Another solution is to let customers, users, or project owners determine a minimum degree of a certain quality characteristic, code metric, or code analysis tool, that a project should meet (e.g. have 0 vulnerabilities according to HP fortify). Doing so can help stakeholders determine whether the desired specification of the project is met or if the team should focus on improving the project. 36

42 Chapter 5 Discussion As described in chapter 1: The purpose of this project, as defined by our study regulation, is for us to gain an insight into and an ability to communicate a contemporary research problem in the field of computer science. [94] The problem of determining the quality of a piece of software has been topical for many years, but the software engineering community has yet to agree on what software quality is. [87] 5.1 The Problem Formulation Process The project did not start with the problem specified above however, and has changed notably since its inception at the beginning of the semester, as roughly illustrated in figure 5.1. Our initial ideas for a problem to investigate were about looking at how to utilize software testing as a tool for strengthening the development of software in a distributed manner (software development where the developers are not working from the same location). This was relevant as distributed software development (DSD) is here to stay, yet comes with a number of difficulties compared to collocated development [30, 75, 58, 46, 7, 52], so this seemed like a good problem for us to deal with. This approach had a number of challenges however; e.g. how can we determine new ways of using tests that are also useful, and how do they relate to distributed software development. These challenges made us realize that our problem needed refining. From here we went through several iterations of the project where we tried to find a good connection between testing and distributed software development. We looked at how test driven development might impact DSD, and later we experimented with creating a contingency model between dif- 37

43 Chapter 5. Discussion ferent testing methods and testing risks in DSD [51]. We had a hard time finding a concrete and manageable problem however. Going through these different ways of looking at testing in DSD also made us realize that there was a more fundamental problem of how you determine what to test; developers want to deliver high quality software, so what kind of tests will ultimately add the most to a given project, and for who? This led us focus more on what software quality is and how it can be quantified and communicated. We initially kept it in the context of distributed development, as one the big challenges with DSD is the communication between distributed developers; looking at how to quantify and communicate the quality of a piece of code should thus alleviate some misunderstandings as the communication becomes less ambiguous. The DSD context was however dropped when it became clear that it was not actually adding to the project, but had remained purely as a setup for a problem that did not really need one. And thus we arrived at the problem we have built this report on. Figure 5.1: A rough outline of the process we went through to arrive at our final problem. 5.2 The Problem Exploration Process In section 1.1.4, we specified that our goal with quality measurement was to give feedback to developers about the work they are doing, along with a few concrete examples of why this is useful. But one could imagine other benefits from creating better feedback mechanisms for developers too. For example, using code metrics as a basis of communicating quality also allows for some standardization of communication to take place. That is, if code metrics prove viable for aiding the communication of software quality, then having them spread across the community will allow us to communicate more clearly with less ambiguity, because they provide an unambiguous foundation to discuss from. And by providing developers with more feedback on the work they are doing, it would stand to reason that they can deliver their products faster than without the feedback, as they will better know where they need to put their effort. There is however a risk in trusting this too much if the level of feedback given across the project is not the same as that provided 38

44 Chapter 5. Discussion by the code metrics. The perceived product state could be incorrect if the code metrics show that one area needs a lot of attention, though other areas actually need more, but the feedback for that area does not adequately illustrate this. A third example of potential benefits from this feedback could be a learning effect. That is, if developers can quickly see when something they are creating is good or bad, they may better be able to learn what constitutes high quality code versus low quality code. This will only apply to quality that is directly measurable, and as such this is unlikely to ever be a surefire way to teach someone how to write overall high quality code. But e.g. with metrics that can tell which parts of a piece of software are more prone to faults [47], developers getting rapid feedback on this could potentially improve their intuition for what code is risky to develop. Our choice of using the ISO/IEC standard as a model for categorizing the metrics was based on the assessment of the standard in section 1.1.1, but there are a number of things to be mindful of when trying to make data fit into an existing model. If there is a discrepancy between the model and the data, it is important not to twist the data or overly simplify it to try to make it fit the model for example. This is a risk when taking state of the art data, such as the focus of this project, and try to make it fit an older model. However, as the standard is fairly broad and was last revised in 2011 (though the literature the revision is based will be older than that), we believe it is adequate for our purpose. 5.3 Limitations There are a number of limitations that this project has been created under. For one, we are not very experienced with navigating in the state of the art in an academic field, which introduces risk that we may miss important papers in our search. This is always a risk when doing literature reviews of course, though the risk is exacerbated by our limited existing knowledge of the field. Our search has however been iterated upon multiple times, so we are confident that we have found a representative selection of the current state of the art of code metrics. And as previously noted, we used a notable amount of project time on finding and specifying our problem while doing a type of project that is new to us (the literature review). This has been a challenge and has made it difficult to know what needed to be changed at different times, adding some confusion and chaos to the process. We believe we have managed to mitigate this risk, but we would undoubtedly have been able to create a more comprehensive project if we had not done both of these. 39

45 Chapter 5. Discussion These risks have also resulted in some issues that we did not discover early enough to mitigate. For example, though we deal with software quality, the Software Quality Journal is not included in our literature review. Though we have on purpose limited ourselves to the top 100 journals, according to the impact measure on Web of Knowledge, a journal such as this may have been worth investigating regardless. With the code analysis tools, we have a few additional risks. The first is with our choice of search engine; with our search, we used Google (through DuckDuckGo to anonymize the search), a search engine built to find any kind of information available on the Internet. Google ranks the different web pages according to relevance, which is based on more than 200 different factors [23]. Aside from the fact that Google keep many of them secret, it is also clear that it makes it difficult to say which tools will surface in our results, and it is possible that some important solutions did not make the cut. Since it is almost impossible to find and categorize all existing code analysis tools, we prioritized identifying a more limited grouping of the most relevant tools that could be used as a reliable target group. The second risk is from the fact that the information we have on these tools comes mainly from the creators own websites; sites which are often made to sell the tool. This means that the data that is made available may have been picked because it is more likely to sell the product, thus introducing a bias in the data. Some of tools are however open source, and could thus be checked if we had wanted to be more thorough in our analysis of them, though our resources did not permit this. 5.4 Future Work While we have gathered an overview of the state of the art of code metrics, there is still much research to be done. One potential avenue for future research to go into is whether code metrics can expose information about something other than the quality and state of the source code. An example could be to look at the number of iterations that a piece of code goes through throughout its life cycle, which might help expose the strength of the design process behind the code. That is a fairly simple example, but one could imagine that other information could be exposed through some combination of code metrics, like how source code size and work time can show a form of productivity. It would also be good to establish whether the code metrics used in exposing a given quality are necessary. This would help in minimizing unnecessary measurements, freeing up resources that can then be used else- 40

46 Chapter 5. Discussion where. One could however imagine that there were other reasons to use a specific metric; if a specific metric is used to measure quality of some kind, then it would seem natural that the developers would seek to develop code that scores well according to the metric. The question is whether a metric that is not part of the necessary set of metrics could still benefit the developers through behavioral changes that are not reflected in the final quality of the product. Previously in this chapter, we discussed how we chose the problem of the project, summarized in figure 5.1. The subject of software quality was derived from a problem we started seeing in distributed software development, and it could be interesting to go back to it now that we have a more firm grasp on the fundamentals. This could include looking at how the feedback given by various code metrics could help alleviate communication issues in DSD, e.g. internally amongst developers, but also in the communication between developers and managers. Doing more practical investigations of communications would also be interesting. A potential way to go about this is to look at specific communications in a development team, such as how using specific tools and metrics can affect the communication between developers and testers. With all our code metrics generally looking at how maintainable code is, the information exposed by these metrics could also help the testing effort, through e.g. looking at the coupling of code and seeing which pieces of code are thus more likely to share bugs [74]. A different kind of future work could be to see how the work performed in this project could be turned into an actual tool for measuring software quality. This was a task for us during one of our courses for the semester, which challenged us to build a business model around our project. This resulted in a business that is designed around having software developers pay for our tools, which will allow them to communicate the level of quality of their software to their customers, such that both parties can understand what the quality is, even though the customer is not an expert in the field of software engineering. More details on the business plan can be found in appendix 7.8 (though it is only available in danish). As for how to design and build the actual tool, after we built our business model, we discovered the firm Scrutinizer, with a near identical business model. This validates our idea of how we can progress with the tool, though it will be important to differentiate ourselves from them of course. 41

47 Chapter 6 Conclusion In this project, our study regulation required that we gain insight into the field of computer science and gain the ability to communicate a contemporary research problem [94]. As we showed in the introduction in chapter 1, the question of what constitutes quality software is both topical and unanswered, which is why we decided to investigate the subject. We narrowed the problem down to focusing on how developers can get better feedback on the software they are developing using both code metrics, but also existing code analysis tools. Our investigation on code metrics was done with a literature review on the state of the art within the field of code measurement. This showed that most code metrics that are viable for automation, expose information on quality related to the maintainability of software according to the ISO/IEC standard, with each metric falling broadly in one of the following four categories of measurement: coupling, cohesion, inheritance, and complexity. Some metrics also exposed information relevant for the reliability of the given software. The investigation we performed into existing code analysis tools was based on similar principles as our literature review, but was performed on tools rather than academic literature. This showed that the tools span a much broader selection of qualities than the code metrics, as they cover almost the entire product quality model of the ISO/IEC standard. When looking at our results from a broader perspective of software quality (as given in section 1.1), we see that there are multiple ways of interpreting the understanding of quality these metrics and tools are built on. They are very clearly concerned with the internal quality of a product and its overall excellence, but the focus on maintainability seen in many code 42

48 Chapter 6. Conclusion metrics can also help drive down costs for the customers, thus providing better value. And depending on the project that is undertaken, the customer can have expectations for cost (development and maintenance), faults, security, performance, and other areas where these tools and metrics can make a difference in development. This makes it important to keep the customers expectations in mind when looking at quality as well. In conclusion, we do not believe there is a single definition of quality that will be applicable to every software project; what constitutes quality is far too individual for this, and will depend on the context. Based on our research, we do however believe that using code metrics and code analysis tools to give feedback to developers is beneficial, as it will allow them to better know their product and thus what kind of focus will best give them the quality they seek to deliver. 43

49 Chapter 7 Appendix 7.1 Quality Characteristics Model The figure is intentionally placed on the next page, to make room for a bigger image. 44

50 Chapter 7. Appendix Figure 7.1: The eight characteristics and their sub-characteristics of the product quality model. 45

51 Chapter 7. Appendix 7.2 Quality characteristics [37] characteristics Functional suitability Functional completeness Functional correctness Functional appropriateness Performance efficiency Time behavior uti- Resource lization Capacity Compatability Co-existence Interoperability Usability Appropriateness recognizability Definition Degree to which a product or system provides functions that meet stated and implied needs when used under specified conditions Degree to which the set of functions covers all the specified tasks and user objectives Degree to which a product or system provides the correct results with the needed degree of precision Degree to which the functions facilitate the accomplishment of specified tasks and objectives Performance relative to the amount of resources used under stated conditions Degree to which the response and processing times and throughput rates of a product or system, when performing its functions, meet requirements Degree to which the amounts and types of resources used by a product or system, when performing its functions, meet requirements Degree to which the maximum limits of a product or system parameter meet requirements Degree to which a product, system or component can exchange information with other products, systems or components, and/or perform its required functions, while sharing the same hardware or software environment Degree to which a product can perform its required functions efficiently while sharing a common environment and resources with other products, without detrimental impact on any other product Degree to which two or more systems, products or components can exchange information and use the information that has been exchanged Degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use Degree to which users can recognize whether a product or system is appropriate for their needs 46

52 Chapter 7. Appendix Learnability Operability User error protection User interface aesthetics Accessibility Reliability Maturity Availability Fault tolerance Recoverability Security Confidentiality Integrity Nonrepudiation Accountability degree to which a product or system can be used by specified users to achieve specified goals of learning to use the product or system with effectiveness, efficiency, freedom from risk and satisfaction in a specified context of use Degree to which a product or system has attributes that make it easy to operate and control Degree to which a system protects users against making errors Degree to which a user interface enables pleasing and satisfying interaction for the user Degree to which a product or system can be used by people with the widest range of characteristics and capabilities to achieve a specified goal in a specified context of use Degree to which a system, product or component performs specified functions under specified conditions for a specified period of time Degree to which a system, product or component meets needs for reliability under normal operation Degree to which a system, product or component is operational and accessible when required for use Degree to which a system, product or component operates as intended despite the presence of hardware or software faults Degree to which, in the event of an interruption or a failure, a product or system can recover the data directly affected and re-establish the desired state of the system Degree to which a product or system protects information and data so that persons or other products or systems have the degree of data access appropriate to their types and levels of authorization Degree to which a product or system ensures that data are accessible only to those authorized to have access Degree to which a system, product or component prevents unauthorized access to, or modification of, computer programs or data Degree to which actions or events can be proven to have taken place, so that the events or actions cannot be repudiated later Degree to which the actions of an entity can be traced uniquely to the entity 47

53 Chapter 7. Appendix Authenticity Degree to which the identity of a subject or resource can be proved to be the one claimed Maintainability Degree of effectiveness and efficiency with which a product or system can be modified by the intended maintainers Modularity Degree to which a system or computer program is composed of discrete components such that a change to one component has minimal impact on other components Reusability Degree to which an asset can be used in more than one system, or in building other assets analyzability Degree of effectiveness and efficiency with which it is possible to assess the impact on a product or system of an intended change to one or more of its parts, or to diagnose a product for deficiencies or causes of failures, or to identify parts to be modified Modifiability Degree to which a product or system can be effectively and efficiently modified without introducing defects or degrading existing product quality Testability Degree of effectiveness and efficiency with which test criteria can be established for a system, product or component and tests can be performed to determine whether those criteria have been met Portability Degree of effectiveness and efficiency with which a system, product or component can be transferred from one hardware, software or other operational or usage environment to another Adaptability Degree to which a product or system can effectively and efficiently be adapted for different or evolving hardware, software or other operational or usage environments Installability Degree of effectiveness and efficiency with which a product or system can be successfully installed and/or uninstalled in a specified environment Replaceability Degree to which a product can replace another specified software product for the same purpose in the same environment Table 7.1: This table contains all the definitions of the sub-characteristics from the product quality model 48

54 Chapter 7. Appendix 7.3 Articles of the Code Metric Literature Review Article Journal Year Authors Software describing attributes Computer Standards and Interfaces (1) Empirical evaluation of reuse Elsevier - Information sensitiveness of complexity and Software Technology metrics (2) Measuring software evolution at a nuclear fusion experiment site: a test case for the applicability of OO and reuse metrics in software characterization Fault-Threshold Prediction with Linear Programming Methodologies A Metrics Suite for Object Oriented Design An empirical validation of object-oriented metrics in two different iterative software processes Benchmarking classification models for software defect prediction - A proposed framework and novel findings Empirical analysis of software fault content and fault proneness using Bayesian methods The confounding effect of class size on the validity of object-oriented metrics A Complexity Measure A Unified Framework for Coupling Measurement in Object-Oriented Systems Elsevier - Information and Software Technology Empirical Software Engineering (2) IEEE Transactions on Software Engineering (10) IEEE Transactions on Software Engineering IEEE Transactions on Software Engineering IEEE Transactions on Software Engineering IEEE Transactions on Software Engineering IEEE Transactions on Software Engineering IEEE Transactions on Software Engineering 2008 B Jereb 1998 T Kamiya, S Kusumoto, K Inoue, Y Mohri 2002 G Manduchi, C Taliercio 2003 M Pighin, V Podgorelec, P Kokol 1994 S R Chidamber, C F Kemerer 2003 M Alshayeb, W Li 2008 S Lessmann, B Baesens, C Mues, S Pietsch 2007 G J Pai, J B Dugan 2001 K El Emam, S Benlarbi, N Goel, S N Rai 1976 T J McCabe 1999 L C Briand, J W Daly, J K Wüst 49

55 Chapter 7. Appendix A new generalized software complexity metric for distributed programs A fuzzy classifier approach to estimating software quality Identification of defect-prone classes in telecommunication software systems using design metrics Software architecture graphs as complex networks - A novel partitioning scheme to measure stability and evolution Improving design quality using meta-pattern transformations - a metric-based approach The application of product measures in direct software maintenance activity Object-Oriented Metrics that Predict Maintainability Information and Software Technology (5) Information (3) Sciences 1998 W J Tsaur, S J Horng 2013 B A Kitchenham, R T Hughes, S G Linkman Information Sciences 2006 A Janes, M Scotto, W Pedrycz, B Russo, M Stefanovic, G Succi Information Sciences 2007 S Jenkins, S R Kirk Journal of Software Maintenance and Evolution (2) Journal of Software Maintenance and Evolution Journal of Systems and Software (1) 2004 L Tahvildari, K Kontogiannis 2007 M P Ware, F G Wilkie and M Shapcott 1993 W Li, S Henry Table 7.2: Articles found valid for the literature review with ranking #1. Article Journal Year Authors Using structural and textual information to capture feature coupling in object-oriented software The Effect of Object-Oriented Frameworks on Developer Productivity A Practical Approach to Size Estimation of Embedded Software Components A vector-based approach to software size measurement and effort estimation Empirical Software Engineering 2011 M Revelle, M Gethers, D Poshyvanyk IEEE Computer (1) 1996 S Moserl, O Nierstrasz IEEE Transactions on Software Engineering IEEE Transactions on Software Engineering 2012 K Lind, R Heldal 2001 T E Hastings, A S M Sajeev 50

56 Chapter 7. Appendix Measuring the Maintainability of a Communication Protocol Based on Its Formal Specifications A framework for evaluating reusability of core asset in product line engineering Determinants of software quality in offshore development - An empirical study of an Indian vendor Quality evaluation of floss projects Application to ERP systems The software evaluation framework SEF extended IEEE Transactions on Software Engineering Information and Software Technology Information and Software Technology Information and Software Technology Information and Software Technology 2003 S J Huang, R Lai 2007 J S Her, J H Kim, S H Oh, S Y Rhew, S D Kim 2011 G Kannabiran, K Sankaran 2013 L Aversano, M Tortorella 2004 B Wong Table 7.3: Articles found valid for the literature review with ranking #2. 51

57 Chapter 7. Appendix 7.4 Tools of the Code Analysis Tools Search # Name Stated Goal 1 Sotoarc [28] Architecture 2 Resource Miner [57] Architecture and Dependencies 3 Netbeans Code Inspector [60] Common Programming Flaws 4 CodePeer [2] Common Programming Flaws 5 Understand [77] Common Programming Flaws 6 XCode [4] Common Programming Flaws 7 Parasoft [68] Defect Detection 8 FxCop [55] Improvements 9 SQuORE [92] Quality of ISO Protecode [71] Open Source Licenses Check 11 ConQAT [14] Quality 12 Optimyth [62] Quality 13 Scrutinizer [82] Quality 14 SonarQube [88] Quality 15 Polyspace [53] Runtime Errors 16 Armorize [91] Security 17 Coverty [13] Security 18 Diggity [21] Security 19 Fortify [31] Security 20 IBM App Scan [33] Security 21 Veracode [35] Security 22 Checkmarx [10] Security and Bugs 23 Clang [12] Security and Bugs 24 DevBug [16] Security and Bugs 25 Flawfinder [18] Security and Bugs 26 GrammaTech [24] Security and Bugs 27 Prefast [56] Security and Bugs 28 Splint [89] Security and Bugs 29 Klocwork [34] Security and Reliability 30 Yasca [78] Variety Table 7.4: Table of the code analysis tools with their stated goal that we have identified. 7.5 Tool Quality characteristics The figure is intentionally placed on the next page, to make room for a bigger image. 52

58 Chapter 7. Appendix Figure 7.2: The tools identified categorised in quality characteristics. 53

59 Chapter 7. Appendix 7.6 Top 100 Journals Journals have been sorted in alfabetical order. ID Journal Title 1 ACM Transactions on Applied Perception 2 ACM Transactions on Autonomous and Adaptive Systems 3 ACM Transactions on Computer-Human Interaction 4 ACM TRANSACTIONS ON DATABASE SYS- TEMS 5 ACM Transactions on Embedded Computing Systems 6 ACM TRANSACTIONS ON GRAPHICS 7 ACM TRANSACTIONS ON INFORMATION SYSTEMS 8 ACM Transactions on Information and System Security 9 ACM Transactions on Internet Technology 10 ACM TRANSACTIONS ON MATHEMATICAL SOFTWARE 11 ACM Transactions on Multimedia Computing Communications and Applications 12 ACM Transactions on Sensor Networks 13 ACM TRANSACTIONS ON SOFTWARE ENGI- NEERING AND METHODOLOGY 14 ACM Transactions on the Web 15 Ad Hoc Networks 16 ADVANCES IN ENGINEERING SOFTWARE 17 ANNUAL REVIEW OF INFORMATION SCI- ENCE AND TECHNOLOGY 18 COMMUNICATIONS OF THE ACM 19 COMPUTER-AIDED DESIGN 20 COMPUTER COMMUNICATIONS 21 COMPUTER COMMUNICATION REVIEW 22 COMPUTER GRAPHICS FORUM 23 Computer Networks 24 COMPUTERS & SECURITY 25 COMPUTER STANDARDS & INTERFACES 26 COMPUTER 27 DATA & KNOWLEDGE ENGINEERING 54

60 Chapter 7. Appendix 28 DATA MINING AND KNOWLEDGE DISCOV- ERY 29 DECISION SUPPORT SYSTEMS 30 Electronic Commerce Research and Applications 31 EMPIRICAL SOFTWARE ENGINEERING 32 Enterprise Information Systems 33 EUROPEAN JOURNAL OF INFORMATION SYSTEMS 34 GEOINFORMATICA 35 GRAPHICAL MODELS 36 IBM JOURNAL OF RESEARCH AND DEVEL- OPMENT 37 IEEE Communications Surveys and Tutorials 38 IEEE COMPUTER GRAPHICS AND APPLICA- TIONS 39 IEEE INTERNET COMPUTING 40 IEEE MICRO 41 IEEE MULTIMEDIA 42 IEEE NETWORK 43 IEEE PERVASIVE COMPUTING 44 IEEE SOFTWARE 45 IEEE Systems Journal 46 IEEE Transactions on Computational Intelligence and AI in Games 47 IEEE Transactions on Dependable and Secure Computing 48 IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN BIOMEDICINE 49 IEEE TRANSACTIONS ON INFORMATION THEORY 50 IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 51 IEEE TRANSACTIONS ON MOBILE COMPUT- ING 52 IEEE TRANSACTIONS ON MULTIMEDIA 53 IEEE TRANSACTIONS ON RELIABILITY 54 IEEE TRANSACTIONS ON SOFTWARE ENGI- NEERING 55 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 56 IEEE WIRELESS COMMUNICATIONS 57 IMAGE AND VISION COMPUTING 55

61 Chapter 7. Appendix 58 INFORMATION & MANAGEMENT 59 INFORMATION PROCESSING & MANAGE- MENT 60 INFORMATION SCIENCES 61 INFORMATION AND SOFTWARE TECHNOL- OGY 62 INFORMATION SYSTEMS 63 INFORMATION SYSTEMS FRONTIERS 64 INTERNATIONAL JOURNAL OF ELEC- TRONIC COMMERCE 65 INTERNATIONAL JOURNAL OF GEO- GRAPHICAL INFORMATION SCIENCE 66 INTERNATIONAL JOURNAL OF MEDICAL INFORMATICS 67 International Journal on Semantic Web and Information Systems 68 Internet Research 69 JOURNAL OF THE ACM 70 JOURNAL OF THE AMERICAN MEDICAL IN- FORMATICS ASSOCIATION 71 JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOL- OGY 72 Journal of Ambient Intelligence and Smart Environments 73 Journal of the Association for Information Systems 74 Journal of Chemical Information and Modeling 75 Journal of Cheminformatics 76 JOURNAL OF INFORMATION SCIENCE 77 JOURNAL OF INFORMATION TECHNOLOGY 78 JOURNAL OF MANAGEMENT INFORMA- TION SYSTEMS 79 JOURNAL OF MATHEMATICAL IMAGING AND VISION 80 JOURNAL OF NETWORK AND COMPUTER APPLICATIONS 81 Journal of Optical Communications and Networking 82 JOURNAL OF SOFTWARE MAINTENANCE AND EVOLUTION-RESEARCH AND PRAC- TICE 56

62 Chapter 7. Appendix 83 JOURNAL OF STRATEGIC INFORMATION SYSTEMS 84 JOURNAL OF SYSTEMS AND SOFTWARE 85 JOURNAL OF VISUAL COMMUNICATION AND IMAGE REPRESENTATION 86 Journal of Web Semantics 87 MATHEMATICAL AND COMPUTER MOD- ELLING 88 MATHEMATICAL PROGRAMMING 89 METHODS OF INFORMATION IN MEDICINE 90 MIS QUARTERLY 91 MOBILE NETWORKS & APPLICATIONS 92 ONLINE INFORMATION REVIEW 93 Personal and Ubiquitous Computing 94 SIAM Journal on Imaging Sciences 95 SIMULATION MODELLING PRACTICE AND THEORY 96 Software and Systems Modeling 97 SOFTWARE TESTING VERIFICATION & RELI- ABILITY 98 VLDB JOURNAL 99 WIRELESS COMMUNICATIONS & MOBILE COMPUTING 100 WORLD WIDE WEB-INTERNET AND WEB IN- FORMATION SYSTEMS 57

63 Chapter 7. Appendix 7.7 Decision Model The code metrics have been referenced according to the corresponding letter at table 2.3, the code analysis tools have been referenced according to appendix

64 Chapter 7. Appendix 7.8 Entrepreneurship Project The following section is the report written during the work that we did in the course Entrepreneurship at the 9th semester of our education at Aalborg University. The report describes an idea for how the knowledge gathered from this project can be used to create a business. The Entrepreneurship Project therefore focuses on the business model as opposed to the computer science technicalities of the project. Project 59

65 Metrication For små virksomheder med store ambitioner