Integrating Management and Engineering Processes in Software Product Development

Transcription

1 Integrating Management and Engineering Processes in Software Product Development Daniel Karlström Department of Communication Systems Lund Institute of Technology

2 Integrating Management and Engineering Processes in Software Product Development ISSN ISRN LUTEDX/TETS SE+230P Printed in Sweden E-husets tryckeri Lund

3 3

4 Integrating Management and Engineering Processes in Software Product Development This thesis is submitted to Research Board FIME - Physics, Informatics, Mathematics and Electrical Engineering - at Lund Institute of Technology (LTH), Lund University, in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Engineering. Contact Information: Daniel Karlström Department of Communication Systems Lund University Box 118 SE LUND Sweden Tel: Fax: daniel.karlstrom@telecom.lth.se 4

5 Abstract The intangible nature of software means that traditional processes for managing product development are not sufficiently effective. This, in combination with the increasing share of software in technical products, has had the effect of making it difficult to identify improvement proposals for engineering processes in many cases. This thesis uses both qualitative and quantitative research methods to investigate methods that involve the entire development organisation in software process improvement. The research is performed in cooperation with several companies developing software products in different countries. The studied organisations range from small development companies, using light, informal methodologies, to large corporations, with thousands of developers, using large frameworks of formal process models. The need for organisations to use both local experience as well as generally accepted best practice in process improvement is addressed. A method that uses input from the whole organisation for strategic software process improvement is created and evaluated. The introduction of one agile methodology, Extreme Programming, is studied in a small team and a decision support method is evaluated for introducing the methodology. Also, a framework based approach to process improvement is applied to testing practices. The approach allows a small, rapidly evolving, company to introduce suitable practices that solve problems apparent in the current state of the company. This allows the processes to evolve in small steps without introducing formal practices too early in the company s evolution. An integration of agile methods and high-level product management processes is proposed and evaluated on both the engineering level and the product management level of organisations. The integration allows contrasting elements of the processes to remain effective within each respective domain and in some instances even perform synergetically. 5

6 Integrating Management and Engineering Processes in Software Product Development 6

7 Acknowledgements This work was partly funded by The Swedish Agency for Innovation Systems (VINNOVA), under a grant for the Centre for Applied Software Research at Lund University (LUCAS). The research performed in Japan was funded using contributions from the Sweden-Japan Foundation, Per Westling s memorial foundation, Hosei University Tokyo, the Ernhold Lundström scholarship, and the Saskawa foundation. Travel support was also received from the Royal Physiographic Society in Lund. First and foremost I would like to thank my supervisor, Dr. Per Runeson, for supporting, guiding, inspiring and restraining me during the work presented in this thesis. I would also like to thank my second supervisor, Dr. Martin Höst for always being available to answer my questions. I would like to take this opportunity to thank Prof. Claes Wohlin for convincing me to start as a PhD student in the group. I would also like to thank my co-authors and all other people who have contributed to the research performed in the thesis. A special thanks to all my colleagues at the department of Communication systems for creating a professional working environment. Especially I would like to thank the members of the Software Engineering Research Group, Carina Andersson, Enrico Johansson, Lena Karlsson, Johan Natt och Dag, Josef Nedstam, Bengt Ljungquist, Dr. Björn Regnell, Dr. Thomas Thelin, and former group members Dr. Magnus C. Ohlsson, Dr. Tomas Berling Dr. Håkan Petersson and Thomas Olsson for providing the opportunity for many rewarding discussions both on and off the topic of software engineering. Thanks also to the Telecom floor ball players for allowing me to clobber them every Monday evening. Finally, I am grateful to Kerstin, my friends outside the department and my family. Thank you for putting up with me, especially during the last year while performing my studies away from home and, of course, during the last few months of finishing the thesis. 7

8 Integrating Management and Engineering Processes in Software Product Development 8

9 Contents Introduction Background Research Methodology in Software Engineering Research Presentation References...56 Paper 1: Aggregating Viewpoints for Strategic Software Process Improvement A Method and a Case Study Introduction Processes Distortion and Viewpoints Method Case Study at Fuji Xerox Summary and Conclusions Acknowledgements References...88 Paper 2: Introducing Extreme Programming A Case Study Introduction Methodology Online Telemarketing The Development Project Experiences of the XP Practices Quality of Conclusions Summary and Conclusions Acknowledgements References

10 Integrating Management and Engineering Processes in Software Product Development Paper 3: Decision Support for Extreme Programming Introduction and Practice Selection Introduction XP Introduction Strategies Methodology Evaluation Results Summary Acknowledgements References Paper 4: Minimal Test Practice Framework for Emerging Software Organisations Introduction Related Work The Minimal Test Practice Framework Derivation of the Framework Application of the Framework Survey Validation of the Framework Validity of the Studies Summary and Future Work Acknowledgements References Paper 5: Agile Methods and Software Product Management An Integrative and Hybrid Approach Introduction Background Integrating Project Management Models and Agile Methods Example Challenges for the Future Acknowledgement References

11 Paper 6: Integrating Agile Software Development into Stage-Gate Managed Product Development Introduction Study Scope Case Contexts Study Design Study Validity Results Summary and Conclusions Acknowledgements References Paper 7: Exploring Agile Influences in Stage-Gate Management of Software Product Development Introduction Vodafone Group Global Product Development Study Design Study Validity Results Summary and Conclusions Acknowledgements References Reports on Communication Systems

12 Integrating Management and Engineering Processes in Software Product Development 12

13 Introduction Software has become a part of almost every product being developed today. The intangible nature of software means that traditional processes for managing industrial projects are not effective. This, in combination with the increasing size and complexity of software products, has had the effect that improvement strategies for development processes are in most cases hard to identify. The field of software processes and process improvement has had to evolve constantly in order to keep up with changes in the type, size and complexity of the software being developed. In addition to this, the number of people performing the development has increased greatly as well as the variety of different types of software. An excellent example is the extent of Internet software developed in the early 1990s compared to early Currently there are two quite different approaches to process improvement being used in the industry, (1) methods based on maturity models, and (2) agile methodologies. The maturity models approach is regarded a more formal and heavy approach and has evolved from the Capability Maturity Model (CMM) from the beginning of the 90 s. The agile approaches have evolved partly as a reaction to the first and are considered more lightweight. Currently the agile approach is thought less compatible with large-scale software product development, often performed in strict, staged, management models. The research performed in this thesis indicates that there may indeed be several advantages both on the engineering and management levels of the organisation by using agile concepts instead of traditional heavyweight concepts. The research presented in this thesis is performed in both small development organisations, using light, less formal methodologies, known as agile methodologies, and large organisations, with thousands of developers, using large, more formal development process frameworks. 13

14 Integrating Management and Engineering Processes in Software Product Development A method that uses input from the whole organisation for strategic software process improvement is created and evaluated. This method allows the process owners to base software process improvement on input from the entire development organisation. This is thought to anchor the decision more firmly in the organisation. The introduction of one of the main agile approaches, Extreme Programming, is studied in a case study of a small development project in a real world context as well as in a large stage gate based product development context in two other cases. A decision support method is evaluated for identifying practices within Extreme Programming that might be perceived to be more or less effective in the developer group before the introduction. Hence the introduction of the practices can be tailored based on the results of the analysis. A framework approach to process improvement for testing is investigated that allows the company to introduce suitable practices that solve problems apparent in the current state of the company. This allows the test processes to evolve in small steps without introducing too many formal practices too early in the company s evolution. The research presented in this thesis attempts to reduce inherent resistance to process changes in organisations by increasing the involvement of the whole organisation in the improvement work and thereby increasing acceptance for process improvement. The seven research papers in the thesis can be classified into research performed in a context of a small or large company and into research in the use of traditional or agile methods. This classification is shown in Table 1.1. The papers could be read in any order, but have been numbered chronologically as the studies have been performed. The papers are provided exactly as they have been published, only the layout has been altered to suit the format of this thesis. This has the advantage of making each paper more understandable as a unit but has the disadvantage of some repetition of material in the introduction and background.parts of the papers. Large context Small context Traditional Paper 1 Paper 4 Agile Paper 5 Paper 6 Paper 7 Paper 2 Paper 3 Table 1.1 Paper Classification 14

15 The first part of this thesis is an introductory part that summarises and packages the work performed. It is divided into the following sections: 1. Background This section briefly explains the background of software system engineering and software engineering processes, the structure of the discipline, the terminology involved and the challenges present today. 2. Research Methodology This section discusses research methodology used in software engineering research. Parts of this section are based on the following publication. Karlström, D., Effects of Introducing Agile Methodologies in Software System Engineering, 3rd National Conference on Software Engineering Research and Practice in Sweden, SERPS'03, Lund, Sweden, October Research Presentation This section presents the research performed in this thesis, the strategies used, the main contributions made and an outline for further research in the field. Parts of this section are based on the following publication. Karlström, D. Runeson, P., Combining Stage-Gate Product Development with Agile Software Development: Observations from three industrial cases, Submitted the special issue of IEEE Software on software engineering project management, May/June, The second part of this thesis contains the seven research papers that constitute the main contribution of the thesis: Paper 1 - Aggregating Viewpoints for Strategic Software Process Improvement A Method and a Case Study Karlström, D, Runeson, P., Wohlin, C., Published in IEE Proceedings Software, October Paper 2 - Introducing Extreme Programming - An Experience Report Karlström, D., Published in proceedings of the third International Conference on Extreme Programming and Agile Processes in Software Engineering, Paper 3 - Decision Support for Extreme Programming Introduction and Practice Selection Karlström, D., Runeson, P., Published in proceedings of the fourteenth International Conference on Software Engineering and Knowledge Engineering,

16 Integrating Management and Engineering Processes in Software Product Development Paper 4 - Incremental Introduction of a Structured Test Practice Framework in Emerging Software Development Organisations Karlström, D., Runeson, P., Nordén, S., Accepted for publication in the Journal of Software Testing Verification and Reliability, preliminary publication issue: June Paper 5 - Agile Methods and Software Product Management An Integrative and Hybrid Approach Runeson, P., Karlström, D., Technical report, Lund Institute of Technology, CODEN : LUTEDX (TETS-7203) / 1-34 / (2004) & local 16, Paper 6 - Integrating Agile Software Development into Stage-Gate Managed Product Development Karlström, D., Runeson, P., Submitted to the Journal of Empirical Software Engineering, Paper 7 - Exploring Agile Influences in Stage-Gate Management of Software Product Development Karlström, D., Submitted to IEEE Transactions on Engineering Management, Background The background to this thesis starts with an introduction to software engineering (SwE), system engineering (SE) and software system engineering (SwSe). The area of the thesis is mapped into the software engineering body of knowledge (SWEBOK) [1] followed by an introduction to SwE processes, software process improvement (SPI). Process maturity models are described in a separate section as are Agile Methodologies and project management models as these areas are central to the thesis topic. 1.1 Software Engineering and Software System Engineering Software system engineering (SwSE) is considered the application of system engineering (SE) on software development [2]. SE is the practical application of scientific, engineering and management skills required to move from a need to a technical realisation of that need. At the SE level the product is generic, i.e. the product does not necessarily have to include software at all, can be part software or a complete software 16

17 product. A standard for SE is provided in IEEE Std [3]. Here, five aspects of SE are identified: Problem definition, Solution analysis, Process planning, Process control, and Product evaluation. Relationships between SE, SwSE and software engineering are well described by Thayer [2] and are illustrated in Figure 1.1. Here, a sequential V-type model [4] relationship is implied for the activities on all levels. However, there are activities on all levels throughout the course of the product development. System safety is often a very important factor in large systems. The software part of the system is often the hardest part in which to ensure safety. Other aspects of system engineering such as logistic engineering, integrated logistics support (ILS) and logistics support analysis (LSA), although they may be remotely affected by a software development methodology change, fall outside the scope of this thesis. The Software Engineering Body of Knowledge (SWEBOK) [1] lists systems engineering as a related discipline to software engineering. System analysis System engineering System testing System design Software requirements analysis Software system engineering System integration testing Software system testing Architectural software design Detailed software design Software integration testing Software subsystem testing Software engineering Code and unit test Figure 1.1 Engineering relationships between system engineering, software system engineering and software engineering [2] The Structure of the Software Engineering Discipline Software Engineering is a relatively new discipline and it is only recently that structure and terminology has become well established. The collaborative effort creating the Software Engineering Body of Knowledge (SWEBOK) [1] has begun the important task of creating an introduction 17

18 Integrating Management and Engineering Processes in Software Product Development to the body of knowledge. This is, however, a difficult task as the discipline is constantly changing. The overview provided by this initiative is provided in Figure 1.6. SWEBOK divides the discipline into ten knowledge areas, requirements, design, construction, testing, maintenance, configuration management, engineering management, process, tools and methods, and quality. The knowledge areas represent areas where research is being performed today and have been established at different times during the evolution of software engineering processes up to the present. The first five knowledge areas are ordered according to the steps in the traditional waterfall development process [6]. The knowledge areas themselves, however, are not on the same level of abstraction as each other and many are very closely related. Some sub-areas are even part of several other areas. For instance, the software process knowledge area is a meta-area compared to the first five knowledge areas. All the areas however play a significant role in the development of software today. The research presented in this thesis can be categorised into the software engineering process knowledge area of the SWEBOK [1], but parts of the software engineering management area are also closely related to the work. An overview of the guide to the SWEBOK is shown in Figure 1.2 and relevant areas are highlighted. The process knowledge area is subdivided into the following sub-areas: Process implementation and change This sub-area focuses on organisational change. It describes the infrastructure, activities, models and practical considerations for process implementation and change. Process definition This sub-area focuses on how processes are defined. This can be as procedures, a policy or a standard. Process assessment This sub-area focuses on how process assessment or appraisal is carried out. Process and product measurement This sub-area focuses on how measurement is performed both to evaluate the need for process change and the effects of process changes made. 18

19 Software Requirements Software Requirements fundamentals Requirements Process Requirements Elicitation Requirements Analysis Requirements Specification Requirements Validation Practical Considerations Software Design Software Design Fundamentals Key Issues in Software Design Software Structure and Architecture Software Design Quality analysis and Evaluation Software Design Notations Software Design Strategies and Methods Software Construction Software Construction Fundamentals Managing Construction Practical Considerations Guide to the Software Engineering Body of Knowledge (2004 Version) Software Testing Software Maintenance Software Configuration Management Software Engineering Management Software Engineering Process Software Testing Fundamentals Software Maintenance Fundamentals Management of the SCM Process Initiation and Scope Definition Process Implementation and Change Test Levels Key Issues in Software Maintenance Software Configuration Identification Software Project Planning Process Definition Test Techniques Maintenance Process Software Configuration Control Software Project Enactment Process Assessment Test Related Measures Techniques for Maintenance Software Configuration Status Accounting Review and Evaluation Process and Product Management Test Process Software Configuration Auditing Closure Software Release Management and Delivery Software Engineering Measurement Figure 1.2 The Software Engineering Body of Knowledge Overview [1] (Highlights indicate the areas most relevant for this thesis) Software Engineering Tools and Methods Software Tools Software Engineering Methods Software Quality Software Quality Fundamentals Software Quality Management and Processes Practical Considerations Related Disciplines Computer Engineering Computer Science Management Mathematics Project Management Quality Management Software Ergonomics Systems Engineering

20 Integrating Management and Engineering Processes in Software Product Development 1.2 Software Engineering Processes As software engineering processes have developed, the terminology has evolved successively. Sommerville defines the software process as: The software process is the set of activities and associated results which produce a software product [5]. We can model this basic concept by using an elementary process model adapted to software development which gives us a basic software process. This is illustrated in Figure 1.3. Resources Product idea Software process Software Product Figure 1.3 An illustration of the software process [5] Any software process needs to at least address the following activities in some form in order to develop software [5]: Specification. Decide what is to be developed. Development. Develop the product. Validation. Ensure the product meets the specification. Evolution. Handle changes in the product. In today s methods, the execution of the software process is done in many different ways. An example of software development that does not address these activities is the methods that sparked the software crisis, also known as code and fix methods. The lack of planning and requirements in these methods, made the following problems common: Poor structure. After each fix the structure of the code is destroyed making subsequent fixes and additions more and more difficult. Inaccurate results. The resulting program is hardly ever what the customer desired in the first place due to the lack of requirements. 20

21 Expensive. Because of poor structure and lack of planning all modifications and fixes become very expensive. When these problems were observed, new models that addressed planning, requirements, test and modification were developed. Sommerville identifies four different types of such software development models currently in professional use [5]: Waterfall type [6]. Each activity - specification, development, validation and evolution - is executed and signed off sequentially, one by one. Evolutionary type [5]. The activities are interleaved to rapidly produce prototypes of increasing complexity and correctness with regards to customer requirements. Formal transformation [7]. The system is specified as a mathematical system and is then transformed into the finished product using formal mathematical transformations. Component based [8]. The software system is assembled from pre-developed parts. Of these, the waterfall and evolutionary types are most widely used in industry today, but the potential effectiveness of software reuse has caused much interest in component based software engineering. In the classification above, agile methodologies [9] are classified as evolutionary by Sommerville. These are, however, commonly regarded as a separate type. These methodologies are very new and there is a need for more research into the area. The agile methodologies are discussed in section 1.5. One of the current problems within software engineering is the use of many different terms for similar items within different areas of the discipline. This has arisen over time as certain terms have been associated with certain methodologies or have become negatively charged due to their usage in different circumstances. New software processes have changed the terminology in order not to be associated with old values or just in plain ignorance of existing terminology. The terms that are used within this thesis are shown in Table

22 Integrating Management and Engineering Processes in Software Product Development Software Engineering Process Evolution Software engineering processes can be divided into three distinct periods chronologically and we are currently moving into a fourth period [10]. The periods are illustrated in Figure 1.4 and a brief summary is provided below The first period can be referred to as code and fix [11]. Much programming was performed using low-level programming languages, program size was relatively small and the software environment was less complicated with fewer external interactions [12]. This implied that many of today s problems were not apparent during this period, although surprisingly many current problems were mentioned by Brooks in his classic book [13] in Performing software development was relatively simple compared to today s elaborate systems as it meant that bugs could be fixed. Systems could actually be developed that were completely fault free. Term Definition Process A sequence of steps performed for a given purpose [14]. Model Software process Practice Practice framework Method Methodology Improvement method Maturity model A model is a simplified representation of a system or phenomenon with any hypotheses required to describe the system or explain the phenomenon, often mathematically. It is an abstraction of reality emphasizing those aspects that are of interest to someone [15]. The set of activities and associated results which produce a software product [5]. A specific activity within a process or method. A collection of practices creating a form of a software process. A reasonably complete set of rules and criteria that establish a precise and repeatable way of performing a task and arriving at a desired result [16]. A collection of methods, procedures and standards that defines an integrated synthesis of engineering approaches to the development of a product [16]. Defines the strategy and the process by which improvement is pursued. It specifies how to assess a process, to initiate corrective actions, to evaluate them, and to further promote new improvement initiatives. Maturity model defines the ideal profile of a process, i.e., the (minimum) set of requirements that a process should satisfy to qualify for a specific status. Table 1.2 Terminology within software engineering processes 22

23 The second period can be referred to as the structured methods period [11]. This period started after the software crisis in the late 1960 s when new hardware advances enabled software systems to be developed that were too large for the traditional development methods used at the time [5]. Elementary software development models were developed and used in industry. A major characteristic of this period is that the software engineering process was being described graphically. Examples thereof are waterfall and spiral processes [6, 17]. A large portion of practicing software development companies today are still using gate processes similar to these early processes although they are tackling functionality several orders of magnitude larger than in the end of the 70s. This evolution of the waterfall method is commonly referred to as gate or milestone models [18, 19, 5, 20]. Period 1: Code & Fix Period 2: Structured methods Period 3: Process Maturity ~1970 ~1990 ~2000 High Maturity Agile Methodologies? Figure 1.4 Periods in software engineering process evolution The third period, process maturity, introduced a meta-level to the development processes. During this period, software process improvement (SPI) processes and models were developed and implemented [10]. The essence of SPI is to use an improvement process, functioning on a metalevel with respect to the traditional software development process. The current development process is assessed compared to a reference process model or framework and then adjusted accordingly. The most well known example of work originating from this period is the Capability Maturity Model (CMM) from the Software Engineering Institute (SEI), Carnegie Mellon University [21]. Another example is ISO 9000 and 9001 [22, 23, 24, 25]. Most of the third period models are appropriate for larger companies though the research community has made several attempts at adapting them to smaller organisation types [26, 27, 28]. The fourth period was predicted to be the industrialised software engineering phase during the third period [29]. The engineering methods used would resemble the production methods present in other more 23

24 Integrating Management and Engineering Processes in Software Product Development established engineering disciplines such as civil engineering or manufacturing. This fourth period, however, is currently only beginning to take form. There has been a distinct change in the software process community with the introduction of the processes known as the agile methodologies that appear to be in contrast to the continued evolution of the maturity models towards working with higher maturity levels. Both of these have evolved from software engineering using third period methodologies but with different approaches to the problems discovered. Table 1.3 compares an example of inherent properties of software compared to traditionally industrially manufactured items. The properties indicate some of the problems that were thought to be solvable with fourth period methods. Due to the different nature of software, however, the industrialisation may never be possible. Software Traditional Industry 1 Build once Build all the time 2 Reproduction low cost Reproduction high cost 3 Very flexible Inflexible 4 Intangible Physically present 5 Customer often undecided Customer determined 6 High maintenance cost Rel. low maintenance cost Table 1.3 Comparisons between inherent properties of software and traditional industrial artefacts [30] The creation of software can be compared to preparing for industrial production of a product. Here the community has identified some of the most useful items to apply in the field of SE, such as Total Quality Management (TQM) and Plan-Do-Check-Act (PDCA) [31, 32]. The fourth period has started with a division of the SE community. The successful practitioners of the third period models have continued their improvement into high maturity practices. The companies that failed in their intent to introduce third period processes, or never even attempted this, have taken to agile methods as the natural next step. In this second group, there is a strong reaction towards unnecessary formalism and overhead in the third period processes. The development over the next few years will show the approaches converging and being adapted to different circumstances in different environment and it will become more apparent that both groups are striving towards a common goal of producing quality software as efficiently as possible, with as much control as possible, and with content people within the organisations. To do this, influences will be needed from within the entire scope of the practice, and theory, of SE today and 24

25 that each organisation producing a different type of software will require a highly adapted methodology for their software type, organisation type and individual people types involved in development. This can be identified in Brooks early paper [33] from which we can quote the classic There is no silver bullet for software development. 1.3 Software Process Improvement Software process improvement (SPI) involves working with the existing software process in a company and improving it. The area can be divided into the process for performing the improvement work and the reference needed for comparison [34] Software Process Improvement Methods A generic SPI method can be simplified to three steps: start, assess and change. The SPI initiative starts by the insight that the current development process can and needs to be improved. In this first step, start, sponsorship from senior management is ensured. This is in order to make sure that support and resources are available and to spread knowledge and acceptance about the effort. During this initial step a general plan is created and the goals for the initiative are also drawn up. Assessment of the software process in the next step, assess, concerns making an evaluation of the current process and comparing it to a reference of some kind. This can be performed both internally by the company, externally by assessors or a combination of both. The actual change is performed in the final step, change. Typical methods for introducing change include training and introduction of new process documentation. Typical examples of process improvement methods are SEI s IDEAL method [35], specialized for self-assessment and capability evaluation, CMM based appraisal for internal process improvement (CBA-IPI) [36], Software Capability Evaluation (SCE) [37], and the forthcoming SPICE/ISO15504 standard [38]. The IDEAL method was developed by SEI as a way of using the CMM based appraisal method for process improvement. As the method does not refer to a specific model for comparison, this model could be used in combination with other models than the CMM. The IDEAL method is illustrated in Figure 1.5. Note the similar structure to the basic phases described earlier in this section. Here the initialising phase of the 25

26 Integrating Management and Engineering Processes in Software Product Development IDEAL model corresponds to the start phase, diagnosing corresponds to the assess phase and establishing, acting and leveraging correspond to the change phase. Leveraging Stimulus for improvement Initialising Set context and establish sponsorship Diagnosing Establish improvement infrastructure Appraise and characterize current practice Develop recommendations and document phase results Revise organisational approach Set strategy and priorities Document and analyse Define lessons processes and measures Plan and execute pilots Plan, execute and track installation Establish process action teams Plan actions Acting Establishing Figure 1.5 SEI IDEAL process improvement model [35] A process improvement should be relevant, i.e. actually improve the process, as well as applicable, i.e. possible to introduce both technically and socially in the target context. In order to be effective a process improvement should therefore contain both improvements based on the local situation as well as general improvements. This dual nature of process improvement is not reflected clearly in the SPI methods referenced. It is referred to as tailoring by some but no real guidelines are provided. Improvements based on local situation have the positive characteristics that they are easily accepted in the organization and are also effective in that organization. They have the negative characteristics that they are most probable not found in general SPI frameworks and so can be difficult to identify. Secondly they have a low degree of transferability. A generic improvement process based on the local situation in a company is shown in Figure

27 Improvements based on general frameworks have positive characteristics that their need may not be apparent in the organisation despite their relevance. Secondly they have a high degree of transferability. They have the negative characteristics that they can be perceived to be forced onto the practitioners in the organisation and may not be as effective. A generic improvement process based using a general framework is shown in Figure 1.7. Actual behaviour Observe Decide What & How Improve Figure 1.6 Improvement based on local situation Actual behaviour Compare Decide How Improve Reference Figure 1.7 Improvement based on general framework The most effective SPI process therefore would attempt to use improvements based on both the local situation and general frameworks. This hybrid process is illustrated in Figure

28 Integrating Management and Engineering Processes in Software Product Development Observe Decide What & How Actual behaviour Improve Compare Decide How Reference Figure 1.8 Improvement based on both local situation and general framework A further issue here is identifying general frameworks that apply to different context within SE. For example frameworks for small vs. large companies, product development vs. contract based development and so on. This creates an in-between to completely general frameworks such as the CMM [16] and completely local based improvement methods such as TQM [31, 39, 40]. Looking at SPI and SPI models on another level we can examine the origin of the reference framework and improvement method and how these were established. Most frameworks are established as collections of best practices based on the aggregated experience of a varying number of individuals or developing organisations. This ranges from frameworks without any source traceability what so ever to the CMM [16], which can be classified as one of the largest collections of software practices performed. The generic SPI processes are composed of three types of steps: The first is either observe or measure, the second decide and the third do. Each step can potentially be improved using a different strategy from SwE research methodology, presented in Section 2, due to that the goal of the work involved are similar to that of the research performed: Observe Qualitative research validity theory (see Section 2.2). Measure Quantitative research validity theory.(see Section 2.1) Decide using Internal & external experts or analysis, different sources of experience and decision support methods. Do improve anchorage, motivation and support for process changes. 28

29 In addition, the analogy can be extended to flexible research being similar to non-framework based SPI and fixed research being similar to framework based SPI. The synergy effects of integrating fixed and flexible research approaches can therefore be analogous to integrating the different types of SPI methodologies. We can use this analogy to improve both the application of software process improvement in the target context as well as to improve creation of software process frameworks. One of the most difficult steps is the decision step. Based on observations made, and potentially comparisons to a framework, a decision is taken based on the decision takers knowledge, experience and beliefs. The person or group making the decision as well as the apparent basis he takes the decision on affects this decision significantly. This aspects is often not included in SPI methodologies A general process framework is created through a large number of observations and/or experiences generalised to a framework. The generalisation step involves deciding how to incorporate differences between cases, which can be difficult. The process, which is parallel in nature, is illustrated in Figure 1.9 Once created, a general process framework needs to evolve to keep up with industry practice. This process is more serial in nature and is illustrated in Figure Different Contexts (similar and different in nature) Actual behaviour Actual behaviour Actual behaviour Observe Observe Observe Generalise Generalise Generalise General Framework Figure 1.9 Genesis of a general framework 29

30 Integrating Management and Engineering Processes in Software Product Development Framework Framework Framework Compare Compare Compare Decide Decide How Decide How How Improve Improve Improve Actual Actual behaviour Actual behaviour behaviour Figure 1.10 Evolution of a general framework Two cases, CMM [16] and agile methodologies [9], are discussed here to demonstrate the evolution of two different software process models that are of significance in the third and fourth periods respectively. It is interesting to note the opposite direction of introduction in the two cases. In the case of agile methodologies the source of the methodology is industry and the introduction is bottom up from developer to manager whereas in the CMM case the source of the method is academia in cooperation with the US Department Of Defence (DoD). The cases illustrate different roles that academia has taken during the evolution of two different software processes. The current trends in software processes must affect not only what the academia is researching, but also how the research is performed. For instance, in the CMM case the intent of each part of the process is clear, but the actual industrial effect is unclear and needs to be investigated. This was done for the lower maturity levels in the CMM as many companies reached the lower levels relatively quickly. The effects of the higher levels have not really been seen until the latter part of the 90 s and the content thereof can be seen as largely speculative based on what was though to be needed once the initial practices at the lower levels were in place. This has resulted in updates due to experience gathering at for example high maturity workshops [41]. In the second case however the intent of the practices are not as clear to the academic community as the formulation of the practices have not involved the academic community. As in the CMM case the actual effects of the practices need to be studied, but the fact that the models from an academic point of view seems to contain several dangerous activities and excludes several activities proven to be effective and are still used needs to be investigated. 30

31 1.3.2 Challenges in SPI Humphrey describes six basic principles that need to be followed during the course of process improvement. These provide an overview of some basic issues involved in software process change [42]: 1. Major changes to the software process must start at the top. Senior management leadership is required to launch the change effort and to provide continuing resources and priority. 2. Ultimately everyone must be involved. Software engineering is a team effort and everyone who does not participate in the improvement will miss the benefits and may even inhibit progress. 3. Effective change requires a goal and knowledge of the current process. To use a map you have to know where you are. 4. Change is continuous. SPI is not a one shot effort; it involves continuous learning and growth. 5. Software process changes will not be retained without conscious effort and periodic reinforcement. Effort must be taken not only to introduce changes, but also to retain them. 6. SPI requires investment. SPI takes planning, dedicated people, management time and capital investment. One of the goals of the improvement effort is to improve the relationship between actual required resources compared to the estimation of resources. The process can be affected in two ways: Predictability Moving the estimation of the required resources towards the average of the actual required resources. Control Decreasing the variance of the actual required resources. The official process described in a company exists in different forms depending on which phase of the process improvement we investigate it. A presentation of the different kinds of processes that can be present in an organisation is given in Table 1.4 [43]. 31

32 Integrating Management and Engineering Processes in Software Product Development Process Name Description Reference The sources from which the process ideas originate (Theoretical papers, books, etc). Desired The process that the person interpreting the references wants to introduce. Official The process described on paper by the organisation. Perceived The process that the performing subjects believe they should perform Actual What is actually performed. Observed The observed process. Table 1.4 Different types of the same process [43] Each process type is related to the next by a mechanism of transfer. For example the perceived process is obtained from the official process through teaching, peer information and self-study. The mechanisms are summarised in Table 15. It can be seen that three of the processes shown in the table are transferred by interpretation. Interpretation is highly subjective, i.e. highly dependant on the person involved. A person s subjective interpretation of the process is known as that person s viewpoint [44]. From To Mechanism Description Reference Desired Interpretation Reference processes are interpreted by the person(s) creating the official process. Desired Official Formulating The official process is formulated from the desired process and written down as the official process. Official Perceived Interpretation The user interprets the official process based on what he is taught, what he reads and how his co-workers influence him. Perceived Actual Performing What is actually performed is a result of the perceived new process and the old process. Actual Observed Interpretation The observed process is a result of an interpretation of the actual process. Table 1.5 Mechanisms of transfer between types of processes 32

33 The actual process is unique in the sense that this is the process that actually creates the product. In many ways this makes it the most important process, as this is the process that we must ultimately improve. It is observed through the interpretations of observers but cannot be objectively evaluated by observation alone. The use of some form of metrics, however, offers an objective unbiased picture of the results of the actual process [45]. This indicates the importance of metrics in SPI work. SPI involves changing existing practices, processes and structures. The study of the phenomena observed when introducing changes to a group of people belongs more to the social sciences than engineering science. It is however very important to remember that there are humans involved in the development process and any action made needs to take this fact into account. All changes must therefore be practised, improved and will then eventually become a natural part of the company s practices after the introduction. A typical response to change and the effort exerted by the subject is illustrated in Figure It must also be remembered that the current process being used at a company is the result of many years of experience, and most of it is usually good practice. Effort Anger, Rage Bargaining Acceptance Status Quo Denial Testing Stunned paralysis Depression Figure 1.11 Typical human response to change [10] Time 1.4 Process Maturity Models There are several process maturity models used in practice, both commercial and non-commercial. Without any doubt the most used is the CMM model [16]. The framework quagmire [46] shows an overview of many of the alternatives that exist. How these relate to each other also gives an idea of how they have evolved. Here the term framework is used as a collective terms for process standards, quality standards, maturity or capability models, appraisal methods and guidelines so as to compare them directly. The choice of framework is, however, often much more limited due to pre-set requirements from, for example, contractors, management, the current process and the improvement goal itself. The 33

34 Integrating Management and Engineering Processes in Software Product Development most influential frameworks in the quagmire are the CMM-based models [16], ISO 9000 based standards [22], and various military standards. These frameworks are also the most widely used today and many companies are assessed in several standards. Process improvement models are not only different in their composition and content. They also have different purposes and uses. Some models are intended for use in one special project because the customer demands it, while others are intended to be used consistently across a whole organisation, such as the CMM. The Software-CMM (SW-CMM) [16] contains a framework of practises that define different levels of maturity in the software development process. The model is composed of five levels of increasing maturity. Each level contains practises that would normally be expected in a software process at the corresponding maturity level. Organisations that do not implement CMM are said to be at level 1, the initial stage. Each level, except for level 1, is divided into several key process areas (KPA), which describe the areas that the level addresses. For an organisation to be performing at a certain CMM level, it must successfully implement the practices described at that level and at all lower levels. The CMM can be used in five different ways [16]: 1. CMM based assessment to identify process strengths and weaknesses. 2. Evaluating subcontractors for selection. 3. Developing new appraisal methods based on the CMM. 4. Teaching upper management about SPI introduction. 5. As a guide for technical staff and process improvement groups in their work to define and improve their processes. Since its introduction the SW-CMM [16] has evolved in to the Capability Maturity Model Integrated (CMMI) [47], by being updated and integrated with the system engineering CMM (SE-CMM) [48]. This model is available both in a staged version such as in the original SW- CMM, or a continuous version from the SE-CMM. When using the CMM models it should be remembered that they were created at the initiative of the Department of Defence and that their way of acquiring software is usually to select one of several contractors to complete the project at a fixed price. The projects involved are usually quite long, not market driven or of a product type and have relatively stable requirements. The software industry has changed radically during the 90s with the popularisation of the Internet and exponential growth of 34