Sven Ziemer. Trade-offs for web application development: understanding and improving current industrial practices

Transcription

1 Sven Ziemer Doctoral theses at NTNU, 2009:02 Sven Ziemer Trade-offs for web application development: understanding and improving current industrial practices ISBN (printed ver.) ISBN (electronic ver.) ISSN Doctoral theses at NTNU, 2009:02 NTNU Norwegian University of Science and Technology Thesis for the degree of philosophiae doctor Faculty of Information Technology, Mathematics and Electrical Engineering Department of Computer and Information Science

2 Sven Ziemer Trade-offs for web application development: understanding and improving current industrial practices Thesis for the degree of philosophiae doctor Trondheim, January 2009 Norwegian University of Science and Technology Faculty of Information Technology, Mathematics and Electrical Engineering Department of Computer and Information Science

3 NTNU Norwegian University of Science and Technology Thesis for the degree of philosophiae doctor Faculty of Information Technology, Mathematics and Electrical Engineering Department of Computer and Information Science Sven Ziemer ISBN (printed ver.) ISBN (electronic ver.) ISSN Doctoral Theses at NTNU, 2009:02 Printed by Tapir Uttrykk

4 If you don t know where you are going, any road will do. Chinese proverb

5 ii

6 Abstract When web applications are developed with a strong emphasis on time-to-market, the resulting development and trade-off practices are informal and make use of uncertain and fragmented knowledge. Developers and other stakeholders have to perform trade-offs using imprecise specifications, uncertain knowledge, and their beliefs, opinions and interests. In addition, this knowledge is fragmented between the stakeholders, since each one holds his own views, opinions and interests. Performing trade-offs therefore involves sharing the stakeholders knowledge and interests before a balanced decision can be made. The aim of this research is to understand how trade-offs are performed in web application development and to improve the current trade-off practices. In this research we have studied the development and trade-off practices in industrial web application development by interviewing eight Norwegian companies. The interviews with the companies identified real world problems that have been used further in the research. An analysis of the collected data has resulted in the discovery of several improvement opportunities for the applied trade-off practices. Working with these opportunities has resulted in a trade-off strategy and several trade-off tools. The trade-off strategy deals with the observation that many companies are unaware that they are performing trade-offs and that the trade-off tools use qualitative assessments of the stakeholders belief and opinion to perform such trade-offs. Two trade-off tools are tested experimentally by running a total of six experiments. The results of the experiments are used to improve a release planning method. By using real world problems from the interviewed companies the research has a realistic approach, yielding results that are relevant for industrial web application development. The following contributions are claimed: First, it describes development practices found in eight small Norwegian Web Application development companies, and focuses in particular on their trade-off practices. Second, a trade-off strategy to create awareness for trade-offs, and to enable knowledge sharing between stakeholders, is developed. Third, qualitative assessments of trade-off options are applied and tested, thus enabling prioritisation of trade-off options. Fourth, trade-off methods for enabling and facilitating knowledge sharing are developed. These include a release planning method that uses qualitative assessments to express the stakeholders beliefs and interests. Finally, a subprocess for managing change in web application development is provided, implementing a learning loop for knowledge management. iii

7 iv ABSTRACT

8 Preface This thesis is submitted to the Norwegian University of Science and Technology (NTNU) for the doctoral degree Doctor of Philosophy (PhD) The work has been part of the Websys project, which was financed by the Norwegian Research council (NFR), and has been carried out at the Software Engineering group, Department of Computer and Information Science (IDI) under the supervision of Professor Tor Stålhane. I also had the opportunity to visit the IMSE group at Vrije Universiteit in Amsterdam (The Netherlands) for 10 months. After working in industry for some years it was an exciting step to go back to university to start this PhD research. My hope was to build on my experience as project manager for a software process improvement project, and to further improve my insight into how software processes and practices are shaped within companies. This was the start of my travel through my PhD research, which took me to some unexpected situations and experiences. With time, my research was motivated by understanding the social relations between the involved stakeholders, how they communicate and how each stakeholder seeks the fulfilment of his interests. This travel has opened a new view for me, one which combines several disciplines, such as software engineering, organisational theory, decision making and communication theory. When submitting this PhD thesis I thus feel that I have not come to an end but rather to a starting point for the continuation of this work. Trondheim, 15 September 2008 Sven Ziemer v

9 vi PREFACE

10 Acknowledgements This work would not have been possible without the help, support, guidance, and inspiration of my colleagues, friends and family, and I am indebted to many. First of all, I am thankful to my supervisor Professor Tor Stålhane, who encouraged me in my work, who was open to many questions and discussions, and who let me have the freedom to approach the research in my own way. A special thanks also to my second supervisor, Professor Monica Divitini, who was open to discuss issues on research methods and the research design with me, and to Professor Reidar Conradi, who advised me on the structure of the thesis, and who gave me valuable feedback on my thesis. While working on my PhD thesis I made two wonderful friends: Siv Hilde Houmb and Xiaomeng Su. Most importantly, I am grateful for their friendship, which enriches my life. They have also supported me during my research work, and I would not have been able to complete this thesis without their support in difficult times. A special thanks to Xiaomeng for her help with the structure of this thesis and for her feedback. A special thanks goes to Ilaria Canova Calori who helped me to design, run and analyse the experiments on the release planning method for web application development. Part of my work was carried out at Vrije Universiteit in Amsterdam and I am thankful to Professor Hans van Vliet for hosting my stay. Many colleagues have made my time at the Software Engineering Group at NTNU a memorable time, with many interesting discussions and social activities. My thanks go to Kirsti Elisabeth Berntsen, Birgit Krogstie, Jeniffer Sampson, Sobah Abbas Petersen, Gry Seeland, Anita Gupta, Thomas Østerlie, Glenn Munkvold, Jingye Li, Darijus Strasunskas, Christian Mönch, Øyvind Hauge, Gasparas Jarulaitis, and Geir Kjetil Hansen. Thanks to the administrative staff, who helped me with many practical issues, and to Sari Cunningham, who proofread my thesis and helped to prepare this manuscript. Finally, I want to thank my wonderful wife Kari for her love and for her enduring support, understanding and encouragement. You convinced me to accept this PhD position at NTNU while you were starting your own PhD work in Nijmegen (The Netherlands). I am indebted to you for all the experiences I made during my PhD work and the view this work has given me on my own future. vii

11 viii ACKNOWLEDGEMENTS

12 Contents Abstract Preface Acknowledgements Contents List of Figures Abbreviations iii v vii ix xiii xv I Research context and results 1 1 Introduction Problem outline Research context Research questions Research design Papers Contributions Thesis structure Related work Web Engineering Web development practices Web development methods and tools Trade-off analysis methods Architectural trade-off analysis Requirement prioritisation trade-off analysis Development practices trade-off analysis Software release planning Release planning methods ix

13 x CONTENTS Empirical studies on release planning Decision making and knowledge sharing Knowledge sharing Empirical studies on decision making Summary Research Methods Overall research approach Research phases Phase 1: Data collection Phase 2: Analysis Phase 3: Problem Solving Phase 4: Validation Discussion of the overall research approach Research methods used in this thesis Qualitative research interview Postmortem Analysis (PMA) Grounded Theory Controlled experiment Results Web applications and development practices Web application characteristics Web applications lifetime phases Web application development practices Trade-off situations Trade-off strategy Tacit knowledge Improving trade-off practices Qualitative trade-off tool Trade-off assessment A release planning method Change process for web application development Evaluation Research questions revisited RQ 1: What characterises web application development? RQ 2: How are trade-offs performed in web application development, and how are trade-offs performed that involve time-to-market as a factor? RQ 3: How can a long-term trade-off strategy be enabled in web application development? RQ 4: How can knowledge sharing be enabled in web application development? Websys Goals revisited Websys main goal

14 CONTENTS xi Subgoals G1 G Contributions C1: Web development practices C2: Trade-off strategy for web application development C3: Qualitative assessment of trade-off options C4: Enabling and facilitating knowledge sharing C5: Knowledge management for web application development Contributions in relation to related work C1: Web development practices C2: Trade-off strategy for web application development C3: Qualitative assessment of trade-off options C4: Enabling and facilitating knowledge sharing C5: Knowledge management for web application development Relevance of contributions Validity discussion Conclusion and future work Conclusion Future work Web application development trade-offs Knowledge sharing II Papers 83 7 Paper Paper Paper Paper Paper Paper Paper Paper A Interview guide template 173 B Notes from an interview 179 C Post experiment questionnaire 187

15 xii CONTENTS Bibliography 191

16 List of Figures 1.1 Research overview The contribution of this work The research activities of this research and their relation to the contributions and publications Comparison of trade-off approaches Overview of companies involved in the research The four research phases with the applied research methods Excerpt from a KJ-diagram Negative experiences An Ishikawa diagram for an experience from figure General model of an experiment The experiments in this research at a glance Web application development classification Lifetime phases of a web application Fragmented knowledge (part A) vs. shared knowledge (part B) Qualitative trade-off tool Five steps in qualitative trade-off assessment A combination of two SWOT analyses The configuration decision process The relation between the five experiments to evaluate the release planning method The effect of introducing a new method Subprocess for managing changes in the development environment The contributions and their relation to the research approach and methods xiii

17 xiv LIST OF FIGURES

18 Abbreviations AHP ATAM AWE CBD CMC COVAP BBN EF FMEA IFM KJ diagram NRP NTNU OO OOHDM OVAC QFD PG PMA PSERM PWC RCA SEI SWOT analysis TPWC TTM XP UML Analytical Hierarchy Process Architectural Tradeoff Analysis Method Agile Web Engineering Component Based Development Computer-mediated Communication Cost-Value Approach for Prioritising Requirements Baysian Belief Network Experience Factory Failure Mode and Effect Analysis Incremental Funding Method Affinity diagram (devised by Jiro Kawakita) Next Release Problem method Norwegian University of Science and Technology Object Orientation Object-Oriented Hypermedia Design Method The Optimising Value and Cost in Requirement Analysis method Quality Function Deployment Planning Game Postmortem Analysis Planning Software Evolution with Risk Management Pair-Wise Comparison Root Cause Analysis Software Engineering Institute Strengths, Weaknesses, Opportunities and Threats analysis Tool Supported Pair-Wise Comparison Time-to-market extreme Programming Unified Modelling Language xv

19 xvi ABBREVIATIONS

20 Part I Research context and results 1

21

22 Chapter 1 Introduction This chapter gives a short overview of the research work reported in this thesis. It starts with the problem outline for the research, and presents the research background and context. The chapter also shows how the research questions, the research design, the claimed contributions, and the papers that have been published as part of this research, are related. 1.1 Problem outline Performing trade-offs in software development involves balancing several opposing and interconnected factors in such a way that the outcome of each factor is acceptable for a given situation and for a given set of stakeholders. The task of performing a trade-off requires an understanding of how the involved factors are connected, and how they affect the development project. Performing trade-offs under time pressure, and with incomplete and uncertain knowledge of the involved factors and potential consequences of each factor, adds to the complexity and difficulty of this task. This research addresses how trade-offs are performed in web application development [81, 69], and how trade-off practices can be improved, given the development practices applied in typical web application development projects. This research focuses on web applications that are developed with a strong emphasis on time-to-market, and in competition with other web applications. While some web applications are developed in a traditional way, where web technology is only used for convenience, other web applications are developed with a strong emphasis on time-to-market and within a competitive context. In the latter case, the use of web technology enables development practices that focus on short time-to-market, rapid changes and continuously deploying new versions of an application. Web applications that are developed this way face a strong competition, that comes from two directions: (1) the competitors are only a mouse click away, since their products are deployed on the Internet as well, and (2) every competitor would like users to return to their web application, 3

23 4 CHAPTER 1. INTRODUCTION since many web applications are competing for the same users. Thus, developing web applications has two top priorities: to meet the users expectations and to develop applications within the time constraints with respect to competing web applications. The priorities, however, are in conflict with each other and leave the developers with the task of performing trade-offs between time-to-market and quality issues of an application. An informed trade-off depends on the developers abilities to assess the consequences of their choices and to negotiate between the conflicting priorities. Web application development is characterised by (1) small development teams, (2) many small software deployments, (3) involvement of several stakeholders from non-technical fields, (4) informal and anecdotal specifications or requirements, and (5) short deadlines. Developers depend on their domain knowledge when developing web applications since these development practices involve mainly tacit knowledge. These development practices may also be found elsewhere. However, what makes web application development special is the intensity with which these practices are applied [81]. Typical trade-offs in web application development involve factors such as time-to-market, functional requirements and quality attributes. Examples include application scalability (how many users can use an application simultaneously), portability (how many browsers are supported) and performance (how long does a user have to wait for a response). The outcome of such trade-offs has an effect on when new functionality is deployed and to what degree the users expectations are met. Both priorities time-to-market and quality have their supporters among the stakeholders of an application. Each stakeholder seeks the fulfilment of his needs and wishes that corresponds to his interests and goals. Hence the conflict between these two priorities will also be found between the stakeholders. The problem is to facilitate a resolution of the conflicting priorities between the stakeholders and to negotiate a way to balance the factors. This has to be done without a precise specification of the quality attributes involved, without explicit knowledge about the consequences of the available options, and on the fly. Without a precise specification of requirements and the lack of explicit knowledge about the development activities, consequences of candidate choices can only be described and compared imperfectly [117]. To address this problem this research deals with two subproblems. First, can the stakeholders tacit knowledge about the development activities be converted into explicit knowledge [76]? The stakeholders have their opinions, expert judgement and experiences. How can this knowledge be expressed and used to assess different options of a trade-off? Second, can this knowledge be shared in such a way that the total knowledge reflects the views of all stakeholders? By reducing the expectation of one factor from optimal to something that is acceptable, the outcome of other factors can rise to something that is acceptable to the other stakeholders. Can this give and take between the stakeholders on their choices for each factor result in trade-offs with an outcome that in the end is more acceptable to a majority of stakeholders than the optimal fulfilment of a single factor is [33, 106]? This research focuses thus on expressing the stake-

24 1.2. RESEARCH CONTEXT 5 holders tacit knowledge and on sharing all stakeholders knowledge to perform a trade-off between two top priorities in web application development. 1.2 Research context The research in this thesis is part of the WebSys project [100], a research project financed by the Norwegian Research Council. The objective and the research goals of the WebSys project [100] are... to provide a better overall understanding of the trade-off between time-to-market and reliability in developing Web-based systems. This understanding will manifest itself as better methods and standard processes that can be applied, tested and refined through empirical work in Norwegian software companies. By this, we will gain insight into and improve the companies development processes. Main goal: To develop guidelines for the industrial development of timely and reliable Web-based systems. Sub-goals: G1 Better understanding of the software process and related technologies on how to make trade-offs between time-to-market (TTM) and reliability. G2 Contribute to better methods for Web-based systems development where reliability and TTM need to be considered together. G3 Dissemination and exchange of the gained knowledge, with a strong international component. The project description calls for co-operation with Norwegian software companies. Contact was made with eight companies for either an interview or a workshop on web application development. This provided this research with an understanding of industrial web development. The results of this PhD research have been published at international workshops and conferences. In addition, an international workshop on trade-offs was organised by the Websys project in Research questions The research questions for this research have been formulated incrementally. An initial set of research questions was formulated before the research started. During the course of the work they have been revised and influenced by the results and feedback of this research. The aim of this research is twofold: to understand how web applications actually are developed (and in particular how

25 6 CHAPTER 1. INTRODUCTION trade-offs are performed), and to improve the development practices in performing trade-offs of this type of software development. This has led to four research questions: RQ 1: What characterises Web Application Development? RQ 2: How are trade-offs performed in web application development, and how are trade-offs performed that involve time-to-market as a factor? RQ 3: How can trade-offs in web application development be improved? RQ 4: How is knowledge transferred within web development teams? 1.4 Research design The research questions are dealt with using several research phases and methodologies. The research is done in co-operation with companies that are developing web applications, and therefore web development within these companies can be studied and real world problems identified. This led to a research design with four phases. Each phase addresses at least one of the research questions, and contributes to the results of this research. An overview is given in figures 1.1 and 1.3, and the research approach and research phases are described in sections 3.1 and 3.2. This research is iterative, in the sense that the phases are used a bit back and forth, and not executed in a sequential manner. The results from one phase may affect not only the next phase but also the previous phase. A brief description of the research phases is given below. 1. Data collection phase Here, data are collected on industrial web development, and trade-off practices used for time-to-market are collected. Another objective for this phase is to identify problems and opportunities for co-operation with the companies. The problems have to be related to the companies development and trade-off practices in web application development, and should be feasible for finding improvements. Relevant research methods: The qualitative research interview and Postmortem Analysis. 2. Analysis phase Here, we build up an understanding of industrial web application development and of the factors that influence the performed trade-offs. Other relevant issues are to understand how development practices emerge, and on what assumptions trade-offs are made. Relevant research methods: Grounded theory, and applying a framework to understand how development practices emerge [98, 68]. 3. Construction phase Here, some of the identified problems and cooperation opportunities are selected for further work. The objective is to improve the development and trade-off practices and to solve the problems,

26 1.4. RESEARCH DESIGN 7 Research Phases 1. Data Collection 2. Analysis 3. Problem Solving 4. Validation Research questions RQ 1, RQ 2 RQ 1, RQ 2 RQ 3, RQ 4 Contribution C1 C1 C1, C2, C3, C4, C5 C4 Research methods Interview Post Mortem Analysis Literature study Grounded Theory Root cause analysis Publications None. P3: Web Application Development and Quality Observations from interviews with companies in Norway Other Initial research questions Identification of problem situation Research question revised Analysis of problem situations P1: The use of Trade-offs in the Development of Web Applications P4: A decision modelling approach for analysing requirements configuration trade-offs in time-constrained Web Application Development P5: Trade-off s in Web Application Development How to assess a trade-off opportunity P8: Managing change in software development More general problems are selected, not specific company problems RQ 3, RQ 4 Experimentation Interview P2: Trade-off Analysis in Web Development P6: Knowledge sharing through a simple release planning method for web application development P7: An experiment with a release planning method for web application development Student experiments, using scenarios constructed from problem situations Figure 1.1: Research overview

27 8 CHAPTER 1. INTRODUCTION by providing methods and guidelines. More general problems are selected, not specific company problems. The construction of new methods and guidelines is inspired by the analysis from the previous phase, as well from theories, concepts and other methods, such as quality function deployment [3], value-based software engineering [13] and the Delphi method [65]. A goal for method construction is that the method must be easy and intuitive to use by developers, and that there is no need for intensive training. 4. Validation phase Here, the above constructed methods and guidelines are validated. This is done by means of student experiments. In each experiment the subject has to solve a task using one of the methods. Each task is part of a scenario constructed using the collected data from real life projects. 1.5 Papers The following eight papers have been published as part of this research. Figure 1.1 shows how these papers relate to the overall research design and the research questions. Papers P1 P8 are included in part II of this thesis. A short description of the papers is shown below: P1: Sven Ziemer and Tor Stålhane: The use of Trade-offs in the Development of Web Applications. In Proc. First International Workshop on Web Quality (WebQuality 04) at 4th International Conference on Web Engineering (ICWE 2004), July 2004, Munich, pp Also printed in Maristelle Matera and Sara Comai (Eds.): Engineering Advanced Web Applications, In Proc. Workshops in Connection with the 4th International Conference on Web Engineering (ICWE 2004), Munich, Germany, July, Published by Rinton Press, ISBN This paper presents a trade-off tool inspired by the Quality Function Deployment method (QFD) [3]. The tool uses a simple qualitative assessment to evaluate the consequences of an option in a decision, with respect to other requirements and quality attributes. This makes it possible to compare different options and to perform a trade-off, knowing the consequences. P2: Sven Ziemer, Tor Stålhane and Magnar Sveen: Trade-off Analysis in Web Development. International Conference on Software Engineering, 3-WoSQ: Proceedings of the third workshop on Software quality, pp , ACM Press, In Xiaoqing (Frank) Liu, Yan Sun, Gautam Kane, Yuji Kyoya, and Kunio Noguchi (Eds.): Proc. Third Workshop on Software Quality, held at the 27th International Conference on Software Engineering (ICSE 05), St Louis, Missouri, May 17, Published by ACM, ISBN , pp

28 1.5. PAPERS 9 This paper reports on a student experiment to evaluate the trade-off tool presented in paper P1 [124]. The experiment is a one factor experiment with two treatment design, comparing the trade-off tool with an ad-hoc approach to select an option for a decision. The results of the experiment showed that the groups using the trade-off tool experienced fewer difficulties in making the selection, fewer conflicts between the stakeholders, and a better consensus on the final decision. However, the groups using the trade-off tool also experienced their communication as less constructive. Taken together, the experiment showed that performing a trade-off became easier by using the provided trade-off tool. P3: Sven Ziemer and Tor Stålhane: Web Application Development and Quality Observations from interviews with companies in Norway. In José A. Moinhos Cordeiro, Vitor Pedrosa, Bruno Encarnação, and Joaquim Filipe (Eds.): Proc. Second International Conference on Web Information Systems and Technologies: Internet Technology / Web Interface and Applications (WEBIST 2006), Setúbal, Portugal, April 11 13, 2006, Vol. 1. Published by INSTICC Press, ISBN , pp In this paper, results from interviews on development practices with seven companies that develop web applications are reported. The results presented here are on development practices for requirements, quality issues and development process. The interviews revealed that most companies did not have a clear trade-off strategy, and that the most important quality issues for the developers were related to the user s experience of using an application. P4: Sven Ziemer, Pedro R.F. Sampaio and Tor Stålhane: A decision modelling approach for analysing requirements configuration tradeoffs in time-constrained Web Application Development. In Proc. Eighteenth International Conference on Software Engineering and Knowledge Engineering (SEKE 2006), San Francisco, 5-7 July Published by Knowledge System Institute Graduate School, Skokie, IL, USA, ISBN , pp This paper presents a release planning method for web application development. The method uses a value based assessment of requirements. It gives guidance on the construction of candidate releases that fit within the time constraint and supports the decision on the final release. The method is consensus driven, and facilitates knowledge sharing between the involved stakeholders. P5: Sven Ziemer and Tor Stålhane: Trade-off s in Web application development How to assess a trade-off opportunity. In José Cordeiro and Slimane Hammoudi (Eds.): Proc. Third International Conference on Web Information System and Technologies (WEBIST 2007), Barcelona, Spain, March 3 6, Published by INSTICC Press, ISBN , 8p.

29 10 CHAPTER 1. INTRODUCTION Trade-offs with respect to time-to-market often involve quality attributes. With no specification it is the developers task to assess what are acceptable levels for important quality attributes of a web application. However, these acceptance levels will change, and with the informal development practices found in most companies, it is hard to be aware of the change in the end-users expectations. The trade-off tool presented here uses a series of SWOT analyses [45] to assess the options in a trade-off, and uses a cross analysis to compare them with the current context of an application. P6: Sven Ziemer and Ilaria Canova Calori: Knowledge sharing through a simple release planning method for web application development. In Daniel Cooke et al. (Eds.): Proc. Nineteenth International Conference on Software Engineering & Knowledge Engineering (SEKE 2007), 9 11 July 2007, Boston, USA. Published by Knowledge System Institute Graduate School, Skokie, IL, USA. ISBN , pp This paper reports on a student experiment to evaluate the release planning method for web application development presented in P3 [123]. The experiment uses a one factor with two treatment design, comparing the release planning method with an ad-hoc release planning approach. After the experiment, the students fill in a post-experiment questionnaire, with questions regarding knowledge sharing, understanding, consensus, requirement prioritisation and stakeholder satisfaction. The results show that the control groups using the ad-hoc approach performed better on knowledge sharing and understanding, while the treatment group using the release planning method performed better on the remaining factors. P7: Sven Ziemer and Ilaria Canova Calori: An experiment with a release planning method for web application development. In Pekka Abrahamsson, Nathan Baddo, Tiziana Margaria, and Richard Messnarz (Eds.): Proc. 14th European Conference on Software Process Improvement (EuroSPI 2007), September 2007, Potsdam, Germany. Published in Springer LNCS 4764, ISBN , ISSN , pp This paper reports the results from four experiments with the release planning method presented in P3 [123]. The experiments are planned and operated as the first experiment in P6 [122]. The experiments are planned to test how experience in using the release planning method affects knowledge sharing and understanding, and what effect an improved communication style in using the release planning method has on knowledge sharing and understanding. The results show that in both cases the treatment group using the release planning method improved in knowledge sharing and understanding, compared to the control group that used the ad-hoc approach. P8: Sven Ziemer: Managing change in software development. In: Lech Madeyski, Miroslav Ochodek, Dawid Weiss and Jaroslav Zendulka

30 1.5. PAPERS 11 (ed.): Software Engineering in Progress. In conjunction with 2nd IFIP Central and East European Conference on Software Engineering Techniques (CEE-SET 2007), Work-in-progress session, October 2007, Poznan, Poland. Published by Wydawnictwo Nakom, Poznan, Poland, ISBN , pp In this paper, a process to manage the change that occurs in the context of web development is presented. The process presented in this paper aims at improving the trade-off practices, by facilitating a deeper understanding of the involved factors over time. This understanding is achieved by offering simple activities that plan, perform and evaluate a trade-off. A knowledge base for the project is created, and lessons learned are made available for other stakeholders. In addition, this research has contributed to two other publications. Papers P9 and P10 are not included in this thesis, since they do not report on research results that answer any of the research questions. For reference, we include a small summary of the two papers here. P9: Glenn Munkvold, Gry Seland, Siv Hilde Houmb and Sven Ziemer: Empirical assessment in converging space of users and professionals. In Proceedings of the 26th Information Systems Research Seminar in Scandinavia (IRIS 26), Helsinki, August 9-12, 2003, 14 p. Implementing generic standardized systems in local practices often implies a process of change involving people from various domains and communities. In this paper we put particular emphasis on the new emerging community of people this implies. Based on experience from the Software Engineering and HCI domain, we discuss a methodology supporting these new challenges. We use Communities of Practice as a possible conceptual framework, and action research as a methodological approach to better understand the ongoing situated activities and processes in practice. P10: Ilaria Canova Calori, Tor Stålhane and Sven Ziemer: Robustness analysis using FMEA and BBN Case study for a web-based application. In José Cordeiro and Slimane Hammoudi (Eds.): Proc. Third International Conference on Web Information System and Technologies (WEBIST 2007), Barcelona, Spain, March 3 6, Published by INSTICC Press, ISBN , pp Time pressure and quality issues represent important challenges for those who develop web-based systems. The ability to analyse a system s quality and implement improvements early in the development life cycle is of great practical importance. For our study we consider robustness as a critical quality issue. Our objective is to propose a general framework for conducting robustness analysis of web-based systems at an early stage of software development, providing a tool for evaluating failure impact severity and supporting trade-off decisions during the development process.

31 12 CHAPTER 1. INTRODUCTION 1.6 Contributions The contributions of this research are assigned in three groups: Understanding web development practices, trade-off methods for web application development, and facilitation of knowledge sharing between stakeholders in web application development. An overview of the contributions and their relation with the published papers is shown in figure 1.2. Figure 1.3 shows in what order the different activities have been performed and how they relate to the publications and contributions. Contribution Development practices Trade-off methods Knowledge sharing Paper C1: Development practices X P3 C2: Trade-off strategy X P1 C3: Qualitative assessment of trade-off options C4: Enabling knowledge sharing X X P1, P2, P4, P5 X P6, P7 C5: Knowledge management X P8, P5 Figure 1.2: The contribution of this work C 1 Web Development Practices A description of such practices in eight Norwegian companies, based on interviews and Postmortem Analyses with the developers and managers in the companies. The development practices include trade-offs between time-to-market and quality attributes. C 2 Trade-off strategy for web application development Strategy based on the principles of (a) creating awareness for trade-offs, (b) balancing short and long term wishes, (c) sharing stakeholders knowledge, and (d) using qualitative assessment of options in trade-offs. The strategy is to utilise trade-offs by balancing stakeholders interests, focusing on the opportunities, and by using the freedom of choice to find the means to achieve the opportunities. C 2.1 Subprocess for change management (same as C 5.1) The research provides a subprocess for managing the change in web application development, using the trade-off strategy mentioned above. C 3 Qualitative assessment of trade-off options and decision support Here, several ways of assessing the options of a trade-off qualitatively are used. This enables prioritisation of trade-off options and gives valuable support for decision making in situations with mainly tacit and uncertain knowledge. C 3.1 Trade-off tools for assessing options Tools used to assess trade-off options under consideration qualitatively.

32 1.7. THESIS STRUCTURE 13 C 4 Trade-off methods for enabling and facilitating knowledge sharing Several factors that contribute to enable such sharing are tested. Knowledge sharing builds up a common understanding between stakeholders, and contributes to an assessment of trade-off options reflecting all stakeholders views. C 4.1 Release planning method Enables knowledge sharing between stakeholders, using qualitative assessments and a consensus driven decision process. C 5 Knowledge management for web application development A learning loop to facilitate learning and to create a knowledge base to assist in trade-offs. The accumulated knowledge is used to manage web application projects in a rapidly changing context. Over time this will increase the confidence in the available knowledge. C 5.1 Subprocess for change management A subprocess for managing change in web application development, which implements a learning loop and accumulates a knowledge base for web development projects. 1.7 Thesis structure This thesis is divided into two parts: Part I presents the research questions, research results, and the evaluation of the research, as well as an introduction to the field, and puts the research into context. It covers chapter 1 6. Part II contains papers P1 P8 listed in section 1.5. They provide detailed results and discussion of the individual research activities. An overview of the chapters in part I is given below: Chapter 2 Related Works This chapter covers related works in Web Engineering, release planning, knowledge sharing, trade-off methods and decision making. Chapter 3 Research Methods This chapter presents the research approach, research phases and research methods that have been applied in this research. Chapter 4 Results The results of this research are presented, covering a description of development practices, a release planning method for web development, and a process for managing change in web development. Chapter 5 Evaluation The research in this work is evaluated, by revisiting the research questions and by linking the research to the research goals. Chapter 6 Conclusion and further work The final chapter of part I concludes the work done and points out directions for future research.

33 14 CHAPTER 1. INTRODUCTION Data Collection Phase Analysis Phase Construction Phase Validation Phase Activity 1: Interview I PMA I Activity 2: Analysis I P1 Activity 3: Qualitative trade-off tool C1 C2 C3 P5 C2 C3 Activity 8: Trade-off assessment P2 Activity 5: Student experiment (E 0) Activity 4: Interview II PMA II Activity 6: Analysis II C1 P8 Activity 11: Change process C3 P6 C4 Activity 9: Student experiment (E 1) Activity 1: Interviews with 4 companies and a PMA with 1 company Activity 4: Interviews with 8 companies and a PMA with 1 company Explanation: Px = Paper (see section 1.5) Cx = Contribution (see section 1.6) P3 Timeline = Influence between two studies= P4 C3 C4 Activity 7: Release planning method C4 C5 P7 C4 Activity 10: Student Experiment (E 2.1 E 3.2) Figure 1.3: The research activities of this research and their relation to the contributions and publications

34 Chapter 2 Related work This chapter gives a view on the related works for web engineering, trade-off analysis methods, release planning methods, decision making and knowledge sharing, and thus shows the research context that this work took place in. These fields are related to work in this thesis and have inspired and guided the work done for this thesis. An evaluation of this work s results with respect to the related work will be given in section Web Engineering The aim of studying the literature on web engineering is to get an understanding of how web applications are developed and what development practices are applied to do so. Another point of interest is to understand how development methods for web applications are constructed to support the distinct characteristics of web applications and development practices to develop them. Web engineering is a subdiscipline of software engineering and studies the development of web applications and systems. A definition of this discipline was given by Murugesan et al. [73]: Web engineering is the establishment and use of sound scientific, engineering and management principles and disciplined and systematic approaches to the... development, deployment and maintenance of high quality Web-based systems and applications. Not all researchers view the development of web applications as a separate discipline, but rather as a new application domain of software engineering [52]. However, all agree that web application development has some distinctive characteristics, and that a systematic and disciplined approach to develop web applications is needed. All experts agree to avoid ad-hoc development approaches which result in a crisis of web application development [7]. Without a disciplined approach, web applications will neither deliver the desired quality 15

35 16 CHAPTER 2. RELATED WORK nor be developed on time. Web application development calls for continuous updates and refinements, which constitute an evolution of the application without specific releases (as opposed to traditional software development). This makes web application development behave like gardening. The gardening metaphor raises the question if an engineering approach is appropriate for web application development. Murugesan et al. [73] believe that engineering principles need to be adapted to the web environment. Developing web applications is different from developing other software applications, for a number of reasons. First, the web is both an application and a delivery medium. Web applications have a greater focus on the look and feel, and incorporate multimedia at various degrees in presentation and interface. This results in a greater bond between art and science. Finally, web applications are developed within a short time frame, thus making it difficult to apply the same level of formal planning used in traditional software development. Web engineering is a multidisciplinary field, combining traditional software engineering disciplines such as requirement engineering, evolutionary system development and security, with disciplines such as graphical design, copyright and intellectual property rights, multimedia and hypermedia [28, 73] Web development practices Development practices in web application development are of particular interest to this research. Several studies on web development practices are found in the literature and are briefly presented here. In their study of web application development Ramesh et al. [81] focus on understanding the features that characterise web application development. The study involves nine companies developing web applications in the US. The main finding of this study is that there are special web application development practices and that they are caused by three factors: demand for rush to market, operating in a different kind of market environment, and the lack of experience in developing such products. The resulting web application development practices are characterised by parallel development activities, release orientation, tool dependence, customer involvement, prototyping, fixed architecture, components, ignored maintenance, and tailored methodology. Some of the practices are further described and analysed in [9]. The characteristics of web application development practices are not necessarily unique for web application development, and may be found in traditional software development as well. However, the intensity with which they are applied is unique for web application development. Another issue raised by this study is that of a different development culture. The culture of web application development appreciates less structure, smaller teams size and diverse team compositions. The development culture is, however, evolving together with the product, and when the market demands quality and maturity over speed, the development culture begins to resemble traditional software development. A survey of web engineering in practice by McDonald et al. [70] identifies similar practices. In this survey, web application development practices are

36 2.1. WEB ENGINEERING 17 identified and described as a first step to developing a web application development process. Such a process has to address practices such as short development life-cycle times, multidisciplinary teams and small development teams working in parallel on similar tasks. Yusop et al. [117] performed a study on five web application development projects with companies from Singapore. In this study they addressed the impact of non-functional requirements in web application development projects, and found that non-functional requirements are not specified sufficiently. This is often due to uncertainty about the customer s needs, lack of time and lack of knowledge of the importance of non-functional requirements. Functional requirements are, however, specified in a structured manner. Two non-functional requirements that are important to the companies in this study are security and integration with other systems. As a result of the insufficient specification of non-functional requirements, developers have to rely on previous experience in developing similar systems. In a survey of Jordanian web companies conducted by El Sheikh et al. [95], it is shown that small web application development companies find it hard to apply web engineering practices such as web metrics, standards and procedures, control of the development process and organisation issues. The survey was carried out by using the Software Best Practice Questionnaire from ESI [48]. No explanation other than a difference between European companies and small web application development companies in Jordan is given. However, this study stresses the same points as the studies cited above, such as small development teams, tight time constraints and lacking experience in developing products under these circumstances. The development of web application has some distinctive characteristics, as shown by these surveys. These include small development teams, tight time constraints, short development cycles with frequent updates, a multidisciplinary nature of development activities, and a development culture that appreciates less structure. None of these characteristics are exclusive for web application development, and can be found in other application domains as well. However, what makes web application development different from other application domains is the intensity with which these practices apply [81] Web development methods and tools A number of software development methods for web application development have been constructed to address the special characteristics of web applications. This is the case for web design methods that address the special ways of designing web applications. One such method is the Object-Oriented Hypermedia Design Method (OOHDM) [93, 85] that uses a five-step process to build a web application. This method applies models for both the conceptual design, navigational design and the interface design, using UML with some small extensions. Other design methods are HDM [37], W2000 [6], WebML [18] and Autoweb [35]. A commonality is that web applications are generated from platform independent models. These models focus on the web interface of the applications and