Software Quality Assurance in Agile, XP, Waterfall and Spiral A Comparative Study S. Vijayakumar vijsy003@students.unisa.edu.au School of Computer and Information Science University of South Australia, Adelaide Software quality engineering is essential to ensure that the developed software is of high quality. Quality assurance is an essential process in the development of any software. In order to achieve successful software products, quality is assessed in two ways. Firstly, quality is assessed during the development of the software and secondly, once the software is developed, it is assessed to check whether quality attributes are met. This paper focuses on the steps taken by Waterfall, Spiral, Agile, XP models to ensure quality. This comparative study proposes a de-facto standard to ensure quality using various software development models. S I. INTRODUCTION OFTWARE products developed are successful only if it assures high quality on delivery. Each application developed in a software project has its own quality issues. In order to overcome these issues the software development project must define its quality attributes initially during the planning phase [1].These operations depend on cost reduction but at the same time enhancing productivity and increasing income [2]. Quality according to the American National Standards Institute (ANSI) and the American Society for Quality (ANQ) is the features and characteristics of a product or service that impact the ability to assure given requirements. Quality frameworks have been developed by many researchers. For example, Garvin [3] developed a product quality framework. 1. Reliability: Prevents malfunctioning. 2. Conformance: Meets specified standards. 3. Features: Complements basic qualities. 4. Durability: Products lifetime measure. 5. Serviceability: Easy repair. 6. Performance: Basic working qualities. Parasuraman et al. [4] developed a five dimension service quality framework. 1. Tangibility: Equipments look and operation. 2. Reliability: Ability to perform accurately. 3. Responsiveness: Service support to customers. 4. Assurance: Builds trust among customers. 5. Understanding customer needs. This paper examines the software process models. This will be used to assist in choosing suitable models for developing high quality software. Following this, section II focuses on four software process models namely Waterfall, Agile, extreme Programming (one of the agile models) and Spiral. This section describes the phases of software development. Following this, section III describes the software quality development and assurance practices that are common in all the software development process models. Following this, section IV evaluates these process models and describes the quality assurance activities. Following this, in section V the author suggests a de-facto standard for the development of quality software. Finally the results are summarized in section V. Followed by the Employability skills in the appendix. II. BRIEF FOCUS ON SOFTWARE PROCESS MODELS A. Waterfall Model The software engineering community has introduced a number of methodologies to develop software since late 60s. As a result of research and continuous refinement, the models have now reached a stable level [5]. Each of these models addresses different developmental conditions and issues. The most traditional software development model is the waterfall model. This model could be used in both small and complex engineering projects and has been effective in complex projects [5]. The software development lifecycle is divided into five stages as shown in figure 1. In waterfall model the developmental activities include 1. Requirement definition 2. System and software design 3. Implementation and unit testing 4. Integration and system testing 5. Operation and maintenance [6]. The author has chosen this model because of its effectiveness in complex projects in addition to being traditional. Since it is linear, the quality assurance processes are clearly presented. Even though the waterfall model is successful in complex projects, it is unable to face changing requirements during its developmental process [5]. On the other hand, agile methods overcome the problems faced in the waterfall model.
2 B. Agile model Fig. 1: Waterfall model [7]. Agile methods were developed to face changing requirements. Agile model is a nine stage process as shown in figure 2. It is an iterative, self-organizing, incremental and emergent group of software development process [8]. Agile models deliver products in a short period when compared to waterfall model because of the following characteristics [8]. 1. Iterative: The entire system is developed by the analyst and based on the requirements and the subsystems are updated. 2. Self-organizing: Based on the skills the members of the development project are divided into teams and they report to their management. 3. Incremental: Each subsystem is developed in such a way that subsystems could be added to add functionality based on the requirements. 4. Emergent group: The project is based on increments, team and technology. The author has chosen this model because of its popularity of delivering products very quickly. The main aim of the author is to ensure whether the quickly delivered products meet the quality requirements. C. extreme Programming (XP) XP is based on the Agile model and is extensively used in small projects and ensures high quality products [9]. Agile methods like XP deliver products within a short period of time because of the techniques it follows. Some of the important techniques are 1. Simple planning 2. Short iteration 3. Early release 4. Frequent feedback from customers. The success of XP in delivering high quality products lies in the involvement of the end user as well as testing and reduced design efforts. A XP model is chosen when the team is small (consisting of around twenty developers) and there is no prior documentation of the project [9]. XP model is a five stage process [10] as shown in Figure 3. 1. Exploration: During this phase it is ensured whether the application could be developed using this model. The programmer estimates the time taken to develop the product and also experiment them based on their limitations. 2. Planning: In this phase the programmer and the customer decide on the date of delivery of the product. The time taken is approximately between two to six months. Furthermore this period is divided into iterations each with duration of four weeks. Test cases are defined in all iterations. 3. Productionizing: At this stage importance is given to ensure performance. Based on the feedback the production activities are slow down and risk management steps are taken. 4. Maintenance: This phase deals with adding new functionality to the already developed portion of the application. This phase makes sure the software satisfies all requirements and functions well. 5. Death: The final phase of the XP model is death. The model final dies when it is not able to satisfy further requirements of the customer. Fig. 2: Agile model [8]. Fig. 3: XP model [10].
3 D. Spiral model The spiral model is a combination of the waterfall and prototyping model [11]. A Spiral model can accommodate other software development process models in its stages based on which process suits its development. The Spiral model consists of four phases as shown in Figure 4 [11]. 1. Determine Objectives: In this phase the objective of the project, function, alternate methods to implement, constraints on choosing the alternate methods are identified. Communication between the developers and all the internal and external users takes place. 2. Identify and resolve risks: A preliminary design is laid for the system. This is the most important phase as it evaluates all the available alternatives and selects the best available method to develop a project at minimum costs. This phase identifies the possible risks that could be encounter during the development. If there is any uncertainty with the requirements then prototyping is used to find out the solution. 3. Development and test: The prototype is developed taking into consideration each characteristic that has to be present in the final product. This prototype represents the characteristics of the final product to be developed. 4. Plan the next iteration: The second prototype is developed after evaluation of the first prototype. The second prototype considers the strengths, weakness and risks of the previously designed prototype. Having considered the above factors the second model is planned, designed and tested. Fig. 4: Spiral model [11]. III. QUALITY ASSURANCE Software Quality Assurance ensures quality by examining each step of development during the entire process. Based on an analytical bottom-up approach certain quality development and assurance practices have been found to be common in these software development models [12]. 1. Iterative Software Development: The developmental process is iterative and incremental in order to maintain high quality standards. Based on the results obtained, the iterations have to be refined. The systems developed must be capable of accommodating subsystems. Each iteration cycle must include all the phases such as analysis, design, implementation, testing and deployment [8]. Iterative model is capable of dealing with changing user requirements. This model forces immediate feedback from the customers which further leads to quality improvement. Agile models are strictly iterative and hence demands components to be built on a daily basis. It also forces the developers to check for errors frequently and fix them within short period of time [13]. Integration of the components built must also be done daily. In order to maintain high quality, a lot of planning is required [14]. 2. Quality: Quality of the process must be considered as an important objective. The quality goals are documented by both the project team as well as the customer so that they can be achievable [14], [15]. The Spiral model defines the target of the project at the start and ensures their fulfillment at the completion of the project. 3. Continuous verification of Quality: Every change done to the process has to be documented. Only if the workflow is clearly defined with all changes been made, risk management would be easy. The quality of the product has to be tested extensively by means of both internal and external acceptance tests. The external test with the customer has to be done throughout the developmental phase. In Agile and Spiral models, quality is verified frequently through internal and external tests. This is essential to develop high quality products [14]. 4. Customer requirements: The requirements of the customer have to be documented not only in the beginning of the project but also document the changing requirements throughout the process. It is very essential to ensure that the requirements of the customer are satisfied to assure quality. In Agile, the customer involvement is high when compared to various traditional models [16]. 5. Architecture driven: Architecture has a major impact on software quality. The main reason for this is because of integration. Software development in recent time has time and cost constraints and hence reuse has become very popular [8], [9]. Hence an architecture that is well defined will allow integration and reuse. In Agile the architecture is well defined and hence ensures quality. 6. Focus on teams: The entire team is responsible for the success or failure of the project. In such teams motivation is an important factor that would lead to the development of successful projects. For a software development project to be successful a team has to be well defined, with tasks clearly defined and good communication between them. In agile based on the skills the project members are divided into teams and their tasks are well defined [13].
4 7. Pair programming: Pair programming is another criterion that is essential for high quality software. XP demonstrates pair programming in which one person implements the current method while the other works on integration. Better solutions are more likely in pair programming because each individual has a different perspective and hence compliments others in the team to solve the problem. Pair programming is a dynamic quality assurance technique in XP [17]. 8. Tailoring with restrictions: The core elements are developed in such a way that it could be added to other complex projects [8]. The number of core elements should be low in order to ensure quality. Spiral model focuses on incremental and integration. 9. Configuration and change management: All the objectives have to be documented along with the changes being made each time. A well defined configuration and change management ensures high quality in the developed project [18]. Agile method gives utmost importance to documentation. 10. Risk Management: Risk management is very essential to ensure quality. Risk management provides an opportunity to act rather than reacting to problems. Effective management of risk cuts down cost of rebuilding the entire product at later stages of development [11]. Spiral model mitigates risk in its exploration phase. IV. EVALUATION OF THE QUALITY SUPPORT IN AGILE, XP, SPIRAL AND WATERFALL In this section the author makes a comparative study of the Quality assurance activities present in Agile, XP with Waterfall, Spiral. The reason for the author s choice of comparing Agile, XP with Waterfall, spiral is because of their quality assurance activities that are found to be similar. A. Quality Assurance (QA) in Agile, XP: In the Agile model the phases overlap. The exploration phase in XP deals with risk management by exploring the limitations and experimenting to find out the risks involved. This determines whether the product could be developed using this model in a cost effective manner within the time constraints. This activity ensures quality as it identifies risks and eliminates them. The development phases in the agile model are similar to the waterfall model. XP uses dynamic technology [17]. The two main differences between the Agile and Waterfall models are: 1. Some phases in agile have both developments as well as QA activities [17]. 2. The output produced is immediately sent to assure quality even if the output is small. This is the main reason for projects developed using agile to be delivered quickly [17]. Some of these which support development as well as quality assurance are a. Formal architecture is avoided and system metaphor is used instead. The metaphor satisfies two main purposes. Firstly, it acts as a bridge between the developers and the users Secondly it contributes to the team s development [13], [19]. b. Architectural spike helps the developers and users to evaluate the architecture [20]. c. On-site customer feedback: The customers support the development team to verify their requirement and modify them [16]. Customer involvement is more in agile when compared to other models. d. Create unit tests [16]. e. Refactoring is a technique of altering the internal structure to enhance quality without affecting its peripheral behavior. Each modification done is called refactoring but does not have much effect. A number of alterations produce major restructuring. Each refactoring is kept as small as possible so that the possibility of getting it wrong is less and its functionality is checked after each refactoring [15]. Refactoring also adds to code inspection. f. Pair programming is used in agile in which two programmers work simultaneously on the same code. When they work in pairs each programmer has a unique perspective of looking at problems and hence capable of solving those easily thereby reducing defects and improving quality [14]. Pair programming is termed as continuous code inspection [14]. g. Stand up meeting [16]. h. Cyclic Redundancy Check (CRC) simplifies problems [16]. i. Continuous integration: In agile model codes, integration is not done once or twice but instead the code is kept integrated all the time. Continuous integration detects errors early [15]. j. Acceptance testing is carried out once unit test cases have been cleared. This is a vibrant quality assurance technique [7]. k. Customer feedback: Customer involvement is very high and hence this is one of the most important characteristic of agile model [17]. Fig. 5: Agile development methods and QA techniques [17].
5 The Quality Assurance techniques are based on: a. Dynamic QA techniques applied very early (acceptance testing). b. Developers responsible for ensuring quality (pair programming, code inspection, refactoring, standards coding). c. Validation of products (continuous integration, customer involvement, small release) [17]. B. Quality Assurance in Waterfall, Spiral model: Spiral model is iterative in nature [18] and focuses on risk mitigation [11]. This feature differentiates it from other models. Since it identifies and removes risk, this could also serve as one of the major factors assuring quality. Spiral model can accommodate other models and it usually accommodates waterfall model as it is the simplest model for the development of software. Waterfall model uses static and dynamic techniques to assure quality [16], [17]. In the waterfall model, the activities at each stage are well defined. The output from one stage is the input for the next subsequent phase and hence new phases cannot begin until their predecessor phases have been successful. The requirements are defined initially and using reviews, prototyping and model validation, they are evaluated and verified. Quality assurance practices are embedded in each phase as shown in figure 6. a. Customer involvement: The customers only define their requirements initially but are not involved during the developmental phase. Some milestone reviews may be set up but the customer involvement is very less in waterfall when compared to other models such as agile [19]. b. Continuous integration: There is no continuous integration and hence bugs are detected in later stages of development and integration is not frequent as it is in agile [16]. c. Acceptance testing: Acceptance tests are done only once during the later developmental stages. Acceptance testing is one of the dynamic quality assurance techniques [16]. V. PROPOSED STANDARD On comparing these processes, there are some practices found to be common in all the models. These practices could be considered to form the de-facto standard. A de-facto standard could be defined as something that is so dominant that everybody seems to follow it like an authorized standard. De-facto standard for quality support in software development process practices 1. Iterative software development. 2. Continuous verification of quality 3. Customer requirements 4. Architecture driven 5. Focus on teams. Including these software practices into the software development process assures development of high quality software [12]. VI. EMPLOYABILITY SKILLS The importance of quality assurance and the steps taken to assure quality has been discussed in this paper. In order to work as a member of a software development team, IT professionals must possess certain employability skills. This section explains how these software assurance practices present in the development of software, helps to enhance employability skills. Developers who possess these skills have better chances of employment. A. Communication: Effective communication plays a vital role in the software development industry. Developers initially communicate with the customers in order to know their requirements [14]. Developers working with these models will gradually improve their communication skills, as they will have a lot of chances to communicate with the users as well as other members of the team. B. Team Work: In Agile, the products are delivered quickly only because of teamwork. The development group is divided into teams and tasks are clearly defined. Only if there is sharing of knowledge and skills among the team members, quality products could be developed. In XP pair programming is one dynamic quality assuring activity [17]. Hence graduates who possess the capacity to work in teams will be able to deliver quality products. C. Problem Solving: Developers who work with these models must possess problem solving skills. During risk, it is the duty of the developer to solve it before the delivery of the product to the customer [17]. When the developers work in teams they develop this skill, when a problem is being solved in a team, each person has a different perspective of the same problem and hence will be able to solve problems efficiently. Fig. 6: Waterfall model and its QA activities [17]. D. Self Management: In order to ensure quality in these software models, the tasks are divided among the members and each of them have a specific role to perform [14]. Therefore self management skill
6 is essential to finish the task and deliver them within the specified time. When graduates work as developers they learn to manage themselves and thereby have increased chances of being efficient developers. E. Planning and Organizing: To deliver quality products the activities of the development team must be planned and organized [8]. Each member must possess these skills, as they are important to solve risks. Organizing activity includes proper documentation of all the activities carried out. Hence, people who have the habit of planning and organizing will have better employment prospects. F. Technology: In order to supply efficient products within a short period of time, it is necessary for the developers to be aware of the latest technologies and adapt them into their models. In certain software development teams, there are specific members who are responsible for adapting new technologies into their usual activity. G. Learning: Every individual in the IT industry must possess this skill of lifelong learning. As the technology keeps changing, it is necessary to learn them to keep in date with the current industry. H. Personal attributes/skills: Apart from the above mentioned employability skills, people working with these models should be responsible and committed towards their duty. In order to encourage developers, training and motivation programs could be conducted. Developers could be rewarded with incentives in order to encourage them. Thus, people working as part of a software development team possess these valuable employability skills. [4] A. Parasuraman, L. Berry and V. Zeithaml, Refinement and Reassessment of the SERQUAL scale, Journal of Retailing, vol. 67, no.4, pp. 420-450, 1991. [5] I. Sommerville, Software engineering, 6 th ed. Harlow, England; New York: Addison-Wesley, 2000. [6] A. Abran and J. W. Moore, "Guide to the software engineering body of knowledge: trial version (version 0.95)." Los Alamitos, CA: IEEE Computer Society, 2001. [7]Waterfallmodel[online].Available: http://2.bp.blogspot.com/_6s3- HNVqMy4/R5meiUT1A_I/AAAAAAAAAKs/ntjbIG944E/s320/waterfall+model.JPG. [8] M. Lindvall, V. R. Basili, B. Boehm, P. Costa, K. Dangle, F. Shull, R. Tesoriero, L. Williams, and M. V. Zelkowitz, Empirical findings in agile methods, in Proc. of Extreme Programming and agile Methods: XP/agile Universe, 2002, pp. 197-207. [9] M. Frank and M. Grigori, What You Always Wanted to Know about Agile Methods But Did Not Dare to Ask, ICSE 05, IEEE 2005. [10] B. Kent, Extreme Programming Explained ; Massachusetts: Addison- Wesley Professional, 2000. [11] W. B. Barry, A Spiral Model for Software Development and Enhancement, vol. 21, no. 5, pp. 61-72. May 1988. [12] W. Zuser, S. Heil and T. Grechenig, Software quality development and assurance in RUP, MSF and XP: a comparative study, in Proc. Third Workshop on Software Quality, St. Louis, Missouri, 2005. 3-WoSQ. ACM, New York, NY, pp. 1-6. [13] E. T. James, H. James, How Useful Is the Metaphor Component of Agile Methods? A Preliminary Study, School of Computer Science, Carnegie Mellon University, 2003. [14] A. Cockburn and L. Williams, "The Costs and Benefits of Pair Programming," in Extreme Programming examined, G. Succi and M. Marchesi, Eds. Boston: Addison-Wesley, 2001, pp. 569. [15] M. Fowler, "Information about Refactoring," Addison Wesley, Canada, 2004. [16] F.Martin (2006, May, 01), Continuous Integration, Available: http://www.martinfowler.com/articles/continuousintegration.html. [17] H. Ming, V. June, Z. Liming, A. B. Mohammad, Software Quality and Agile Methods, COMPSAC 04, IEEE 2004. [18]S.I. Hashmi, J. Baik, Software Quality Assurance in XP and Spiral A Comparative Study, Computational Science and its applications, pp.367-374. [19] J. Grenning, "Launching extreme programming at a process-intensive company," Software, IEEE, vol. 18, pp. 27-33, 2001. [20] G. Jean, Schneider, V. Rajesh, Agile Practices in Software Development- Experiences from Student Projects, ASWEC 06, IEEE 2006. VII. CONCLUSION The software process models Agile, XP, Waterfall, Spiral show that each of them has their activities defined to ensure quality in their developed products. These models have quality assurance activities embedded in their developmental phase and hence do not have a separate department for quality assurance. There are some practices that are found to be common in all the models and hence based on these practices the author suggests the de-facto standard. Including these software practices into the software development process assures development of high quality software. REFERENCES [1] L. Lazic, A. Kolasinac and D. Avdic, The Software Quality Economics Model for Software Project Optimization, WSEAS Transaction on Computers, vol. 8, no. 1, pp. 21-47 Jan. 2009. [2] H. Li and J. Meissner, Improving Quality in Business Process Outsourcing through Technology, 2008. Available: http://www.meiss.com/download/bpo-li-meissner.pdf. [3] A. Garvin, Competing on the Eight dimensions of Quality Havard Business Review, pp. 101 109, Nov.-Dec. 1987.