Quality Assurance Plan

Transcription

1 Quality Assurance Plan Suunto Training Program Planner Publish date: team IMARA [ee-mah-rah] strong, resolute; traditional Swahili name Failure is not an option, it comes embedded in software - Jerry Krasner, Ph.D. -

2 Changelog Date Author Comments Reviewer Status Ville Initial draft DRAFT Ville Writing document DRAFT Ville Finalizing document Andreas FINAL Ville Small fixes Eric FINAL Contents 1 INTRODUCTION QUALITY ASSURANCE GOALS QUALITY ASSURANCE PRACTICES AND TOOLS LEVELS OF TESTING TESTING PRACTICES QUALITY PRACTICES TESTING TOOLS SUMMARY DEFECT TRACKING RESOURCES AND SCHEDULING DELIVERABLES...11

3 1 Introduction This document describes the planned quality assurance goals, practices, tools, and scheduling for the Suunto Training Program Planner project. This document will be continuously updated during the course of this project. The QA Manager is the main responsible for maintaining this document. The intended audience of this document is presented in Table 1. This document is a part of the requirements for the course T Software Development Project course at Helsinki University of Technology. Table 1 - Intended audience of this document. Group of the readers Customer Project Team System Developers Testers Mentor Reason for reading To get informed on the planned quality assurance practices of the project and verify them To plan, follow, and document the quality assurance process and guidelines of the project To understand and follow the planned quality assurance practices To understand and follow the planned quality assurance practices To evaluate the quality assurance process and give feedback 2 Quality Assurance Goals The quality goals for this project are mostly derived from the non-functional requirements and are of qualitative nature. Many of the risks identified in the project plan have a negative effect on product quality if materialized, thus successful risk management is also crucial for quality. Table 2 Quality goals ID Quality Goal Description Verification QG-1 Functionality QG-2 Usability QG-3 Code quality All the required functionality is implemented The User interface should be intuitive and as easy to use as possible The system suffers from no serious defects and can be regarded to be of production quality - Team verifies that all marked functionalities are implemented - Customer makes the final verification - Team evaluates the UI usability internally - A peer group will evaluate the UI usability - An external focus group might test the usability (organized by the customer) - Customer makes the final verification - The defect tracking system does not have any major/critical defects and not a large amount of smaller ones at the end of the project

4 QG-4 Layout QG-5 Localization The graphical look must be in line with the Suunto brand The system must enable localization (use of the text resources files) - Customer makes the final verification - The system is tested and proven to work with different languages QG-6 Risk management Risks must be successfully managed in order to prevent negative impact on quality - The risk management process has been followed according to plan

5 3 Quality Assurance Practices and Tools 3.1 Levels of Testing We will follow the four levels of testing according to the V-model of software development during this project. Figure 1 illustrates the links between testing and according development levels. In the following chapters, the practices and methods that will be used for quality assurance will be presented and the deployment method will be explained. After that the used environments and software used for testing will be presented. Finally, the methods and practices, as well as the utilized tools will be linked to according test levels of the V-model. Requirements analysis Acceptance Test Design User Acceptance Testing System Design System Test Design System Testing Architecture Design Integration Test Design Integration Testing Module Design Unit Test Design Unit Testing Coding Figure 1 - V-Model of software development (adapted from

6 3.2 Testing Practices Unit testing Course requirements demand a reasonable amount of unit level testing. We want to ensure high quality work throughout the project and proactively prevent unwelcome surprises late into the project. Therefore, unit testing is strongly emphasized throughout the development work. Unit testing will be performed in all the backend layers: The Data Access Layer (DAL), Business Logic Layer (BLL), and Service Access Layer (SAL). All methods in these layers will have at least one corresponding unit test and therefore the test coverage of the backend will be close to 100 percent. It is the responsibility of the developer to write the unit tests needed whenever a method in the backend is added or modified Integration Testing In this project, integration testing means the integration of different parts of Suunto Training Program Planner. Later the system needs to be integrated into the mysuunto portal, but this is not within the scope of this project. Integration testing in this project will be handled by frequent builds of the system System and Acceptance Testing The software should be of production quality at the end of this project, thus System and Acceptance testing are of great importance for this project. Three best practices are deployed in order to ensure successful system and acceptance testing: test-cases, explorative testing, and heuristics Test-cases The course requires a minimum of 50% of implemented functional requirements to be tested based on test-cases. Test cases will be used as a foundation both when testing by hand and also when Selenium tests are deployed through the Robot Framework. Most functional requirements will be tested with either method. When testing by hand, the test cases will be written based on the Activity Diagrams and possible use cases. The QA manager is responsible to oversee that test cases are performed for the planned requirements. The test-cases are written according to the Extended Use-Case Test-Case Design Pattern ( When executing test cases the following steps must be documented in a test log according to the template given at the course homepage ( - input values - steps - expected results - date - tester - test environment (hw, sw, build...) - list of executed test cases with status (passed/failed) - bug IDs Furthermore following practices will be followed when executing test-cases:

7 Boundary value analysis - analyze the most critical input values and focus testing efforts to the most likely fault locations. These are found at boundaries, where a change in input values cause a change from one equivalence class to another equivalence class Equivalence partitioning input values are divided into equivalence classes and testing one value from an equivalence class is representative for the whole class, and thus saves time Explorative testing Explorative testing is encouraged throughout the project. It will be performed more rigorously at the end of each sprint by the customer. Also the peer group will be given the opportunity to exploratory test the software. Session Based Test Management will be organized at the latter part of the project to ensure seamless functionality between software components. For these sessions, test charters must be prepared ( and the exploration log in the test charter must be filled during each session Heuristics Usability and visual requirements will be tested through active involvement of the customer, peer group evaluation, and possibly focus group feedback. 3.3 Quality Practices Also other quality related best practices will be used during the course of this project. Next each are explained more in detail Coding standards The use of coding standards is required by the course. We use a modified version of Dotnet Spider s coding convention. More details are available in the project plan and the team s wiki page Pair programming Especially in the beginning of the project pair programming will be used actively. This is because only few of the developers have former experience in.net and C# and therefore pair programming functions as a useful practice to enable fast learning amongst the developers. Practically pair programming will be arranged during weekly meetings and also at other points of time according to agreement Code reviews According to course requirements code reviews must be applied to at least on one critical component of the system. We will organize code review session as needed during the weekly meetings. This supports the learning goals and helps in forming a general understanding of the system as a whole. The code reviews should also check and point out problems regarding: - Functionality of the code - Performance of the code - Documentation of the code - Bad coding syntax

8 3.3.4 Document reviews The project management team is responsible to review, and when needed update, all documentation at least in the end of each sprint Customer involvement Active customer involvement is a must due to the iterative process of the project. A meeting is scheduled in the beginning of each sprint. Furthermore a weekly status report will be sent to the customer for commenting. At other times, customer contact persons are available by phone and Testing Tools NUnit NUnit is used as the unit test framework. It is applied to the backend layers: DAL, BLL, SAL. Different input values will be used depending on the test case Selenium Selenium will be used for acceptance and system level testing. The extent to which Selenium tests will be applied is still to be defined by the test manager of the customer Robot Framework Robot Framework will be used to run and produce a human readable report of all the Selenium tests JIRA JIRA will be used for defect tracking. Each found defect will be stored and prioritized in JIRA.

9 3.5 Summary Table 3 summarizes the most important quality practices with according testing levels and tools. Table 3 Summary of the testing levels, testing practices, and tools. Testing Level Testing Practices Tools Unit Testing Developers are responsible for writing unit tests NUnit Integration Testing Frequent builds Visual Studio 2008 System Testing Test-cases Selenium Explorative testing Robot Framework Acceptance Testing Test-cases Explorative testing Selenium Robot Framework Heuristics In Table 4 the planned testing and quality practices are linked to according quality goals. The realized effect and amount of use will be updated at the end of each iteration. Table 4 - Summary of QA practices (x = use was planned, 0-3 = realized effect on achieving the goal, green yellow red color = amount of use) Practices QUALITY GOALS QUALITY RISKS Functionality Usability Code quality Layout Localization Risk management R2 R3 R9 Unit testing x x Integration testing System testing Acceptance testing Coding standards Pair programming x x x x x x x x x x x x x x x x x x x x x Code reviews x x x

10 Document reviews x x x Customer involvement x x x x x The quality related risks are derived directly from the risk analysis in the project plan. Following risks directly linked to quality assurance practices were identified: R2 - Communication problems leads to delays and inconsistencies in information sharing. R3 - Poor product quality due to insufficient testing practices. R9 - The team members cannot adapt to the used development tools on time. 4 Defect Tracking Each found defect is marked as a new issue to JIRA. The person who has found the defect is responsible for submitting the defect to JIRA. The type and the priority of the defect are defined by the submitter. We are using two types of defects: Bug a defect that needs fixing. This can be assigned for fixing without consulting the customer. Enhancement when somebody discovers a way of doing something better than planned. An enhancement will not be fixed before discussing the issue with the customer. We are using five levels of priorities offered by JIRA: Blocker Blocks the use of the entire system. It is a number one priority to fix the defect. Critical A large part of the system is inoperable or affected by this defect. Major Severely affects the use of the system. Default priority. Minor A defect that is tolerable to some degree during the use of the system. Trivial Minor cosmetic issues.

11 5 Resources and Scheduling This schedule and the assigned resources will be updated along as the project progresses. Also new tasks will be added along the way. Deadline Task description Responsible Delivery of QA plan QA Manager Peer-programming session Write unit tests Code review session 7.12 Delivery of QA report QA Manager 20.1 Delivery of QA plan QA Manager Write test-cases and Selenium tests Planning of peer group testing Session Based Exploratory Testing 22.2 Delivery of QA report QA Manager 6 Deliverables The course requires the following quality related deliverables: - Quality assurance plan at the beginning of iteration 1 (this document, updated regularly) - Quality report at end of each iteration (see template o Evaluation of QA Practices o Quality Status (including defect report) o Action Points - Iteration demo QA input at the end of each iteration - Other documentation: o Test-cases o Test-case test logs o Test charters