Softjourn, Inc. s QA Testing Process

Transcription

1 Softjourn, Inc. s QA Process Date of Last Update: June 05, 2007 Version: 2.0 Author: Softjourn, Inc. Headquarters Paseo Padre Pkwy #251 Fremont, CA USA p: f: Operations &Engineering Center Schevchenko #3, 4 th Floor Ivano Frankivsk, Ukraine Office US Tel: Office:

2 Revision History Date Version Description Author June 05, Updates to diagrams and details Emmy B. Gengler Aug 23, Initial document Table of Contents 1 Introduction Purpose Overview Initial Criteria Activities Defining the requirements to be tested Developing the Test Plan Creating the Test Cases Scenarios description Functional test cases GUI Test The preparation of test materials Test cases and scenario updating Test execution Preliminary testing System and functional testing Performance/Stress testing GUI testing Regression testing Install/uninstall testing Defect reporting Exit Criteria...7 Page 2 of Paseo Padre Pkwy, Suite 251 Fremont CA Phone: Fax:

3 1 Introduction 1.1 Purpose The purpose of this document is to define in general the types of tests, the responsibilities and the steps of testing performed in a project. 1.2 Overview The purpose of Software Quality Assurance is to ensure that the system works as expected. It involves looking for discrepancies between the actual and the expected results. The purpose of testing is to systematically uncover the defects spending minimal time and efforts. In the Quality Assurance Process, is defined both as making certain that the requirements are followed and that they are implemented correctly. Time and effort needed to do testing depends on the specifics of the project and the phase of the project. If a new application is being developed, testing activities should be performed during each phase of the development. For different phases, however, the intensity of testing may vary Roles and Responsibilities The following chart defines the various roles and responsibilities within a QA testing program. Role QA Manager QA Engineer QA Tester Description/Responsibilities Defines the requirements, prepares the Test Plan, and assigns tasks regarding creating test cases, test execution, bug reporting. Creates the test cases, executes the tests, and reports the bugs. Executes the tests, and reports the bugs. Table 1: QA Roles and Responsibilities: 2 Initial Criteria The following initial criteria must be met before the QA process begins: QA Manager and QA Engineers are informed about the project scope, goal, time estimate and schedule, list of project team members, their roles and responsibilities. The requirements are defined and approved (as stated in Project Proposal and Requirements Specifications). The Unit, Stress and Integration Code Tests are performed by developers. Approach is defined. Environment is set. 3 Activities The work content depends on the quality of the input information, namely: specifications, project features and project requirements. Some kinds of tests can be performed simultaneously to increase Page 3 of Paseo Padre Pkwy, Suite 251 Fremont CA Phone: Fax:

4 efficiency. The QA manager should always be made aware of all implemented features as well as of the list of required tests. He also should be informed about the timeline requirements. 3.1 Defining the requirements to be tested The QA manager defines the priority of the features to be tested. The priorities should be based on the list of requirements which have to be implemented up to the current phase of the project. For the particular phase, test cases will be created. 3.2 Developing the Test Plan In the case that QA testing is to be part of a new development project, the Test Plan will be created for a certain period of time and for a certain phase. The Test Plan includes the following: - Test Process - Components to test - Test Approach o Test Focus o Test Coverage o Test Types o Workflow o Test Design - Acceptance testing strategy - Risks and Assumptions - Test environment - Features to be tested - Features not to be tested - QA Entrance criteria - Pass/Fail criteria - Exit criteria - Test deliverables - Test cases - Test Results Summary - Project team and contacts - Schedule The Test Plan must be approved by the Project Manager and the customer. 3.3 Creating the Test Cases Creation of the test cases should start before coding (implementing of the requirements) or at the very least at the same time that coding is in process. This eliminates prejudicing the test cases to the code as opposed to, to the requirements. If you build a test case to the code, of course it cannot fail. Therefore it is more appropriate and more accurate to build the test cases to the original specifications. The Test Cases are created by the QA Analyst or Business Analyst or Product manager. Page 4 of Paseo Padre Pkwy, Suite 251 Fremont CA Phone: Fax:

5 3.3.1 Scenarios description The functionality implemented in the application is called scenario. A scenario models a user s behavior. Scenarios are based on the defined test cases and represent the eventual working process. A scenario is made up of several test cases Functional test cases Functional requirements have to be divided into groups accordingly to the test type; Performance, for example, must support x number of users, x transaction must be completed in x seconds, etc., and functionality, for example, the user must be able to print at this point, etc. The following is defined for each function: Input data Sequence of Operations Output data The form of test cases can differ accordingly to the project or specific functionality. The information concerning the particular test case should be shown in the following format: Input Data ## Test Case Name Inputs and constraints Sequence of Operations Step #/Name Output data Expected results If a test case consists of a more complicated situation, or calculations need to be defined, these test cases can be described in other forms (in an excel spreadsheet for example). Complicated processes are better understood when described in the form of state diagrams (showing all possible workflows). It is important to mention that the diagram of the test case should not only describe the standard workflow, but also all possible deviations from the standard (exceptions: For example, the user is supposed to press Y and they press any other key, etc.). As a result, different number of workflows corresponds to each test case and every workflow should be tested. Workflows describe states, actions, operations with data and expected results (messages, data, and other events). The QA Manager should keep in close contact with the developers team when formulating the test cases in order to ensure that both the customer and the developers understand the specification equally GUI Test To test the GUI requirements a checklist should be developed. It must be based on standard conventions and customer s specific requests. It is helpful to classify the requirements in to those related to the overall look and feel versus specific requirements. Page 5 of Paseo Padre Pkwy, Suite 251 Fremont CA Phone: Fax:

6 3.3.4 The preparation of test materials It is necessary to specify a full set of related data (databases, values, etc.) in order to create a complete test plan. This could be the creation of a test client database, creating of user ids and passwords, etc. The data set should be specified beforehand in order not to delay the implementation process Test cases and scenario updating Test cases and scenarios should always be updated when modifications are requested or new requirements appear. The functions, which worked incorrectly during the previous iterations, should also be retested. The results of running each test case must be saved. Revision tracking is mandatory. 3.4 Test execution There are different types of tests. Tests should be separated in a project plan, some of them can be performed simultaneously, others should be performed only in a proper logical sequence Preliminary testing The purpose of preliminary testing is to determine whether a new software version is stable enough to perform additional testing. For example, if an application crashes every 5 minutes, bogging the system down to a crawl, or destroying databases, it may not be stable enough to warrant further testing System and functional testing System testing ensures that all system modules function as a single unit, from beginning to end. This kind of testing is performed in parallel with functional testing in most of cases. Functional testing ensures that each element of the application meets the functional requirements of the business as outlined in the specification documents and other documents produced during the course of the project (records of modification requests, feedback, etc.) Performance/Stress testing Stress testing is similar to system or functional testing. The feature is that stress testing is carried out given unusually heavy loads, frequent repetition of certain actions or inputs, input of large numerical values, large complex queries to a database system, etc. Performance testing verifies whether the application meets performance requirements GUI testing GUI should be tested separately from the business logic based on GUI requirements and customer s GUI guide style (if one exists). GUI testing should be carried out periodically during the entire life cycle of the project Regression testing Regression testing is performed at each iteration to verify whether the previously implemented functionality still works correctly after the latest changes were implemented. Regression testing will look at any unexpected impacts, and will be performed on each new version of the software or application. Regression testing ensures that by fixing module A or changing module A, something in module B has not been broken. Page 6 of Paseo Padre Pkwy, Suite 251 Fremont CA Phone: Fax:

7 3.4.6 Install/uninstall testing of full, partial, or upgrade install/uninstall processes. 3.5 Defect reporting All defects found should be registered in the Defect Tracking Database with the proper attributes assigned (summary, description and others). Then the defects are tracked according to the Defect Tracking Procedure defined for the project. Config. Mgmt. Prepares Environment Build 1 Other types of testing (performance, stress, load testing) Defect Fix Verification (if available) Functional (Functional Req. are met) Integration (Business Scenarios) Config. Mgmt. Prepares Environment Build x All defects will be reported and processed using the Defects Tracking Procedure. Regression Other types of testing (performance, stress, load testing) Defect Fix Verification Functional (Functional Req. are met) Integration (Business Scenarios) A QA report will be delivered stating the results of testing All defects will be reported and processed using the Defects Tracking Procedure. Complete Deployment Stage Fig. 1 Standard Quality Assurance Activities 4 Exit Criteria Page 7 of Paseo Padre Pkwy, Suite 251 Fremont CA Phone: Fax:

8 Details of Exit Criteria For each project phase, all testing will be completed until the acceptance criterion is met. has been performed according to the Test Plan. Page 8 of Paseo Padre Pkwy, Suite 251 Fremont CA Phone: Fax: