G53QAT COURSEWORK BY ADEOLU OPEOLUWA OPEODU AXO16U. Test Planning

Transcription

1 Test Planning The topic that I will discuss in this essay is Test Planning. I will address the topic of test planning in relation to software test planning. Defining the scope During the course of this essay I will focus on these aspects in relation to Test planning: 1.0 What is software Testing & its objectives? 2.0 What is Test Planning? 3.0 What to Test? 4.0 Which sources to use for test cases? 5.0 Who will carry out the test? 6.0 The environment of the test 7.0 When does the test terminate? 8.0 Conclusion Within section further sub section will exist to discuss the Test Planning.

2 1.0 What is Software Testing To be able to address what test planning is, we must look at the definition of what is software testing. Software testing is the procedure which is undertaken by a dedicated team either in-house (in the company) or outsourced (external to the company). Their responsibility is to execute a Software Test Plan (STP) which takes into consideration the testing environments, details and schedules. Daniel Galin gives a definition of Software Testing please see Appendix Figure 3 Quotes. From these definitions above the definition for Test planning can be formulated. Objectives The objectives: Find errors in the software Increase software quality Execute testing within budget resources allocated.

3 2.0 What is Test Planning? Test Planning is the task which should be undertaken before software testing commences because software test planning formulates a structured approach by defining the scope of the problem, the test cases, environments, performers and test termination conditions. Test Planning can be split up into these subsections: What to Test? Test cases will be used? Who will carry out the test? The environment of the test When does the test terminate? 3.0 What to Test? The nature of the test must be defined this is because different projects require different approaches. Some projects/systems/software requires for the whole system to be tested, others may only test a module of the system because this is what is available at the time. There are several types of test what could be carried out, these include: Unit Test Integration Test System Test Unit Test This type of Test Plan looks solely at the small component or modules which comprise the system or software. How does it work? Each individual unit will be tested by itself as a stand alone entity to test it s correct and performs to the specification. When should it be used? Unit test planning should ideal be used during a software development life cycle when units are dependant on each other. This is where one unit must be finished before another unit may be even state to be developed. The software team will have no other choice but to test the new units of the system each cycle. MSDN gives a definition of Unit Test please see Appendix Figure 3 Quotes.

4 Integration Test Integration testing involves testing several modules or components together. How does it work? Several components (which have preferably passed a unit test) will be tested to see how they perform and interact once joined. When should it be used? Integration test planning should ideally be used during an iterative software development life cycle. This is where the users are involved with the design and provide feedback which may change the design. These new changes should undergo an integration test plan. MSDN gives a definition of Integration Test please see Appendix Figure 3 Quotes. System Test System test involves testing the whole system/software. How does it work? All the components of the system will be tested together for their performance according to the specification for the system. When should it be used? System test should ideally be used after integration testing, because it has a great risk of producing bugs, and these bugs may be harder to identify if some other background testing have not been run, which may give the programmer a clue to where the bug may be situated.

5 4.0 Which sources to use for test cases? What is a test case? A test case is a scenario document which the software or component of the software will be put through to test that it performs according to the specification. There are two types of sources for a test case; these are Real Life test case and Synthetic test cases. Real Life test case Real life test case involves testing the system/software with real world conditions e.g. Environment, real data and the target users potentially using the system. Real life test case normally requires low preparation effort however the same test case may be repeated several times. This means a large test case must be collected to be able to get all the possible conditions the software may have to go through. There is a high effort required to perform the actual test case due to the large test case needed. The repetition of test cases reduces efficiency. Real life test case has an opportunity to identify unexpected errors because the system is not being used according to the developer s written test case. Synthetic test case. This is a set of scenarios or steps which will be put together by the designers of the software. These scenarios consist of the input values, parameters and conditions that will be needed to test the system. Synthetic test case takes high preparation effort because the input values, parameters conditions and estimated results must all be taken into consideration. The size of the test case is small because it can avoid repetitions, also the effort required to execute the test case is also less then the Real life test case because the test case it small. Synthetic test case provides a good opportunity to spot error in the design. However it reduces the chance of spotting unexpected errors due to the strict scenarios in the test case. When should Real life or Synthetic test case be used? Daniel Galin gives a suggestion on which source to use, please see Appendix Figure 3 Quotes. The suggestion is that the chosen source to use would depend on the type of test and what is being tested: units, integration or system.

6 5.0 Who will carry out the test? Unit test will normally be carried out by the development team of the units. Integration test This test will also be carried out by the development team. System test System test may be carried out by a dedicated testing team either internally or externally to the company. Internally may include other development teams, externally will take the form of outsourced vendors. Evidence of Research: Software Testing Vendors please see appendix Figure 2 - Vendors

7 6.0 The environment of the test Unit test This test is executed at the developer s site. Integrated test This test is executed at the developer s site. System test This test can be performed at any of these 3 sites: The Developers site The Target Site The Consultants Site. The Developers site The developer s site is a closed controlled environment where the software or component that is being tested is made. The disadvantage of performing a test in there would be that real world environment cannot be simulated. The Target Site This is the environment where the system or component is going to be used after completion. The advantage to this is that unexpected errors or failure can be captured as real world conditions are open and unpredictable. The Consultants Site. The consultant s site is the environment of the software testing outsourced companies. The outsourced vendor will be expected to simulate the real world environment and test the system according. The disadvantage of this is that the test quality will only be as good as the outsourced vendor s skills.

8

9 7.0 When does the test terminate? There are 5 methods of determining when to terminate the test. These methods are dependent on the type of the software of components being tested. These 5 methods are: The completed implementation route The mathematical models application route The error seeding route The dual independent testing teams route Termination after resources has petered out. The completed implementation route This route tries to achieve perfection by completing the test plan repeatedly till there are no errors or faults found. The disadvantage to this approach is that is disregards constraints on the time allocation/ schedule and a resource budget, such as money and staff. The mathematical models application route This route considers a chosen arithmetic model, were estimation would take place to see the undetected errors, based on the error rate. When the undetected errors falls to a value lower than the threshold error rate, the test may be terminated. The disadvantage to this route is that the test may terminate before an error/bug is found. The possible impact of the errors is not taken into consideration whilst this route is used. The error seeding route This route works on the principal that the number of undiscovered errors is related to the number of discovered errors. When the number of undiscovered errors reaches a level of approval then the test will terminate. The disadvantage to this method is that it works on the assumption that all system or components being tested are the same. However this is rarely the case in the real world. The dual independent testing teams route In this route there are two testing teams which will undertake the take of testing the system/component. By doing so, then producing results and cross references both teams results, the number of undetected errors would decline significantly. The disadvantage to this approach is that as more staff is needed; in turn there will be an increase in cost and bringing about a high effort requirement to execute this route. Termination after resources has petered out. This route merely terminates the test once the resources have run out. Such as time, money, staff etc.

10 The disadvantage to this is that there is a danger that the test may terminate early before the error/bug is found.

11 8.0 Conclusion To conclude Test Planning relates to Software Quality Assurance and test quality because it is a necessary step which should be carried out prior to the actual testing. Planning out any activity will provide a structured approach to the task and help consider all possible factors and even some unexpected situations can be taken into consideration. As with planning for the testing phase, it helps to define the scope of the issue, the performers, environment, termination condition, sources for test cases along with other factors. Summary To summarize, this essay has focused on these areas below: 1.0 What is Software Testing and its objectives? 2.0 What is Test Planning? 3.0 What to Test i.e. Unit, Integration or system test. 4.0 Which sources to use for test cases.i.e. Real Life or Synthetic test case sources. 5.0 Who will carry out the test i.e. the development team (internally) or an outsourced company (externally). 6.0 The environment of the test i.e. Developers site, Target site and Consultants site. 7.0 When does the test terminate i.e. o The completed implementation route o The mathematical models application route o The error seeding route o The dual independent testing teams route o The Termination after resources has petered out. Here is an example of a template for a Software Test Plan (STP) See Appendix Figure 1 Word Count (ex. Bibliography Appendix 1922 word).

12

13 Bibliography 1.0 What is Software Testing Definition - Daniel Galin Software Quality Assurance, page 3.0 What to Test? Unit Test definition: Integration Test definition: Which sources to use for test cases? Which sources to use for test cases? - Software Quality Assurance - Daniel Galin page Who will carry out the test? Acutest Thinksoftglobal Maverick independent testing e-testing Appendix Figure 1 - Template for a Software Test Plan (STP) Daniel Galin Software Quality Assurance page 228

14 Appendix Figure 1 - STP Template for a Software Test Plan (STP) (Daniel Galin Software Quality Assurance page 228)

15 Figure 2 - Vendors Evidence of Research: Software Testing Vendors Here is a list of some main Software Testing (outsourced) companies and their clients. Acutest Acutest is an independent software testing consultancy. We concentrate solely on testing software, business processes, and IT with the goal of helping our customers to achieve the benefits of their technology-enabled change faster. Clients: BT, BAA, BBC, Teletext, Post Office, Granada, Royal Mail, Zurich, DSA, Dell, HBOS, NHS & Bradford Bingley Thinksoftglobal Thinksoft is a specialist in financial software testing with over 8 million person hour track record for global clients Clients: DBS, Deutsche bank, Citigroup ABN-AMRO & Morgan Stanley Maverick independent testing As an independent software testing company with world-class software QA laboratories, we work with some of the leading global banks, financial institutions and development companies. We have carried out independent testing engagements across India, UK and Dubai. Clients: HP invent, Standard Chartered, ABN-AMRO & LogicaCMG e-testing e-testing is an independent specialist IT consultancy with a total focus in software testing - consultancy, resourcing, training and managed services. We deliver high quality managed strategies and software testing solutions to systematically locate weaknesses within the complexity of any IT system. Using e-testing on automated software testing projects can quickly generate return on investment, especially when considering mission critical performance testing and regression testing requirements. Clients: Goldman Sachs, O2, Sony Ericsson, Accenture, Halifax, Northern Rock, KMPG

16 Figure 3 - Quotes Evidence of Further Reading: Definition of software testing, Software testing is a formal process carried out by a specialized testing team in which a software unit, several integrated software units or an entire software package are examined by running the programs on a computer. All the associated tests are performed according to approved test procedures on approved test case. (Daniel Galin Software Quality Assurance) Evidence of Further Reading: MSDN defines unit test as, The primary goal of unit testing is to take the smallest piece of testable software in the application, isolate it from the remainder of the code, and determine whether it behaves exactly as you expect. Each unit is tested separately before integrating them into modules to test the interfaces between modules. Unit testing has proven its value in that a large percentage of defects are identified during its use. ( Evidence of Further Reading: Evidence of Further Reading: MSDN defines integration test as, Integration testing is a logical extension of unit testing. In its simplest form, two units that have already been tested are combined into a component and the interface between them is tested ( Evidence of Further Reading: (Which sources to use for test cases?) Each component of the testing plan, dealing with unit, integration or the system test, requires an individual decision about the respective test cases and their sources: - The use of a single or combined source of test case or both - How many test cases from each source are to be prepared - The characteristics of the test cases. (Software Quality Assurance - Daniel Galin page 222)