Chapter 11: Integration- and System Testing

Transcription

1 Chapter 11: Integration- and System Testing Chapter 14: Testing (2/2) Object-Oriented Software Construction Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit)

2 Software Lifecycle Activities...and their models Requirements Elicitation Analysis System Design Object Design Implementation Testing Expressed in Terms of Structured by Realized by Implemented by Verified by class... class... class...? Use Case Model Application Domain Objects Subsystems Solution Domain Objects Source Code class...? Test Cases Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 2

3 Test Team Analyst Professional Tester Programmer too familiar with code User Test Team System Designer Configuration Management Specialist Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 3

4 Types of Testing Unit Testing (last lecture): Individual subsystem Carried out by developers (of components) Goal: Confirm that subsystems is correctly coded and carries out the intended functionality Integration Testing (mainly this lecture): Groups of subsystems (collection of classes) and eventually the entire system Carried out by developers Goal: Test the interface and the interplay among the subsystem Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 4

5 Types of Testing System Testing: The entire system Carried out by developers (testers!) Goal: Determine if the system meets the requirements (functional and global) Functional Testing: Test of functional requirements Performance Testing: Test of non-functional requirements Acceptance and Installation Testing: Evaluates the system delivered by developers Carried out by the client. Goal: Demonstrate that the system meets customer requirements and is ready to use Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 5

6 Integration Testing Strategy The entire system is viewed as a collection of subsystems (sets of classes) determined during the system and object design. Assumption: System Decomposition is hierarchical The order in which the subsystems are selected for testing and integration determines the testing strategy Big bang integration (Nonincremental) Bottom up integration Top down integration Sandwich testing Variations of the above For the selection use the system decomposition from the System Design Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 6

7 Example: Three Layer Call Hierarchy User Interface Layer III Billing System Event Service Learning Layer II Database Network Access Neural Network Layer I Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 7

8 Integration Testing: Big-Bang Approach Unit Test A Unit Test B Unit Test C Unit Test D Don t try this! System Test All components (units) are first tested individually and then together as a single and entire system: Pros: No test stubs (mocks) and drivers are needed Cons: Difficult to pinpoint the specific component responsible for the failure Results in Strategies that integrate only a few components at the time Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 8

9 Bottom-up Testing Strategy The subsystem in the lowest layer of the call hierarchy are tested individually Then the next subsystems are integrated and tested from the next layer up that call the previously tested subsystems This is done repeatedly until all subsystems are included in the testing Only Test Drivers are used to simulate the components of higher layers No Test Stubs! Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 9

10 Bottom-up Integration A Layer III Test E B C D Layer I Test B, E, F E F G Layer I Test F Test C Test A, B, C, D, E, F, G Test G Test D,G Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 10

11 Pros and Cons of bottom up integration testing Pros: Interface faults can be more easily found (the usage of test drivers accomplishes a clear intention of the underlying interfaces of the lower layer) No Stubs are necessary Cons: Components of the User Interface are tested last Faults found in the top layer may lead to changes in the subsystems of lower layers, invalidating previous tests. Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 11

12 Top-down Testing Strategy Test the top layer or the controlling subsystem first Then combine all the subsystems that are called by the tested subsystems and test the resulting collection of subsystems Do this until all subsystems are incorporated into the test Test Stubs are used to simulate the components of lower layers that have not yet been integrated. No drivers are needed Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 12

13 Top-down Integration Testing A Layer III B C D Layer I E F G Layer I Test A Layer III Test A, B, C, D Layer III + II Test A, B, C, D, E, F, G All Layers Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 13

14 Pros and Cons of top-down integration testing Pros: Test cases can be defined in terms of the functionality of the system (functional requirements) More effective for finding faults that are visible to the user Cons: Writing stubs can be difficult: Stubs must allow all possible conditions to be tested. Possibly a very large number of stubs may be required, especially if the lowest level of the system contains many methods. Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 14

15 Sandwich Testing Strategy Combines top-down strategy with bottom-up strategy (parallel testing is possible) The system is view as having three layers A target layer in the middle A layer above the target (top layer) A layer below the target (bottom layer) Testing converges towards the target layer No Test Stubs and Drivers are necessary for bottom and top layer Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 15

16 Sandwich Testing Strategy A Layer III B C D Layer I Test E E F G Layer I Bottom Layer Tests Test F Test G Test B, E, F Test D,G Test A, B, C, D, E, F, G Top Layer Tests Test A Test A,B,C, D Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 16

17 Pros and Cons of Sandwich Testing Pros: Top and Bottom Layer Tests can be done in parallel No Stubs and Drivers (saves development time) Cons: Does not test the individual subsystems on the target layer thoroughly before integration (C in the example) Solution: Modified sandwich testing strategy Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 17

18 Modified Sandwich Testing Strategy Tests the three layers individually before combining them in incremental tests with one another The individual layer tests consists of three tests: Target layer test with drivers and stubs Top layer test with stubs Bottom layer test with drivers The combined Layer Tests consist of two tests: Top layer accessing target layer (top layer replaces drivers) Bottom accessed by target layer (bottom layer replaces stubs) Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 18

19 Modified Sandwich Testing Strategy Test A Test A,B A B C D Layer III Layer II Test A,C E F G Layer I Test B Test C Test D Test A,D Test A, B, C, D Test E Test F Test G Test D,G Test B, E, F Test A, B, C, D, E, F, G Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 19

20 Using the Bridge Pattern to enable early Integration Testing Use the bridge pattern to provide multiple implementations under the same interface. Interface to a component that is incomplete, not yet known or unavailable during testing VIP Seat Interface (in Vehicle Subsystem) Seat Implementation Stub Code Simulated Seat (SA/RT) Real Seat Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 20

21 Which Integration Strategy should you use? Factors to to consider Location of of critical parts in in the the system Availability of of hardware Availability of of components Scheduling concerns Bottom up up approach good for for object oriented design methodologies Test Test driver interfaces must match component interfaces top-level components are are usually important and and cannot be be neglected up up to to the the end end of of testing Detection of of design errors postponed until until end end of of testing Top Top down approach Test Test cases can can be be defined in in terms of of functions examined Need to to maintain correctness of of test test stubs Writing stubs can can be be difficult Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 21

22 System Testing Functional Testing Performance Testing Acceptance Testing Installation Testing Impact of requirements on system testing: The more explicit the requirements, the easier they are to test. Quality of use cases determines the ease of functional testing Quality of nonfunctional requirements and constraints determines the ease of performance tests: Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 22

23 Functional Testing Functional. testing finds differences between functional requirements and the implemented system Essentially the same as black box testing Goal: Test functionality of system Test cases are designed from the requirements analysis document (better: user manual) and centered around requirements and key functions (use cases). Select tests that are relevant to the user and have a high probability of uncovering a failure Use techniques like equivalence tests Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 23

24 Performance Testing Stress Testing Checks if the system can respond to many simultaneous requests (maximum # of users, peak demands) Volume testing Test what happens if large amounts of data are handled Configuration testing Test the various software and hardware configurations Compatibility test Test backward compatibility with existing systems Security testing Try to violate security requirements Timing testing Evaluate response times and time to perform a function Environmental test Test tolerances for heat, humidity, motion, portability Quality testing Test reliability, maintainability & availability of the system Recovery testing Tests system s response to presence of errors or loss of data. Human factors testing Tests user interface with user Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 24

25 Test Cases for Performance Testing Push the (integrated) system to its limits. Goal: Try to break the subsystem Test how the system behaves when overloaded. Can bottlenecks be identified? (First candidates for redesign in the next iteration Try unusual orders of execution Call a receive() before send() Check the system s response to large volumes of data If the system is supposed to handle 1000 items, try it with 1001 items. What is the amount of time spent in different use cases? Are typical cases executed in a timely fashion? Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 25

26 Steps in Testing (only developer s view) Based on on the the integration strategy, select select a a componen to to be be tested. Unit Unit test test all all the the classes in in the the component Put Put selected component together; do do any anypreliminary fix-up fix-up necessary to to make the the integration test test operational (drivers, stubs) Do Dofunctional testing: Define test test cases that that exercise all all uses uses cases with with the the selected component Execute performance tests tests Keep Keep records of of the the test test cases and and testing activities Repeat steps 1 to to 5 until until the the full full system is is tested. The The primary goal goal of of integration testing is is to to identify errors errors in in the the (current) component configuration. Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 26

27 Acceptance Testing Goal: Demonstrate system is is ready for for operational use Choice of of tests tests is is made by by client Many tests tests can can be be taken from from integration testing Acceptance test test is is performed by by the the client, not not by by the the developer. Majority of of all all bugs in in software is is typically found by by the the client after after the the system is is in in use, use, not not by by the the developers or or testers. Therefore two two kinds of of additional tests: Alpha test: Sponsor uses uses the the software at at the the developer s site. site. Software used used in in a controlled setting, with with the the developer always ready to to fix fix bugs. Beta test: Conducted at at sponsor s site site (developer is is not not present) Software gets gets a realistic workout in in target environment Potential customer might get get discouraged Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 27

28 Summary Testing is is still a black art, but many rules and heuristics are available Testing consists of ofcomponent-testing (unit testing, integration testing) and system testing Design Patterns can be be used for for integration testing Testing has its its own lifecycle Armin B. Cremers, Sascha Alda & Tobias Rho (based on Bruegge & Dutoit) Object-Oriented Software Construction 28