Test Design: Functional Testing

Transcription

1 Test Design: Functional Testing Daniel Sundmark How should test cases be designed?

2 Outline Functional Testing Basics Systematic Functional Test Design Input-Space Based Techniques Equivalence Partitioning Boundary Value Analysis Combinatorial Testing Random Testing Model-Based Testing 2

3 Functional Testing Basics Basic idea: Test cases are derived based on the functional specifications of the software under test What does the specifications say that the software is supposed to do? What does the specifications say that the software is not supposed to do (if this is explicitly stated)? Implementation of the software under test is not considered in test case design Hence, functional testing is also historically known as Black-Box Testing 3

4 Functional Testing Basics Basic idea: Test cases are derived based on the functional specifications of the software under test What does the specifications say that the software is supposed to do? What does the specifications say that the software is not supposed to do (if this is This explicitly term has stated)? been critizised by several testing scholars, Implementation of the software since under is claimed test is not to represent considered in test case design an old-fashioned view of Hence, functional testing is also historically testing. known as Black-Box Testing Nevertheless, the term is still widely used. 4

5 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases CREATE Test specifications This is a general approach that can be utilized at all levels of testing, but still needs to be adapted to the situation at hand and the software under test 5

6 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases CREATE Test specifications As previously stated, the functional specifications is the starting point for functional test design 6

7 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases CREATE Test specifications Based on the functional specification, independently testable aspects (ITAs) are identified. An ITA is a part of the functionality of the software that can be tested separately from other functionalities. An important aspect of an ITA is its inputs. 7

8 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Also called independently testable features or functions. Representative values Model Test cases CREATE Test specifications Based on the functional specification, independently testable aspects (ITAs) are identified. An ITA is a part of the functionality of the software that can be tested separately from other functionalities. An important aspect of an ITA is its inputs. 8

9 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases CREATE Test specifications The concept of an ITA is not terribly well-defined. Java methods or isolated C functions are often given as examples, but ITAs exist on all levels of integration. For example, the withdrawal function of an ATM could be an ITA. Exercise: Take a few minutes and suggest an ITA of a system or software of your own choosing. 9

10 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases CREATE Test specifications Based on the nature of the ITA, we may either proceed with identifying representative values for input using different techniques, e.g., Equivalence partitioning of the input space Boundary value analysis Different types of random selection 10

11 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases CREATE Test specifications Based on the nature of the ITA, we may either proceed with identifying representative values for input using different techniques, e.g., Equivalence partitioning of the input space Boundary value analysis Different types of random selection First family of techniques we will look into today 11

12 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases CREATE Test specifications Based on the nature of the ITA, we may either proceed with identifying representative values for input using different techniques, e.g., Equivalence partitioning of the input space Boundary value analysis Different types of random selection or 12

13 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases CREATE Test specifications we may even derive a test Model that describes (parts of ) the specification in a more formal way, and that helps us derive test cases 13

14 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases CREATE Test specifications we may even derive a test Model that describes (parts of ) the specification in a more formal way, and that helps us derive test cases Second family of techniques we will look into today 14

15 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases CREATE Test specifications Based on the representative values, or/and the test model, test specifications are derived. Test specifications are abstract test requirements that must be met during testing. Note: Often a given test specification could be met by a large number of possible sets of test cases. 15

16 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases CREATE Test specifications As a last step in the test design process, we create test cases that meet the test specifications 16

17 A Systematic Approach to Functional Test Design Adapted from Software Testing and Analysis, (Pezzè and Young), and Saarland University course on Software Testing (Fraser and Zeller) Functional specifications IDENTIFY Independently testable aspects Test cases IDENTIFY Again, note here that a test case not only requires an input, but also an expected output in order to be able to provide a verdict. Representative values GENERATE Test specifications The expected output is typically derived based Model As a last step in the test design process, we generate test cases that meet the test on the specifications specifications 17

18 Let s go back to the more technical parts Techniques based on partitioning of the input space 18

19 Identifying representative values Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases GENERATE Test case specifications 19

20 Input Space Each ITA has an input space, which is made up of all possible input combinations Simple example: a function fun(int x, int y) has a two-dimensional input space made up of all possible values of input parameters x and y Each test case input is a (x,y) pair, which essentially corresponds to a unique point in the input space y x 20

21 Equivalence Partitioning Divide and conquer Basic idea: Divide input space into blocks within which the software should behave For each input similarly. parameter, Then just cover all possible blocks instead of all possible inputs. Divide the input space of that parameter into partition blocks based on some characteristic Within each block, the software under test should behave equivalent with respect to that parameter Then test inputs are chosen to cover all blocks rather than the full input space Fundamental rules for a partition p: All partition blocks within p must be disjoint The union of all partition blocks in p must make up the complete input space y x 21

22 Equivalence Partitioning Divide and conquer For each input parameter, Divide the input space of that parameter into partition blocks based on some characteristic Within each block, the software under test should behave equivalent with respect to that parameter Then test inputs are chosen to cover all blocks rather than the full input space Fundamental rules for a partition p: All partition blocks within p must be disjoint The union of all partition blocks in p must make up the complete input space y x 22

23 Equivalence Partitioning Divide and conquer For each input parameter, Divide the input space of that parameter into partition blocks based on some characteristic Within each block, the software under test should behave equivalent with respect to that parameter Then test inputs are chosen to cover all blocks rather than the full input space Fundamental rules for a partition p: All partition blocks within p must be disjoint The union of all partition blocks in p must make up the complete input space y x 23

24 Equivalence Partitioning Divide and conquer For each input parameter, Divide the input space of that parameter into partition blocks based on some characteristic Within each block, the software under test should behave equivalent with respect to that parameter Then test inputs are chosen to cover all blocks rather than the full input space Fundamental rules for a partition p: All partition blocks within p must be disjoint The union of all partition blocks in p must make up the complete input space y x 24

25 Equivalence Partitioning General Process 1. For each ITA, identify all parameters that affect its behavior. These make up the input space. 2. Partition the input space based on a set of characteristics. Strategies: Individual partitioning of each input parameter Functional (or even output-based) modeling of the input space 3. Combine the partitions into an input domain model. In this step, identify and remove invalid combinations 4. Identify representative values 5. Derive test cases by complementing with expected output 25

26 Equivalence Partitioning Example (Ammann and Offutt) 1. For each ITA, identify all parameters that affect its behavior. These make up the input space. 2. Partition the input space based on a set of characteristics. Strategies: Individual partitioning of each input parameter Functional (or even output-based) modeling of the input space 3. Combine the partitions into an input domain model. In this step, identify and remove invalid combinations 4. Identify representative values 5. Derive test cases by complementing with expected output Let s look at how this can be applied to the Triangle categorization problem 26

27 Equivalence Partitioning Example (Ammann and Offutt) 1. For each ITA, identify all parameters that affect its behavior. These make up the input space. 2. Partition the input space based on a set of characteristics. Strategies: Individual partitioning of each input parameter Functional (or even output-based) modeling of the input space 3. Combine the partitions into an input domain model. In this step, identify and remove invalid combinations 4. Identify representative values 5. Derive test cases by complementing with expected output For the triangle categorization problem, these are the three input integers representing its sides. Let s call them x, y and z. 27

28 Equivalence Partitioning Example (Ammann and Offutt) 1. For each ITA, identify all parameters that affect its behavior. These make up the input space. 2. Partition the input space based on a set of characteristics. Strategies: Individual partitioning of each input parameter Functional (or even output-based) modeling of the input space 3. Combine the partitions into an input domain model. In this step, identify and remove invalid combinations 4. Identify representative values 5. Derive test cases by complementing with expected output Partition b 1 b 2 b 3 x Greater Equal to 0 Less y Greater Equal to 0 Less z Greater Equal to 0 Less 28

29 Equivalence Partitioning Example (Ammann and Offutt) 1. For each ITA, identify all parameters that affect its behavior. These make up the input space. 2. Partition the input space based on a set of characteristics. Strategies: Individual partitioning of each input parameter Functional (or even output-based) modeling of the input space 3. Combine the partitions into an input domain model. In this step, identify and remove invalid combinations 4. Identify representative values 5. Derive test cases by complementing with expected output Partition b 1 b 2 b 3 b 4 Geometric classification Scalene Isosceles Equilateral Invalid 29

30 Equivalence Partitioning Example (Ammann and Offutt) 1. For each ITA, identify all parameters that affect its behavior. These make up the input space. 2. Partition the input space based on a set of characteristics. Strategies: Individual partitioning of each input parameter Functional (or even output-based) modeling of the input space 3. Combine the partitions into an input domain model. In this step, identify and remove invalid combinations 4. Identify representative values 5. Derive test cases by complementing with expected output Partition b 1 b 2 b 3 b 4 Geometric classification Scalene Isosceles Equilateral Invalid 30

31 Equivalence Partitioning Example (Ammann and Offutt) 1. For each ITA, identify all parameters that affect its behavior. These make up the input space. 2. Partition the input space based on a set of characteristics. Strategies: Individual partitioning of each input parameter Functional (or even output-based) modeling of the input space 3. Combine the partitions into an input domain model. In this step, identify and remove invalid combinations 4. Identify representative values 5. Derive test cases by complementing with expected output Partition b 1 b 2 b 3 b 4 Geometric classification Scalene Isosceles, not equilateral Equilateral Invalid 31

32 Equivalence Partitioning Example (Ammann and Offutt) 1. For each ITA, identify all parameters that affect its behavior. These make up the input space. 2. Partition the input space based on a set of characteristics. Strategies: Individual partitioning of each input parameter Functional (or even output-based) modeling of the input space 3. Combine the partitions into an input domain model. In this step, identify and remove invalid combinations 4. Identify representative values 5. Derive test cases by complementing with expected output 32

33 Equivalence Partitioning Example (Ammann and Offutt) Partition Geometric classification Scalene Isosceles, not equilateral Equilateral Invalid x Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less y Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less z Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less 33

34 Equivalence Partitioning Example (Ammann and Offutt) Partition Geometric classification Scalene Isosceles, not equilateral Equilateral Invalid x Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less y Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less z Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less 34

35 Equivalence Partitioning Example (Ammann and Offutt) Partition Geometric classification Scalene Isosceles, not equilateral Equilateral Invalid x Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less y Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less z Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less Greater Equal to 0 Less If we feel like testing inputs from all partition combinations, we still have 30 different test cases to worry about hence, after the upcoming lab, we will look at more feasible ways of combining input from different partitions 35

36 Equivalence Partitioning Example (Ammann and Offutt) 1. For each ITA, identify all parameters that affect its behavior. These make up the input space. 2. Partition the input space based on a set of characteristics. Strategies: Individual partitioning of each input parameter Functional (or even output-based) modeling of the input space 3. Combine the partitions into an input domain model. In this step, identify and remove invalid combinations 4. Identify representative values 5. Derive test cases by complementing with expected output We leave these for you to think about 36

37 Equivalence Partitioning - Follow-Up Exercise Consider a function pararea(int x, int y, int z) that, given two sides x and y, and an angle z, of a parallelogram, calculates its area. Equivalence partitioning of inputs: x: y: z: Invalid: x<0, Valid: x>0 Invalid: y<0, Valid: y>0 y z Invalid1: z<0, Valid: 0<z<180, Invalid2: z>180 x 37

38 Equivalence Partitioning - Follow-Up Exercise Consider a function pararea(int x, int y, int z) that, given two sides x and y, and an angle z, of a parallelogram, calculates its area. Equivalence partitioning of inputs: x: y: z: Invalid: x<0, Valid: x>0 Invalid: y<0, Valid: y>0 y Invalid1: z<0, Valid: 0<z<180, Invalid2: z>180 z x Question: How to handle boundary values? 38

39 Boundary Value Analysis Experience has shown that there is a higher chance of finding defects at, or close to, partition boundaries Boundary value analysis strives to identify partition boundaries and suitable test case input on, or just outside or inside boundaries 39

40 Equivalence Partitioning - additional (more difficult) example should go here (No integers) 40

41 Lab Equivalence Partitioning 41

42 EP lab Example Partitions Perfect game: {Score=300} {Score!=300} Strike: {game with strike}{game with no strike} 42

43 EP lab Example Partitions Perfect game: {Score=300} {Score!=300} Strike: {game with strike}{game with no strike} 43

44 EP lab Example Partitions Perfect game: {Score=300} {Score!=300} Strike: {game with strike}{not a game with strike} 44

45 EP lab Some 2-minute sketch partitions Perfect game: {Score=300} {Score!=300} Strike: {game with strike}{not a game with strike} Null pointer: {null}{not null} Number of frames: {less than 10}{10}{11}{more than 11} Spare: {game with spare}{game with no spare} Syntax: {ill-formed input}{well-formed input} 45

46 Equivalence Partitioning - System-level perspective should go here Many things can be partitioned Configuration parameters Devices Time 46

47 Combining Input from Partitions into Test Cases y Ideally, we want to test all combinations. In many cases, this is however not feasible. x 47

48 Combining Input from Partitions into Test Cases Strategies y All combinations (already covered in previous slide) Each choice Pair-wise (and T-wise) Base choice x 48

49 Combining Input from Partitions into Test Cases Strategies y All combinations (already covered in previous slide) Each choice Pair-wise (and T-wise) Base choice x 49

50 Combining Input from Partitions into Test Cases Strategies y All combinations (already covered in previous slide) Each choice Pair-wise (and T-wise) Base choice x 50

51 Equivalence Partitioning - Follow-Up Exercise Consider a function pararea(int x, int y, int z) that, given two sides x and y, and an angle z, of a parallelogram, calculates its area. Equivalence partitioning of inputs: x: y: z: Invalid: x<=0, Valid: x>0 Invalid: y<=0, Valid: y>0 Invalid1: z<=0, Valid: 0<z<180, Invalid2: z>=180 y z x 51

52 Combining Input from Partitions into Test Cases Strategies y All combinations (already covered in previous slide) Each choice Pair-wise (and T-wise) Base choice x 52

53 Input value selection A Note: It has been recommended that input values are given as variables that can be randomly assigned within the partition boundaries, rather than being hard-coded. This however makes it a lot harder to determine the verdict. How is expected output expressed? 53

54 Random Testing There are some proponents of randomly generating input data, at least as a final stage of testing Considering the fact that a large percentage of defects are due to missing requirements, that might not be a bad idea However, defects are not evenly distributed 80% of defects in 20% of software Adaptive Random Testing Other problems with random testing? 54

55 Model-Based Testing Functional specifications IDENTIFY Independently testable aspects IDENTIFY Representative values Model Test cases GENERATE Test specifications Not all ITAs are suitable for equivalence partitioning. Specifically, equivalence partitioning is poorly suited to cover sequences of input, for example use case scenarios. Thus, more elaborate methods are required. 55

56 Model-Based Testing Basic idea: The intended behaviour of the software under test is modeled (more or less) formally Typically, finite state machine models are used, but many other types of models are possible Test specifications (or test cases) derived or automatically generated from the model 56

57 Model-Based Testing Example AEBS: Advance Emergency Braking System AEBS MODES AEBS NORMAL MODE OPERATION Disabled (Ignition Off) No Action Enabled Collision Warning Warning Brake Full Brake Fully Operational (Normal Mode) Not Fully Operational Disabled by Driver Brake Assist Driver Override 57

58 Model-Based Testing Example AEBS: Advance Emergency Braking System Disabled (Ignition Off) AEBS MODES Test cases derived from graph-based specification models are often based on the rationale that all transitions between the states, or some specific scenarios Enabled (paths) should be exercised during testing. Collision Warning AEBS NORMAL MODE OPERATION No Action Warning Brake Full Brake Fully Operational (Normal Mode) However, more complex criteria can naturally be applied Disabled by Driver Brake Assist Not Fully Operational Driver Override 58

59 Example- test specification generation Prime path coverage of the AEBS normal mode operation graph 1. BRAKE ASSIST, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, BRAKE ASSIST, 2. BRAKE ASSIST, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, WARNING BRAKE, BRAKE ASSIST, 3. BRAKE ASSIST, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, WARNING BRAKE, FULL BRAKE, 4. COLLISION WARNING, BRAKE ASSIST, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, 5. COLLISION WARNING, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, 6. COLLISION WARNING, NO ACTION, COLLISION WARNING, 7. COLLISION WARNING, NO ACTION, DRIVER OVERRIDE, 8. COLLISION WARNING, WARNING BRAKE, BRAKE ASSIST, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, 9. COLLISION WARNING, WARNING BRAKE, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, 10. COLLISION WARNING, WARNING BRAKE, FULL BRAKE, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, 11. COLLISION WARNING, WARNING BRAKE, FULL BRAKE, NO ACTION, COLLISION WARNING, 12. COLLISION WARNING, WARNING BRAKE, FULL BRAKE, NO ACTION, DRIVER OVERRIDE, 13. COLLISION WARNING, WARNING BRAKE, NO ACTION, COLLISION WARNING, 14. COLLISION WARNING, WARNING BRAKE, NO ACTION, DRIVER OVERRIDE, 15. DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, BRAKE ASSIST, DRIVER OVERRIDE, 16. DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, WARNING BRAKE, BRAKE ASSIST, DRIVER OVERRIDE, 17. DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, WARNING BRAKE, DRIVER OVERRIDE, 20. FULL BRAKE, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, BRAKE ASSIST, 21. FULL BRAKE, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, WARNING BRAKE, BRAKE ASSIST, 22. FULL BRAKE, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, WARNING BRAKE, FULL BRAKE, 23. FULL BRAKE, NO ACTION, COLLISION WARNING, BRAKE ASSIST, DRIVER OVERRIDE, 24. FULL BRAKE, NO ACTION, COLLISION WARNING, DRIVER OVERRIDE, 25. FULL BRAKE, NO ACTION, COLLISION WARNING, WARNING BRAKE, BRAKE ASSIST, DRIVER OVERRIDE, 26. FULL BRAKE, NO ACTION, COLLISION WARNING, WARNING BRAKE, DRIVER OVERRIDE, 27. FULL BRAKE, NO ACTION, COLLISION WARNING, WARNING BRAKE, FULL BRAKE, 28. NO ACTION, COLLISION WARNING, BRAKE ASSIST, DRIVER OVERRIDE, NO ACTION, 29. NO ACTION, COLLISION WARNING, DRIVER OVERRIDE, NO ACTION, 30. NO ACTION, COLLISION WARNING, NO ACTION, 31. NO ACTION, COLLISION WARNING, WARNING BRAKE, BRAKE ASSIST, DRIVER OVERRIDE, NO ACTION, 32. NO ACTION, COLLISION WARNING, WARNING BRAKE, DRIVER OVERRIDE, NO ACTION, 33. NO ACTION, COLLISION WARNING, WARNING BRAKE, FULL BRAKE, DRIVER OVERRIDE, NO ACTION, 34. NO ACTION, COLLISION WARNING, WARNING BRAKE, FULL BRAKE, NO ACTION, 35. NO ACTION, COLLISION WARNING, WARNING BRAKE, NO ACTION, 36. NO ACTION, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, BRAKE ASSIST, 39. WARNING BRAKE, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, WARNING BRAKE, 40. WARNING BRAKE, FULL BRAKE, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, BRAKE ASSIST, 41. WARNING BRAKE, FULL BRAKE, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, WARNING BRAKE, 42. WARNING BRAKE, FULL BRAKE, NO ACTION, COLLISION WARNING, BRAKE ASSIST, DRIVER OVERRIDE, 43. WARNING BRAKE, FULL BRAKE, NO ACTION, COLLISION WARNING, DRIVER OVERRIDE, 44. WARNING BRAKE, FULL BRAKE, NO ACTION, COLLISION WARNING, WARNING BRAKE, 45. WARNING BRAKE, NO ACTION, COLLISION WARNING, BRAKE ASSIST, DRIVER OVERRIDE, 46. WARNING BRAKE, NO ACTION, COLLISION WARNING, DRIVER OVERRIDE, 47. WARNING BRAKE, NO ACTION, COLLISION WARNING, WARNING BRAKE, AEBS NORMAL MODE OPERATION Collision Warning Brake Assist No Action Warning Brake Full Brake 18. DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, WARNING BRAKE, FULL BRAKE, DRIVER OVERRIDE, 19. DRIVER OVERRIDE, NO ACTION, DRIVER OVERRIDE, 37. WARNING BRAKE, BRAKE ASSIST, DRIVER OVERRIDE, NO ACTION, COLLISION WARNING, WARNING BRAKE, 38. WARNING BRAKE, DRIVER OVERRIDE, NO ACTION, Driver Override

60 Model-Based Testing Exercise Taken from Software Testing and Analysis, (Pezzè and Young) Task: Derive a test model from the following informal requirements document Maintenance: The Maintenance function records the history of items undergoing maintenance. If the product is covered by warranty or maintenance contract, maintenance can be requested either by calling the maintenance toll free number, or through the web site, or by bringing the item to a designated maintenance station. If the maintenance is requested by phone or web site and the customer is a US or EU resident, the item is picked up at the customer site, otherwise, the customer shall ship the item with an express courier. If the maintenance contract number provided by the customer is not valid, the item follows the procedure for items not covered by warranty. If the product is not covered by warranty or maintenance contract, maintenance can be requested only by bringing the item to a maintenance station. The maintenance station informs the customer of the estimated costs for repair. Maintenance starts only when the customer accepts the estimate. If the customer does not accept the estimate, the product is returned to the customer. Small problems can be repaired directly at the maintenance station. If the maintenance station cannot solve the problem, the product is sent to the maintenance regional headquarters (if in US or EU) or to the maintenance main headquarters (otherwise). If the maintenance regional headquarters cannot solve the problem, the product is sent to the maintenance main headquarters. Maintenance is suspended if some components are not available. Once repaired, the product is returned to the customer. (c) 2007 Ch 14, slide 60

61 (c) 2007 Mauro Pezzè & Michal Young Ch 14, slide 61

62 You will go into more complex types of models in the session on model-based testing (c) 2007 Mauro Pezzè & Michal Young Ch 14, slide 62

63 A Tip on Model-Based Testing Go to => Blog => A few pages back in time Video post: Lionel Briand on Model Based Testing and Test Automation Many other test-related video posts on the blog (e.g., by Sigrid Eldh) 63

64 Functional Test Design Summary Functional testing is based on specifications Specifications are broken down into independently testable aspects (ITAs) Depending of the nature and form of the ITA, The input space of each ITA is partitioned into equivalence partitions, or An abstract test model is derived Based on the above, test specifications are derived Test cases are generated that meet the test specifications 64

65 Functional Test Design Summary Functional testing is based on specifications Specifications are broken down into independently testable aspects (ITAs) Depending of the nature and form of the ITA, The input space of each ITA is partitioned into equivalence partitions, or An abstract test model is derived Based on the above, test specifications are derived Test cases are generated that meet the test specifications 65

66 Functional Test Design Summary Functional testing is based on specifications Specifications are broken down into independently testable aspects (ITAs) Depending of the nature and form of the ITA, The input space of each ITA is partitioned into equivalence partitions, or An abstract test model is derived Based on the above, test specifications are derived Test cases are generated that meet the test specifications 66

67 Functional Test Design Summary Functional testing is based on specifications Specifications are broken down into independently testable aspects (ITAs) Depending of the nature and form of the ITA, The input space of each ITA is partitioned into equivalence partitions, or An abstract test model is derived Based on the above, test specifications are derived Test cases are generated that meet the test specifications 67

68 Functional Test Design Summary Functional testing is based on specifications Specifications are broken down into independently testable aspects (ITAs) Depending of the nature and form of the ITA, The input space of each ITA is partitioned into equivalence partitions, or An abstract test model is derived Based on the above, test specifications are derived Test cases are generated that meet the test specifications 68

69 Functional Test Design Summary Functional testing is based on specifications Specifications are broken down into independently testable aspects (ITAs) Depending of the nature and form of the ITA, The input space of each ITA is partitioned into equivalence partitions, or An abstract test model is derived Based on the above, test specifications are derived Test cases are generated that meet the test specifications 69

70 Functional Test Design Summary Functional testing is based on specifications Specifications are broken down into independently testable aspects (ITAs) Depending of the nature and form of the ITA, The input space of each ITA is partitioned into equivalence partitions, or An abstract test model is derived Based on the above, test specifications are derived Test cases are generated that meet the test specifications 70

71 Homework (until next time) 1. Finish the Labs 2. Equivalence partitioning of your own proposal system Think of any software system and one independently testable aspect (ITA) of that software. What is the input space? Note that you might need to broaden your view of input. Anything than can potentially affect the behaviour of the system can be considered an input. Partition the input space based on a suitable set of characteristics and derive an input domain model. If equivalence partitioning is not a suitable technique for your selected case, choose a different ITA. 71