CS3773 Software Engineering

Transcription

1 CS3773 Software Engineering Lecture 2 Test Case Design Software Testing Techniques Functional testing is applied to demonstrate the system meets its requirements, it is also called black-box testing Testers are only concerned with the functionality, performance, and dependability The system is treated as a black box that takes input and produces output Structural testing is applied to expose defects and tests are derived from the knowledge of the internal workings of items Testers understand the algorithm and the structure of systems The system is treated as a white box 2 Functional Testing Boundary Value Analysis Boundary value testing Errors tend to occur near the extreme values of an input Boundary value analysis variables Robustness testing Boundary value analysis focuses on the boundary of the input Worst case testing space to identity test cases Special value testing Boundary value analysis selects input variable values at their Equivalence class testing Minimum Decision table based testing Just above the minimum A nominal value Just below the maximum Maximum 3 4

2 Example of Boundary Value Analysis Single Assumption for Boundary Value Analysis Assume a program accepting two inputs y1 and y2, such that a < y1< b and c < y2 < d Boundary value analysis is also augmented by the single fault assumption principle y2 d c Failures occur rarely as the result of the simultaneous occurrence of two (or more) faults In this respect, boundary value analysis test cases can be obtained by holding the values of all but one variable at their nominal values, and letting that variable assume its extreme values a b y1 5 Generalization of Boundary Value Analysis Limitations of Boundary Value Analysis The basic boundary value analysis can be generalized in two Boundary value analysis works well when the program to be ways: tested is a function of several independent variables that By the number of variables - (4n +1) test cases for n variables represent bounded physical quantities By the kinds of ranges of variables Boundary value analysis selected test data with no Programming language dependent consideration of the function of the program, nor of the Bounded discrete semantic meaning of the variables Unbounded discrete (no upper or lower bounds clearly defined) Logical variables We can distinguish between physical and logical type of variables as well (e.g. temperature, pressure speed, or PIN numbers, telephone numbers etc.) 7 8

3 Robustness Testing Example of Robustness Testing Robustness testing is a simple extension of boundary value analysis In addition to the five boundary value analysis values of variables, we add values slightly greater that the maximum (max+) and a value slightly less than the minimum (min-) The main value of robustness testing is to force attention on exception handling y2 d c In some strongly typed languages values beyond the predefined range will cause a run-time error a b y Worst Case Testing Example of Worst Case Testing In worst case testing we reject the single fault assumption and we are interested what happens when more than one variable has an extreme value Considering that we have five different values that can be considered during boundary value analysis testing for one variable, now we take the Cartesian product of these possible values for 2, 3, n variables y2 d c We can have 5 n test cases for n input variables The best application of worst case testing is where physical variables have numerous interactions 12 a b y1

4 Special Value Testing Equivalence Class Testing Special value testing is probably the most widely practiced form of functional testing, most intuitive, and least uniform Utilizes domain knowledge and engineering judgment about program s soft spots to devise test cases Event though special value testing is very subjective on the generation of test cases, it is often more effective on revealing program faults The use of equivalence class testing has two motivations: Sense of complete testing Avoid redundancy Equivalence classes form a partition of a set that is a collection of mutually disjoint subsets whose union is the entire set Two important implications for testing: The entire set is represented provides a form of completeness The disjointness assures a form of non-redundancy Example of Equivalence Class Testing Example of Equivalence Class Testing The program P with 3 inputs: a, b and c and the corresponding input domains are A, B, and C A = A1 A2 A3 B = B1 B2 C = C1 C2 C3 C4 Define a 1, a 2 and a 3 as: let a i be a representative or typical value within its respective equivalence class (e.g. the midpoint in a linear equivalence class). similarly define b i and c i. Test cases can be stated for the inputs <a,b,c> in terms of the representative points The basic idea behind the techniques is that one point within an equivalence class is just as good as any other point within the same class 15 1

5 Decision Table Decision Table Usage Decision tables make it easy to observe that all possible The use of the decision-table model is applicable when : conditions are accounted for Specification is given or can be converted to a decision table Decision tables can be used for: Specifying complex program logic Generating test cases (Also known as logic-based testing) Logic-based testing is considered as: structural testing when applied to structure, i.e. control flow graph of an implementation functional testing when applied to a specification The order in which the predicates are evaluated does not affect the interpretation of the rules or resulting action The order of rule evaluation has no effect on resulting action Once a rule is satisfied and the action selected, no other rule need be examined The order of executing actions in a satisfied rule is of no consequence Example of Decision Table Structural Testing Conditions Actions Printer does not print Y Y Y Y N N N N A red light is flashing Y Y N N Y Y N N Printer is unrecognized Y N Y N Y N Y N Heck the power cable X Check the printer-computer cable X X Ensure printer software is installed X X X X Check/replace ink X X X X Flow Graph Testing Basis Path Testing Decision-to-Decision Path Test Coverage Metrics Data Flow Testing Check for paper jam X X 19 Printer Troubleshooting 20

6 21 Program Graph Given a program written in an imperative programming language, its Program Graph, is a directed labeled graph in which nodes are either groups of entire statements or fragments of a statement, and edges represent flow of control by P. Jorgensen If i, j, are nodes (basic block) in the program graph, there is an edge from node i, to node j in the program graph if an only if, the statement corresponding to node j, can be executed immediately after the last statement of the group of statement(s) that correspond to node i. Determine the Basic Block FindMean (FILE ScoreFile) { float SumOfScores = 0.0; int NumberOfScores = 0; 1 float Mean=0.0; float Score; Read(ScoreFile, Score); 2 while (! EOF(ScoreFile) { 3 if (Score > 0.0 ) { SumOfScores = SumOfScores + Score; 4 NumberOfScores++; } 5 Read(ScoreFile, Score); } /* Compute the mean and print the result */ 7 if (NumberOfScores > 0) { Mean = SumOfScores / NumberOfScores; printf( The mean score is %f\n, Mean); 8 } else printf ( No scores found in file\n ); 9 } 22 Example of Logic Flow Graph Path Testing Start 1 F 2 T 3 T F 4 5 Path Testing is focusing on test techniques that are based on the selection of test paths though a program graph. If the set of paths is properly chosen, then we can claim that we have achieved a measure of test thoroughness The fault assumption for path testing techniques is that something has gone wrong with the software that makes it take a different path than the one intended 23 7 T F 8 9 Exit 24 Structurally, a path is a sequence of statements in a program unit. Semantically, a path is an execution instance of the program unit. For software testing we are interested in entry-exit paths

7 Example for a Simple CFG Path Testing Process V(G) = 3 1 Unit Input: Source code and a path selection criterion Process: Generation of a Control Flow Graph (CFG) Selection of Paths Generation of Test Input Data Feasibility Test of a Path Evaluation of Program s Output for the Selected Test Cases Basis set: 1, 2, 3, 4,, 7 1, 2, 3, 4, 5, 4,, 7 1, 2,, 7 x = z+5 z = 4*3-y if(x > z) goto A; for( u=0; u < x; u++) { z = z+1; }; A: y = z + k z = z+1 u++ R1 5 R3 t x > z f x = z+5 z = 4*3-y u = 0 R2 u < x t f 7 y = z+k 25 2 Decision-to-Decision Path Decision-to-Decision Path Graph A DD-Path is a chain obtained from a program graph, where a chain is a path in which the initial and terminal nodes are distinct, and every interior node has indegree = 1, and outdegree = 1 Given a program written in an imperative language, its DD- Path graph is a labeled directed graph, in which nodes are DD-Paths of its program graph, and edges represent control flow between successor DD-Paths Internal node is 2-connected to every other node in the chain, and there are no instances of 1- or 3- connected nodes. Feasibility Test of a Path In this respect, a DD-Path is a condensation graph. For example 2-connected program graph nodes are collapsed to a single DD-Path graph node Evaluation of Program s Output for the Selected Test Cases DD-Paths are used to create DD-Path Graphs

8 Example of Path Graph Test Coverage B L first A A D C F E H G I J K Program Graph Nodes DD-Path Name Case # 4 first A 5 9 B 4 10 C 4 11 D E 5 15 F 4 1 G 3 17 H 4 18 I 3 19 J 4 20 K 3 21 L 4 22 last 2 The motivation of using DD-paths is that they enable very precise descriptions of test coverage In our quest to identify gaps and redundancy in our test cases as these are used to exercise (test) different aspects of a program we use formal models of the program structure to reason about testing effectiveness Test coverage metrics are a device to measure the extend to which a set of test cases covers a program last Test Coverage Metrics Data Flow Testing 31 Metric C 0 C 1 C 1 P C 2 C d C MCC C i k C stat C Description of Coverage Every Statement Every DD-Path Every predicate to each outcome C 1 Coverage + loop coverage C 1 Coverage + every dependent pair of DD-Paths Multiple condition coverage Every program path that contains up to k repetitions of a loop (usually k=2) Statistically significant fraction of paths All possible execution paths 32 Data flow testing refers to a category of structural testing techniques that focus on the points of the code variables obtain values (are defined) and the points of the program these variables are referenced (are used) Around faults that may occur when a variable is defined and referenced in not a proper way A variable is defined but never used A variable is used but never defined A variable that is defined twice (or more times) before it is used Parts of a program that constitute a slice a subset of program statements that comply with a specific slicing criterion (i.e. all program statements that are affected by variable x at point P)

9 Data Flow Testing Process Reading Assignments Data-flow testing involves selecting entry/exit paths with the objective of covering certain data definition and use patterns, commonly known as data-flow criteria An outline of data-flow testing is as follows: Draw a data flow graph for the program Select data-flow testing criteria Identify paths in the data-flow graph to satisfy the selection criteria Produce test cases for the selected paths Sommerville s Book Chapter 23, Software Testing 33 34