Quality Assurance in Software Development

Size: px
Start display at page:

Download "Quality Assurance in Software Development"

Transcription

1 Quality Assurance in Software Development Qualitätssicherung in der Softwareentwicklung A.o.Univ.-Prof. Dipl.-Ing. Dr. Bernhard Aichernig Institute for Software Technology Graz University of Technology Austria Summer Term / 19

2 Agenda 1 Organisation 2 Contents 3 Quality of SW 2 / 19

3 Quality Assurance in Summer Term 2013 Lecture: Tue 11:15 12:00 (i12) Exercise: Tue 12:00 12:45 (i12) Optional: Fri 14:15 15:00 (i12) Registration via tugonline strict! Two student assistants: Sandra Fruhmann Martin Tappler 3 / 19

4 Additional Resources WWW: Newsgroup: tu-graz.lv.qs Subject: [QS13] 4 / 19

5 Marking Marking iff registered on (Dreamspark licence) Written exam (Klausur) : (50 %). Three practical tasks (groups of 3): Task 1: publication (10 %) Task 2: publication (10 %) Task 3: publication (15 %) Task 4: publication (15 %) Oral interviews about tasks: after Task 3 Exam and exercises count each 50% Positive if > 50% total points 5 / 19

6 Marking: Nachklausur Date: Beginning of July After excused absence at exam e.g. due to illness with sick certificate (ärztlicher Bestätigung) work is no excuse (take leave!) For all negative! 6 / 19

7 What we will not cover? Quality management: Process management, e.g. Scrum Process improvement, e.g. ISO 9000 Test management, e.g. IBM Rational Quality Manager Version control, e.g. svn, git... 7 / 19

8 General Aims of the Course Getting familiar with quality assurance techniques Raising the awareness for quality of software Practising fault-based thinking Being able to distinguish process-oriented and technical quality assurance Challenging common quality criteria Realising the relations between the foundations of software engineering and quality assurance. 8 / 19

9 Specific Learning Targets and Skills Getting familiar with a professional IDE Writing good tests :( Generating good tests :) Specifying test oracles Using and understanding state-of-the-art test case generators Test coverage metrics "Don t write test cases, generate them!" (John Hughes) 9 / 19

10 Course Contents Introduction and motivation Challenges of software testing Coverage: Measuring the quality of test cases Testing with Concolic Execution Contracts as test oracles Model-based Testing Proof-based Development Tools (exercise): MS Visual Studio Ultimate 2010 MS Pex.NET Code Contracts MS Spec Explorer You will get a DreamSpark licence (registration)! 10 / 19

11 Quality? Today, often a sufficient level of correctness efficiency costs cannot be guaranteed. Vision: Development methods for SW with warranty. 11 / 19

12 Bugs Part of engineering jargon for many decades: Moth trapped in relay of Mark II (Hopper 1946) Little faults and difficulties (Edison 1878): Software bugs Relay #70 Panel F (moth) in relay. First actual case of bug being found. 12 / 19

13 Bugs Part of engineering jargon for many decades: Moth trapped in relay of Mark II (Hopper 1946) Little faults and difficulties (Edison 1878): Software bugs Definition A software bug is the common term used to describe an error, flaw, mistake, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways. (Wikipedia 2012) 12 / 19

14 Some Bugs Become Famous! Ariane 5 test flight (1996) out of control due to software failure controlled destruction! Loss of money and time satellites research (TU Graz) Dijkstra (EWD 1036): call it error, not bug a programmer created it 13 / 19

15 Binary search bug in Java Some Bugs Hide for a Long Time! JDK 1.5 library (2006) out of boundary access of large arrays due to integer overflow 9 years undetected Algorithm was proven correct! Programming Pearls [Bentley86, Bentley00] assuming infinite integers :( 1 public static 2 int binarysearch(int[] a,int key) 3 { 4 int low = 0; 5 int high = a.length - 1; 6 7 while (low <= high) { 8 int mid = (low + high) / 2; 9 int midval = a[mid]; if (midval < key) 12 low = mid + 1; 13 else if (midval > key) 14 high = mid - 1; 15 else 16 return mid; // key found 17 } 18 return -(low + 1); // key not found 19 } Beware of bugs in the above code; I have only proved it correct, not tried it. [Knuth77] 14 / 19

16 Binary search bug in Java Some Bugs Hide for a Long Time! JDK 1.5 library (2006) out of boundary access of large arrays due to integer overflow 9 years undetected Algorithm was proven correct! Programming Pearls [Bentley86, Bentley00] assuming infinite integers :( 1 public static 2 int binarysearch(int[] a,int key) 3 { 4 int low = 0; 5 int high = a.length - 1; 6 7 while (low <= high) { 8 int mid = (low + high) >>> 1; 9 int midval = a[mid]; if (midval < key) 12 low = mid + 1; 13 else if (midval > key) 14 high = mid - 1; 15 else 16 return mid; // key found 17 } 18 return -(low + 1); // key not found 19 } Beware of bugs in the above code; I have only proved it correct, not tried it. [Knuth77] 14 / 19

17 Risks for the Public due to SW The Risks Digest 15 / 19

18 Edsger W. Dijkstra In academia, in industry, and in the commercial world, there is a widespread belief that computing science as such has been all but completed and that, consequently, computing has matured from a theoretical topic for the scientists to a practical issue for the engineers, the managers, and the entrepreneurs. [...]... then he characterises the software crisis and concludes: I would therefore like to posit that computing s central challenge, "How not to make a mess of it," has not been met. 16 / 19

19 Edsger W. Dijkstra (cont.) On the contrary, most of our systems are much more complicated than can be considered healthy, and are too messy and chaotic to be used in comfort and confidence. The average customer of the computing industry has been served so poorly that he expects his system to crash all the time, and we witness a massive worldwide distribution of bug-ridden software for which we should be deeply ashamed. (Communications of the ACM, Mar 2001) 17 / 19

20 Why? A possible explanation: Quality assurance requires a certain degree of (healthy) redundancy extra costs! (appr. 50% for critical systems) Examples of redundancy: process management documentation pair programming modelling test cases redundant components in fault-tolerant systems (accepted) 18 / 19

21 Demo: Triangle Example Writing unit tests in VS 2010 Code coverage in VS 2010 Automatic test case generation in Pex How good are the tests? 19 / 19

22 Quality Assurance in Software Development Qualitätssicherung in der Softwareentwicklung A.o.Univ.-Prof. Dipl.-Ing. Dr. Bernhard Aichernig Institute for Software Technology Graz University of Technology Austria Summer Term / 47

23 Agenda 1 The Challenge of Software Testing 2 Definitions 3 Functional testing strategies Boundary Value Testing Random Testing Equivalence Class Testing 2 / 47

24 The Challenge of Software Testing Correctness Requirements Development Process 3 / 47

25 A little joke A mathematician, a physicist, and an engineer are told: All odd numbers are prime. The mathematician says, That s silly, nine is a non-prime odd number. The physicist says, Let s see, 3 is prime, 5 is prime, 7 is prime looks like it s true. The engineer says, Let s see, 3 is prime, 5 is prime, 7 is prime, 9 is prime, 11 is prime looks like it s true. 4 / 47

26 Becoming more serious The little joke illustrates us the following challenges in testing: oracle problem: How to predict the outcome of a test-case? incompleteness: exhaustive testing is usually infeasable. How much testing is needed? test case selection: Are the chosen test cases adequate? correctness of test cases: Are the test cases correctly designed? 5 / 47

27 Testing and Correctness Testing can only demonstrate the presence of errors, not their absence (E. Dijkstra, 1972) Reason: discrete nature of software Testing of all possible inputs would be necessary Exhaustive testing is infeasible Representative test cases have to be selected (testing strategy) 6 / 47

28 Testing and Requirements Quality of testing strongly depends on the quality of requirement documents Qualit"at des Testens h"angt stark von Qualit"at der Anforderungsdokumente ab. Requiremens that are not documented cannot be tested systematically 60 % of all bugs in the requirements phase are due to missing requirements Requirements written in natural language are often incomplete ambiguous inconsistent Formal specifications and formal models help! 7 / 47

29 Testing in the Development Process In the classical waterfall model testing is the last phase. Danger: if project is delayed, testing gets cut Dramatic: most of the large SW projects overrun their deadline Consequence: the test cases have to be designed as early a possible, as soon as the SW requirements are known. this lead to agile software processes High testing costs! Test automation is needed! 1/3 1/2 of the resources in big projects 8 / 47

30 Self test Write a set of test cases that you feel would adequately test this program (Myers, The Art of Software Testing): The program reads three integer values from an input stream. The three values are interpreted as representing the lengths of the sides of a triangle. The program prints a message that states whether the triangle is scalene, isosceles, or equilateral. 9 / 47

31 Self test: questions Do you have... 1 a test case that represents a valid scalene triangle? Test cases (1, 2, 3) and (2, 5, 10) are not valid. 2 a test case that represents a valid equilateral triangle? 3 a test case that represents a valid isosceles triangle? 4 at least three test cases that represent valid isosceles triangles such that you have tried all three permutations of two equal sides (e.g., (3, 3, 4), (3, 4, 3), and (4, 3, 3))? 5 a test case in which one side has a zero value? 6 a test case in which one side has a negative value? 7 a test case with three integers greater than zero such that the sum of two of the numbers is equal to the third? 10 / 47

32 Self test: questions Do you have... 1 a test case that represents a valid scalene triangle? Test cases (1, 2, 3) and (2, 5, 10) are not valid. 2 a test case that represents a valid equilateral triangle? 3 a test case that represents a valid isosceles triangle? 4 at least three test cases that represent valid isosceles triangles such that you have tried all three permutations of two equal sides (e.g., (3, 3, 4), (3, 4, 3), and (4, 3, 3))? 5 a test case in which one side has a zero value? 6 a test case in which one side has a negative value? 7 a test case with three integers greater than zero such that the sum of two of the numbers is equal to the third? 10 / 47

33 Self test: questions Do you have... 1 a test case that represents a valid scalene triangle? Test cases (1, 2, 3) and (2, 5, 10) are not valid. 2 a test case that represents a valid equilateral triangle? 3 a test case that represents a valid isosceles triangle? 4 at least three test cases that represent valid isosceles triangles such that you have tried all three permutations of two equal sides (e.g., (3, 3, 4), (3, 4, 3), and (4, 3, 3))? 5 a test case in which one side has a zero value? 6 a test case in which one side has a negative value? 7 a test case with three integers greater than zero such that the sum of two of the numbers is equal to the third? 10 / 47

34 Self test: questions Do you have... 1 a test case that represents a valid scalene triangle? Test cases (1, 2, 3) and (2, 5, 10) are not valid. 2 a test case that represents a valid equilateral triangle? 3 a test case that represents a valid isosceles triangle? 4 at least three test cases that represent valid isosceles triangles such that you have tried all three permutations of two equal sides (e.g., (3, 3, 4), (3, 4, 3), and (4, 3, 3))? 5 a test case in which one side has a zero value? 6 a test case in which one side has a negative value? 7 a test case with three integers greater than zero such that the sum of two of the numbers is equal to the third? 10 / 47

35 Self test: questions Do you have... 1 a test case that represents a valid scalene triangle? Test cases (1, 2, 3) and (2, 5, 10) are not valid. 2 a test case that represents a valid equilateral triangle? 3 a test case that represents a valid isosceles triangle? 4 at least three test cases that represent valid isosceles triangles such that you have tried all three permutations of two equal sides (e.g., (3, 3, 4), (3, 4, 3), and (4, 3, 3))? 5 a test case in which one side has a zero value? 6 a test case in which one side has a negative value? 7 a test case with three integers greater than zero such that the sum of two of the numbers is equal to the third? 10 / 47

36 Self test: questions Do you have... 1 a test case that represents a valid scalene triangle? Test cases (1, 2, 3) and (2, 5, 10) are not valid. 2 a test case that represents a valid equilateral triangle? 3 a test case that represents a valid isosceles triangle? 4 at least three test cases that represent valid isosceles triangles such that you have tried all three permutations of two equal sides (e.g., (3, 3, 4), (3, 4, 3), and (4, 3, 3))? 5 a test case in which one side has a zero value? 6 a test case in which one side has a negative value? 7 a test case with three integers greater than zero such that the sum of two of the numbers is equal to the third? 10 / 47

37 Self test: questions Do you have... 1 a test case that represents a valid scalene triangle? Test cases (1, 2, 3) and (2, 5, 10) are not valid. 2 a test case that represents a valid equilateral triangle? 3 a test case that represents a valid isosceles triangle? 4 at least three test cases that represent valid isosceles triangles such that you have tried all three permutations of two equal sides (e.g., (3, 3, 4), (3, 4, 3), and (4, 3, 3))? 5 a test case in which one side has a zero value? 6 a test case in which one side has a negative value? 7 a test case with three integers greater than zero such that the sum of two of the numbers is equal to the third? 10 / 47

38 Self test: questions (cont.) 8 at least three test cases in Category 7 such that you have tried all three permutations where the length of one side is equal to the sum of the lengths of the other two sides (e.g., (1, 2, 3), (1, 3, 2), and (3, 1, 2))? 9 a test case with three integers greater than zero such that the sum of two of the numbers is less than the third (e.g., (1, 2, 4)) 10 at least three test cases in Category 9 such that you have tried all three permutations (e.g., (1, 2, 4), (1, 4, 2), and (4, 1, 2))? 11 a test case in which all sides are zero? 12 at least one test case specifying noninteger values? 13 at least one test case specifying the wrong number of values? 14 for each test case the expected outputs specified? 11 / 47

39 Self test: questions (cont.) 8 at least three test cases in Category 7 such that you have tried all three permutations where the length of one side is equal to the sum of the lengths of the other two sides (e.g., (1, 2, 3), (1, 3, 2), and (3, 1, 2))? 9 a test case with three integers greater than zero such that the sum of two of the numbers is less than the third (e.g., (1, 2, 4)) 10 at least three test cases in Category 9 such that you have tried all three permutations (e.g., (1, 2, 4), (1, 4, 2), and (4, 1, 2))? 11 a test case in which all sides are zero? 12 at least one test case specifying noninteger values? 13 at least one test case specifying the wrong number of values? 14 for each test case the expected outputs specified? 11 / 47

40 Self test: questions (cont.) 8 at least three test cases in Category 7 such that you have tried all three permutations where the length of one side is equal to the sum of the lengths of the other two sides (e.g., (1, 2, 3), (1, 3, 2), and (3, 1, 2))? 9 a test case with three integers greater than zero such that the sum of two of the numbers is less than the third (e.g., (1, 2, 4)) 10 at least three test cases in Category 9 such that you have tried all three permutations (e.g., (1, 2, 4), (1, 4, 2), and (4, 1, 2))? 11 a test case in which all sides are zero? 12 at least one test case specifying noninteger values? 13 at least one test case specifying the wrong number of values? 14 for each test case the expected outputs specified? 11 / 47

41 Self test: questions (cont.) 8 at least three test cases in Category 7 such that you have tried all three permutations where the length of one side is equal to the sum of the lengths of the other two sides (e.g., (1, 2, 3), (1, 3, 2), and (3, 1, 2))? 9 a test case with three integers greater than zero such that the sum of two of the numbers is less than the third (e.g., (1, 2, 4)) 10 at least three test cases in Category 9 such that you have tried all three permutations (e.g., (1, 2, 4), (1, 4, 2), and (4, 1, 2))? 11 a test case in which all sides are zero? 12 at least one test case specifying noninteger values? 13 at least one test case specifying the wrong number of values? 14 for each test case the expected outputs specified? 11 / 47

42 Self test: questions (cont.) 8 at least three test cases in Category 7 such that you have tried all three permutations where the length of one side is equal to the sum of the lengths of the other two sides (e.g., (1, 2, 3), (1, 3, 2), and (3, 1, 2))? 9 a test case with three integers greater than zero such that the sum of two of the numbers is less than the third (e.g., (1, 2, 4)) 10 at least three test cases in Category 9 such that you have tried all three permutations (e.g., (1, 2, 4), (1, 4, 2), and (4, 1, 2))? 11 a test case in which all sides are zero? 12 at least one test case specifying noninteger values? 13 at least one test case specifying the wrong number of values? 14 for each test case the expected outputs specified? 11 / 47

43 Self test: questions (cont.) 8 at least three test cases in Category 7 such that you have tried all three permutations where the length of one side is equal to the sum of the lengths of the other two sides (e.g., (1, 2, 3), (1, 3, 2), and (3, 1, 2))? 9 a test case with three integers greater than zero such that the sum of two of the numbers is less than the third (e.g., (1, 2, 4)) 10 at least three test cases in Category 9 such that you have tried all three permutations (e.g., (1, 2, 4), (1, 4, 2), and (4, 1, 2))? 11 a test case in which all sides are zero? 12 at least one test case specifying noninteger values? 13 at least one test case specifying the wrong number of values? 14 for each test case the expected outputs specified? 11 / 47

44 Self test: questions (cont.) 8 at least three test cases in Category 7 such that you have tried all three permutations where the length of one side is equal to the sum of the lengths of the other two sides (e.g., (1, 2, 3), (1, 3, 2), and (3, 1, 2))? 9 a test case with three integers greater than zero such that the sum of two of the numbers is less than the third (e.g., (1, 2, 4)) 10 at least three test cases in Category 9 such that you have tried all three permutations (e.g., (1, 2, 4), (1, 4, 2), and (4, 1, 2))? 11 a test case in which all sides are zero? 12 at least one test case specifying noninteger values? 13 at least one test case specifying the wrong number of values? 14 for each test case the expected outputs specified? 11 / 47

45 Naming the Wrong IEEE Standard Glossary of Software Engineering Terminology: Mistake: a human action that produces an incorrect result. Example: an incorrect action taken by the programmer. Fault: an incorrect step, process, or data definition in a computer progra (also defect, bug). Error: the difference between a computed, observed, or measured value or condition and the true, specified, or theoretically correct value or condition. Failure: the inability of a system or component to fulfill its required functions within specified performance requirements. Dijkstra was against calling it a bug. 12 / 47

46 Naming the Wrong: Example Consider a statement x = y + x; By a mistake a programmer changes it to x = y x, thus characterising a fault. Executing the fault with x = 0 does not lead to an error, and consequently not to an observable failure. 13 / 47

47 Testing, Test, Test Case testing: the act of designing, debugging, and executing test cases.. test: A test is the act of exercising software with test cases. A test has two distinctive goals: to find failures or to demonstrate correct execution. test case: A test case has an identity and is associated with program behaviour. A test case also has a set of inputs and a list of expected outputs. The output portion of a test case is frequently overlooked, which is unfortunate, because it is often the hard part. 14 / 47

48 Black & White Two fundamental approaches are used to identify test cases: functional testing: functional testing is based on the view that any program can be considered to be a function that maps values from its input domain to values in its output range (black box testing). Test cases are identified using the function s specification. structural testing: based on how the function is actually implemented (white box testing). 15 / 47

49 Why We Test validation: the process of evaluating an object to demonstrate that it meets the user requirements. verification: in the ANSI/IEEE Std three possible definitions of verification are given: 1 the process of reviewing, inspecting, testing, etc., if objects, processes, services, documents satisfy the specified requirements. 2 the process of evaluating if an object in a given phase in the software lifecycle, meets the requirements established in the previous phase. 3 formal correctness proofs of programs. falsification: the process of evaluating an object to demonstrate that it does not meet requirements. 16 / 47

50 When We Test The following kinds of tests can be distinguished during sw-lifecycle: unit tests: testing of units: modules, classes, components integration tests: tests that explore the interaction and consistency of successfully tested components. system tests: testing of the whole system in order to explore behaviors that can t be done by unit or integration testing: performance, data integrity, storage management, security, reliability. acceptance tests: does the system satisfy the requirements? Part of the contract. regression tests: tests after changes to prevent unintended changes. 17 / 47

51 The V-model 18 / 47

52 Functional Testing: Programs as Functions In functional testing, a program is considered as a function p : Input Output mapping values from its domain (Input) to values in its range (Output). Types permit the definitions of domain and range. Preconditions restrict the domain further. Postconditions restrict the range. 19 / 47

53 Boundary Value Analysis Focuses on the boundary of input space (domain) to identify test cases. Rationale: errors tend to occur near the extreme values of an input variable. Strategy: input values at 1 minimum 2 just above minimum 3 a nominal value 4 just below maximum 5 maximum 20 / 47

54 Boundary Value Analysis: 2 Input Variables Boundary value test cases for a function of two variables i1 and i2: i2 d c a b i1 Single fault assumption! (no relation between parameters) 21 / 47

55 Generalizing Boundary Value Analysis Number of test cases: 4n+ 1 for a function with n input variables. Different ranges: Triangle problem: e.g. min = 1, max = MAXINT Date input: date corresponds to 3 input variables; e.g. min = , max = Boolean input: does not make sense! Limitations: only good when program is a function of several independent variables that represent bounded physical variables. good: temperature, coordinates bad: date, pin code, telephone number 22 / 47

56 Robustness Testing Exceeding the limits slightly: i2 d c a b i1 Most interesting with expected outputs not inputs! 23 / 47

57 Robustness Testing Examples Robustness testing forces attention on exception handling: Exceeding angle of attack of a plane: Will it stall? Exceeding load capacity of an elevator: Hopefully only a warning!? With strongly typed languages robustness testing may be very awkward: C leads to run-time errors. Exception handling (C#, Java) mandates robustness testing. Specifications: test cases such that pre(in) holds. 24 / 47

58 Worst-Case Testing Rejecting the single-fault assumption: i2 d c a Number of test cases: 5 n, for n input variables! b i1 25 / 47

59 Special Value Testing tester uses his domain knowledge, experience with similar programs, and information about soft spots to devise test cases. most widely practiced form. also called ad hoc testing. very dependent on the abilities of the tester! 26 / 47

60 Example: Commission Rifle 1 salesman sales rifle locks 2, stocks 3, and barrels 4. Locks costs $45, stocks $30, barrels $25. Has to sell at least one complete rifle per month. Has to sell at most 70 locks, 80 stocks, and 90 barrels per month. The gunsmith computes the salesman s commission: 10% on sales up to (and including) $1000, 15% on the next $800, and 20% on any sales in excess of $ Gewehr 2 Verschluss 3 Schaft 4 Lauf 27 / 47

61 Commission: Output Boundary Analysis Case Locks Stocks Barrels Sales ($) Comm ($) Comment output minimum output minimum output minimum output minimum midpoint border point border point border point border point border point border point border point midpoint 28 / 47

62 Commission: Output Boundary Analysis (cont.) Case Locks Stocks Barrels Sales ($) Comm ($) Comment border point border point border point border point border point border point border point midpoint output maximum output maximum output maximum output maximum 29 / 47

63 Random Testing Idea: Random number generator to pick test case values. Motivation: avoiding a form of bias in testing. Question: How many random test cases are sufficient? Answer by 1 coverage criteria 2 reliability models Pro: fully automatic test case generation. Con: more test cases needed to reach structural coverage. At least 2 decades of literature in academia (statistical testing methods). 30 / 47

64 Random Testing of Commission Test Cases 10% Commission 15% Commission 20% Commission avg. 1.01% 3.62% 95.37% 31 / 47

65 Random Testing of Triangle Test Cases NoTriangle Scalene Isosceles Equilateral avg % 47.87% 2.29% 0.01% Program generates random test cases until at least one of each output occurs. Here an upper limit of 200 has been chosen. 32 / 47

66 Software Reliability Engineering reliability = 1 Decide target reliability. number of failures logical unit 2 Operational profile: What use cases (functions) are often executed? 3 More tests for highly frequent use cases. 4 Additional tests for high-risk use cases. 5 Statistical failure models, and the failures detected provide answers, when the target reliability is reached. Book: John Musa, Software Reliability Engineering, 2nd. edition, / 47

67 Equivalence Class Testing Main idea: : Test hypothesis: classes of inputs show equal behaviour Select one test case from each equivalence class. Main motivation: adequate set of test cases (having a sense of completeness) avoiding redundancy! We distinguish: weak / strong equivalence class testing (single vs. multiple faults) normal / robust equivalence class testing 34 / 47

68 Equivalence Classes form a set partition. Definition: (Partition) Given a set A and a set of n subsets A 1, A 2,...,A n of A, the subsets form a partition of A iff and A 1 A 2 A n = A i, j {1,...,n} i j A i A j = {} 1st property provides a form of completeness, 2nd property ensures a form of nonredundancy. 35 / 47

69 Choosing Equivalence Classes Key to Strategy: Choice of equivalence relation. Deduced from the problem specification Triangle: one test case for equilateral (5, 5, 5), we don t expect much more from test cases, like (6, 6, 6), (7, 7, 7). often by guessing a likely implementation Formal interface specifications (contracts): choice of equivalence classes can be automated. 36 / 47

70 Weak Normal Equivalence Class Testing i2 d c a b i1 37 / 47

71 Strong Normal Equivalence Class Testing i2 d c a b i1 38 / 47

72 Weak Robust Equivalence Class Testing i2 d c a b i1 39 / 47

73 Strong Robust Equivalence Class Testing i2 d c a b i1 40 / 47

74 Triangle Example Note that four possible outpus can occur. We can use these to identify output (range) equivalence classes: 1 {(a, b, c) istriangle(a, b, c) = NoTriangle} 2 {(a, b, c) istriangle(a, b, c) = Scalene} 3 {(a, b, c) istriangle(a, b, c) = Isosceles} 4 {(a, b, c) istriangle(a, b, c) = Equilateral} 41 / 47

75 Triangle: Normal Equivalence Class Testing The four weak normal equivalence class test cases are: Test Case a b c Expected Output WN NoTriangle WN Scalene WN Isosceles WN Equilateral No valid subintervals of variables a, b, and c exists strong normal equivalence testing does not add test cases. 42 / 47

76 Triangle: Robust Equivalence Class Testing Let s assume the valid input data has been restricted to [0, 200], and error messages indicate if input is out of its range, then the additional weak robust equivalence class test cases are: Test Case a b c Expected Output WR Value a out of range! WR Value b out of range! WR Value c out of range! WR Value a out of range! WR Value b out of range! WR Value c out of range! 43 / 47

77 Triangle: Robust Equivalence Class Testing We show one corner of the cube of strong robust equivalence class test cases: Test Case a b c Expected Output WS Value a out of range! WS Value b out of range! WS Value c out of range! WS Value a, b out of range! WS Value b, c out of range! WS Value a, c out of range! WS Value a, b, c out of range! 44 / 47

78 Triangle: Input Equivalence Class Equivalence class testing is sensitive to the equivalence relation chosen. Equivalence classes based on the input domain: 1 {(a, b, c) a = b b = c} 2 {(a, b, c) a = b a c} 3 {(a, b, c) a b a = c} 4 {(a, b, c) b = c a b} 5 {(a, b, c) a b a c b c} 45 / 47

79 Triangle: Input Equivalence Class (cont.) Furthermore, we can apply the triangle property to see if they consitute a triange: 6 {(a, b, c) a b + c} 7 {(a, b, c) b a+c} 8 {(a, b, c) c a+b} 46 / 47

80 Triangle: Input Equivalence Class (cont.) We could be even more thorough and distinguish between x = y and x > y in the properties above: 6 {(a, b, c) a = b + c} 7 {(a, b, c) a > b + c} 8 {(a, b, c) b = a+c} 9 {(a, b, c) b > a+c} 10 {(a, b, c) c = a+b} 11 {(a, b, c) c > a+b} 47 / 47

81 Quality Assurance in Software Development Qualitätssicherung in der Softwareentwicklung A.o.Univ.-Prof. Dipl.-Ing. Dr. Bernhard Aichernig Institute for Software Technology Graz University of Technology Austria Summer Term / 18

82 Agenda 1 Functional testing strategies (cont.) Decision Tables 2 Structural testing strategies 2 / 18

83 Decision Tables Stub Rule 1 Rule 2 Rule 3,4 Rule 5 Rule 6 Rule 7, 8 c1 T T T F F F c2 T T F T T F c3 T F T F a1 X X X a2 X X a3 X X a4 X X 3 / 18

84 Decision Table for Triangle c1: a, b, c form a triangle F T T T T T T T T c2: a = b T T T T F F F F c3: a = c T T F F T T F F c4: b = c T F T F T F T F a1: NoTriangle X a2: Scalene X a3: Isosceles X X X a4: Equilateral X a5: Impossible X X X 4 / 18

85 Refined Decision Table for Triangle c1: a < b + c F T T T T T T T T T T c2: b < a + c F T T T T T T T T T c3: c < a + b F T T T T T T T T c4: a = b T T T T F F F F c5: a = c T T F F T T F F c6: b = c T F T F T F T F a1: NoTriangle X X X a2: Scalene X a3: Isosceles X X X a4: Equilateral X a5: Impossible X X X 5 / 18

86 Number of Rules Binary entries: 2 n rules (columns), n being the number of conditions c i don t care entries ( ) reduce the number of rules a rule counter can be established: Rules without don t care entry count as one rule Each don t care in a rule doubles the counter in this rule The rule counter for one table shall be 2 n 6 / 18

87 Decision Table with Rule Counter c1: a < b + c F T T T T T T T T T T c2: b < a + c F T T T T T T T T T c3: c < a + b F T T T T T T T T c4: a = b T T T T F F F F c5: a = c T T F F T T F F c6: b = c T F T F T F T F Rule Count: a1: NoTriangle X X X a2: Scalene X a3: Isosceles X X X a4: Equilateral X a5: Impossible X X X 7 / 18

88 Redundant Decision Table Rule 1 4 Rule 5 Rule 6 Rule 7 Rule 8 Rule 9 c1 T F F F F T c2 T T F F F c3 T F T F F a1 X X X X a2 X X X a3 X X X X X 8 / 18

89 Inconsistent Decision Table Rule 1 4 Rule 5 Rule 6 Rule 7 Rule 8 Rule 9 c1 T F F F F T c2 T T F F F c3 T F T F F a1 X X X a2 X X X X a3 X X X X Regel 1-4 und 9 sind inkonsistent. Die Entscheidungstabelle ist nichtdeterministisch. 9 / 18

90 Guidelines 1 Decision table technique is for applications with prominent if-then-else-logic logical relations between input variables cause-effect relations between input and output 2 Decision tables get quickly inconvenient if logic is complex 3 Several refinement iterations help: Start with simple conditions and then refine them! 10 / 18

91 Structural Testing Strategies A software testing technique whereby explicit knowledge of the internal workings of the item being tested are used to select the test data Based on source code. White-box testing Technology available since the mid-1970s. Most techniques use program graphs. Pro: Amenable to rigorous definitions, mathematical analysis, and precise measurement. Con: Cannot test missing parts, e.g. a missing branch. 11 / 18

92 Program Graph Definition: Given a program written in an imperative programming language, its program graph is a directed graph in which nodes are statement fragments, and edges represent flow of control. 12 / 18

93 Program Graph Example 1 public class Program2 : Program // not very elegant implementation of Triangle 2 { 3 new public static string Triangle(int a, int b, int c) 4 { 5 bool istriangle = false; 6 string msg = ""; 7 8 if (a <= c - b a <= b - c b <= a - c) 9 istriangle = false; 10 else 11 istriangle = true; if (istriangle) 14 { 15 if (a == b && b == c) 16 msg = "equilateral"; 17 else 18 if (a == b b == c a == c) 19 msg = "isosceles"; 20 else 21 msg = "scalene"; 22 } 23 else 24 msg = "no triangle"; return msg; 27 } 28 } 13 / 18

94 Program Graph Example (cont.) Node labels denote line numbers. 14 / 18

95 DD-Paths Definition: A chain is a path in a graph, in which the initial and terminal nodes are distinct, and every interior node has indegree 1 and outdegree 1. Definition: A DD-Path (decision-to-decision) is a chain in a program graph such that one of the following cases hold: 1 it consists of a single node with indeg = 0 2 it consists of a single node with outdeg = 0 3 it consists of a single node with indeg 2 or outdeg 2 4 it consists of a single node with indeg = 1 and outdeg = 1 5 it is a maximal chain of length 1 15 / 18

96 DD-Path Graph Definition: Given a program written in an imperative language, its DD-Path Graph is the directed graph in which nodes are DD-Paths of its program graph, and edges represent control flow between successor DD-Paths. Today it is known as control flow graph 16 / 18

97 Control Flow Graph / 18

98 Test Coverage Metrics Metric C 0 C 1 C 2 C d C MCC C i k C inf Description of Coverage Every statement Every DD-Path (predicate outcome) C 1 coverage + loop coverage C 1 coverage + Every dependent pair of DD-Paths Multiple condition coverage Every program path that contains up to k repetitions of a loop (usually k = 2) All possible execution paths 18 / 18

99 TU Graz Quality Assurance in Software Development Qualitätssicherung in der Softwareentwicklung A.o.Univ.-Prof. Dipl.-Ing. Dr. Bernhard Aichernig Institut für Softwaretechnologie(IST) TU Graz Summer Term /44

100 TU Graz Agenda 1 Symbolic Execution 2 Concolic Execution 2/44

101 TU Graz Literature: Symbolic Execution James C. King. Symbolic Execution and Program Testing, Communications of the ACM, Volume 19, Issue 7. July 1976, Pages /44

102 TU Graz Symbolic Execution Instead of normal program inputs one supplies symbols representing arbitrary values. Execution proceeds like normal execution, except that values may be symbolic formulas over the input values. Interesting: symbolic execution of branching statements(e.g. if-statements) A technique between testing and formal proofs. Testing:executionwithsomeconcreteinputvalues Proofs:noexecution 4/44

103 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) 2 { 3 int x,y,z; 4 x = a + b; 5 y = b + c; 6 z = x + y b; 7 return z; 8 } 5/44

104 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) a=α a,b = α b,c = α c 2 { 3 int x,y,z; 4 x = a + b; 5 y = b + c; 6 z = x + y b; 7 return z; 8 } 6/44

105 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) a=α a,b = α b,c = α c 2 { 3 int x,y,z; x =0,y =0,z =0 4 x = a + b; 5 y = b + c; 6 z = x + y b; 7 return z; 8 } 7/44

106 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) a=α a,b = α b,c = α c 2 { 3 int x,y,z; x =0,y =0,z =0 4 x = a + b; x = α a +α b,y =0,z =0 5 y = b + c; 6 z = x + y b; 7 return z; 8 } 8/44

107 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) a=α a,b = α b,c = α c 2 { 3 int x,y,z; x =0,y =0,z =0 4 x = a + b; x = α a +α b,y =0,z =0 5 y = b + c; x = α a +α b,y = α b +α c,z =0 6 z = x + y b; 7 return z; 8 } 9/44

108 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) a=α a,b = α b,c = α c 2 { 3 int x,y,z; x =0,y =0,z =0 4 x = a + b; x = α a +α b,y =0,z =0 5 y = b + c; x = α a +α b,y = α b +α c,z =0 6 z = x + y b; x = α a +α b,y = α b +α c,z = α a +α b +α c 7 return z; 8 } 10/44

109 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) a=α a,b = α b,c = α c 2 { 3 int x,y,z; x =0,y =0,z =0 4 x = a + b; x = α a +α b,y =0,z =0 5 y = b + c; x = α a +α b,y = α b +α c,z =0 6 z = x + y b; x = α a +α b,y = α b +α c,z = α a +α b +α c 7 return z; return α a +α b +α c 8 } 11/44

110 TU Graz Bsp. Min 1 int min(int x, int y) 2 { 3 int z; 4 if(x<y) 5 z=x; 6 else 7 z=y; 8 return z; 9 } 12/44

111 TU Graz Path Condition Symbolic execution of if-statement requires path condition(pc). pc is a Boolean expression over symbolic inputs. pc never contains program variables! pc is a conjunction of branching conditions. 13/44

112 TU Graz Bsp. Min 1 int min(int x, int y) x = α x,y = α y,pc =true 2 { 3 int z; 4 if(x<y) 5 z=x; 6 else 7 z=y; 8 return z; 9 } 14/44

113 TU Graz Bsp. Min 1 int min(int x, int y) x = α x,y = α y,pc =true 2 { 3 int z; x = α x,y = α y,z =0,pc =true 4 if(x<y) 5 z=x; 6 else 7 z=y; 8 return z; 9 } 15/44

114 TU Graz Bsp. Min 1 int min(int x, int y) x = α x,y = α y,pc =true 2 { 3 int z; x = α x,y = α y,z =0,pc =true 4 if(x<y) Case1:x = α x,y = α y,z =0,pc = α x < α y 5 z=x; 6 else 7 z=y; 8 return z; 9 } 16/44

115 TU Graz Bsp. Min 1 int min(int x, int y) x = α x,y = α y,pc =true 2 { 3 int z; x = α x,y = α y,z =0,pc =true 4 if(x<y) Case1:x = α x,y = α y,z =0,pc = α x < α y 5 z=x; x = α x,y = α y,z = α x,pc = α x < α y 6 else 7 z=y; 8 return z; 9 } 17/44

116 TU Graz Bsp. Min 1 int min(int x, int y) x = α x,y = α y,pc =true 2 { 3 int z; x = α x,y = α y,z =0,pc =true 4 if(x<y) Case1:x = α x,y = α y,z =0,pc = α x < α y 5 z=x; x = α x,y = α y,z = α x,pc = α x < α y 6 else Case2:x = α x,y = α y,z =0,pc = α x < α y 7 z=y; 8 return z; 9 } 18/44

117 TU Graz Bsp. Min 1 int min(int x, int y) x = α x,y = α y,pc =true 2 { 3 int z; x = α x,y = α y,z =0,pc =true 4 if(x<y) Case1:x = α x,y = α y,z =0,pc = α x < α y 5 z=x; x = α x,y = α y,z = α x,pc = α x < α y 6 else Case2:x = α x,y = α y,z =0,pc = α x < α y 7 z=y; x = α x,y = α y,z = α y,pc = α x < α y 8 return z; 9 } 19/44

118 TU Graz Bsp. Min 1 int min(int x, int y) x = α x,y = α y,pc =true 2 { 3 int z; x = α x,y = α y,z =0,pc =true 4 if(x<y) Case1:x = α x,y = α y,z =0,pc = α x < α y 5 z=x; x = α x,y = α y,z = α x,pc = α x < α y 6 else Case2:x = α x,y = α y,z =0,pc = α x < α y 7 z=y; x = α x,y = α y,z = α y,pc = α x < α y 8 return z; return(z = α x α x < α y ) (z = α y α x < α y ) 9 } 20/44

119 TU Graz Bsp. Min(a,b,c) 1 int min(int a, int b, int c) 2 { 3 return 4 min( 5 min(a,b), 6 c 7 ); 8 } 21/44

120 TU Graz Bsp. Min(a,b,c) 1 int min(int a, int b, int c) 2 { a=α a,b = α b,c = α c,pc =true 3 return 4 min( 5 min(a,b), 6 c 7 ); 8 } 22/44

121 TU Graz Bsp. Min(a,b,c) 1 int min(int a, int b, int c) 2 { a=α a,b = α b,c = α c,pc =true 3 return 4 min( 5 min(a,b), (= α a pc = α a < α b ) (= α b pc = α a α b ) 6 c 7 ); 8 } 23/44

122 TU Graz Bsp. Min(a,b,c) 1 int min(int a, int b, int c) 2 { a=α a,b = α b,c = α c,pc =true 3 return 4 min( 5 min(a,b), (= α a pc = α a < α b ) (= α b pc = α a α b ) 6 c c = α c 7 ); 8 } 24/44

123 TU Graz Bsp. Min(a,b,c) 1 int min(int a, int b, int c) 2 { a=α a,b = α b,c = α c,pc =true 3 return 4 min( 5 min(a,b), (= α a α a < α b ) (= α b α a α b ) 6 c c = α c 7 ); (= α a pc = α a < α b α a < α c ) 8 } 25/44

124 TU Graz Bsp. Min(a,b,c) 1 int min(int a, int b, int c) 2 { a=α a,b = α b,c = α c,pc =true 3 return 4 min( 5 min(a,b), (= α a α a < α b ) (= α b α a α b ) 6 c c = α c 7 ); (= α a pc = α a < α b α a < α c ) 8 } (= α b pc = α a α b α b < α c ) 26/44

125 TU Graz Bsp. Min(a,b,c) 1 int min(int a, int b, int c) 2 { a=α a,b = α b,c = α c,pc =true 3 return 4 min( 5 min(a,b), (= α a α a < α b ) (= α b α a α b ) 6 c c = α c 7 ); (= α a pc = α a < α b α a < α c ) 8 } (= α b pc = α a α b α b < α c ) 9 (= α c pc = α a < α b α a α c ) 27/44

126 TU Graz Bsp. Min(a,b,c) 1 int min(int a, int b, int c) 2 { a=α a,b = α b,c = α c,pc =true 3 return 4 min( 5 min(a,b), (= α a α a < α b ) (= α b α a α b ) 6 c c = α c 7 ); (= α a pc = α a < α b α a < α c ) 8 } (= α b pc = α a α b α b < α c ) 9 (= α c pc = α a < α b α a α c ) 10 (= α c pc = α a α b α b α c ) 28/44

127 TU Graz Symbolic Execution Tree Symbolic execution forcs at each if-statement: α a < α b T F α a < α c α b < α c T F T F min = α a min = α c min = α b min = α c 29/44

128 TU Graz From Path Conditions to Concrete Test Cases Calculation:pc 2 =true α a < α b (α a < α c ) Path condition represents equivalence class for all concrete values taking a path. Concrete test cases: find concrete values satisfying pc e.g.forpc2:a =0,b =1,c =0 Result: path coverage 30/44

129 TU Graz Limitations infinite number of paths(loops) infeasible paths(pc = false) limitations of solvers and theorem provers all code must be accessible(white-box) only for single-threaded programs 31/44

130 TU Graz Concolic Execution Concolic = Concrete + Symbolic also called Dynamic Symbolic Execution(Microsoft) Concrete and Symbolic Execution in parallel Due to concrete execution: Noinfeasablepaths! Integrationofblack-boxcomponentspossible. 32/44

131 TU Graz Concolic Execution Loop Algorithm 1 Covered := {} coveredsetofinputs 2 selectnewinputi Covered stopifnoonefound 3 execute Program(i) and record path condition C C(i) holds 4 Covered :=Covered {i C(i)} 5 goto2 Newinputi Coveredafterniterations: solve C i1 C in Stopwhen C i1 C in =false 33/44

132 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) 2 { 3 int x,y,z; 4 x = a + b; 5 y = b + c; 6 z = x + y b; 7 return z; 8 } 34/44

133 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) 2 { a=(α a,1),b = (α b,2),c = (α c,3) 3 int x,y,z; 4 x = a + b; 5 y = b + c; 6 z = x + y b; 7 return z; 8 } 35/44

134 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) 2 { a=(α a,1),b = (α b,2),c = (α c,3) 3 int x,y,z; x =0,y =0,z =0 4 x = a + b; 5 y = b + c; 6 z = x + y b; 7 return z; 8 } 36/44

135 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) 2 { a=(α a,1),b = (α b,2),c = (α c,3) 3 int x,y,z; x =0,y =0,z =0 4 x = a + b; x = (α a +α b,3),y = (0,0),z = (0,0) 5 y = b + c; 6 z = x + y b; 7 return z; 8 } 37/44

136 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) 2 { a=(α a,1),b = (α b,2),c = (α c,3) 3 int x,y,z; x =0,y =0,z =0 4 x = a + b; x = (α a +α b,3),y = (0,0),z = (0,0) 5 y = b + c; x = (α a +α b,3),y = (α b +α c,5),z = (0,0) 6 z = x + y b; 7 return z; 8 } 38/44

137 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) 2 { a=(α a,1),b = (α b,2),c = (α c,3) 3 int x,y,z; x =0,y =0,z =0 4 x = a + b; x = (α a +α b,3),y = (0,0),z = (0,0) 5 y = b + c; x = (α a +α b,3),y = (α b +α c,5),z = (0,0) 6 z = x + y b; x = (α a +α b,3),y = (α b +α c,5), 7 z = (α a +α b +α c,6) 8 return z; 9 } 39/44

138 TU Graz Bsp. Sum 1 int sum(int a, int b, int c) 2 { a=(α a,1),b = (α b,2),c = (α c,3) 3 int x,y,z; x =0,y =0,z =0 4 x = a + b; x = (α a +α b,3),y = (0,0),z = (0,0) 5 y = b + c; x = (α a +α b,3),y = (α b +α c,5),z = (0,0) 6 z = x + y b; x = (α a +α b,3),y = (α b +α c,5), 7 z = (α a +α b +α c,6) 8 return z; return(α a +α b +α c,6) 9 } 40/44

139 TU Graz Bsp. Min: Iteration 1 1 int min(int x,int y) x = (α x,0),y = (α y,0),pc 1 =true 2 { 3 int z; 4 if(x<y) 5 z=x; 6 else 7 z=y; 8 return z; 9 } 41/44

140 TU Graz Bsp. Min: Iteration 1 1 int min(int x,int y) x = (α x,0),y = (α y,0),pc 1 =true 2 { 3 int z; x = (α x,0),y = (α y,0),z = (0,0),pc 1 =true 4 if(x<y) 5 z=x; 6 else 7 z=y; 8 return z; 9 } 41/44

141 TU Graz Bsp. Min: Iteration 1 1 int min(int x,int y) x = (α x,0),y = (α y,0),pc 1 =true 2 { 3 int z; x = (α x,0),y = (α y,0),z = (0,0),pc 1 =true 4 if(x<y) (0 <0) 5 z=x; 6 else 7 z=y; 8 return z; 9 } 41/44

142 TU Graz Bsp. Min: Iteration 1 1 int min(int x,int y) x = (α x,0),y = (α y,0),pc 1 =true 2 { 3 int z; x = (α x,0),y = (α y,0),z = (0,0),pc 1 =true 4 if(x<y) (0 <0) 5 z=x; 6 else x = (α x,0),y = (α y,0),z = (0,0),pc 1 = (α x < α y ) 7 z=y; 8 return z; 9 } 41/44

143 TU Graz Bsp. Min: Iteration 1 1 int min(int x,int y) x = (α x,0),y = (α y,0),pc 1 =true 2 { 3 int z; x = (α x,0),y = (α y,0),z = (0,0),pc 1 =true 4 if(x<y) (0 <0) 5 z=x; 6 else x = (α x,0),y = (α y,0),z = (0,0),pc 1 = (α x < α y ) 7 z=y; x = (α x,0),y = (α y,0),z = (α y,0),pc 1 = (α x < α y ) 8 return z; 9 } 41/44

144 TU Graz Bsp. Min: Iteration 1 1 int min(int x,int y) x = (α x,0),y = (α y,0),pc 1 =true 2 { 3 int z; x = (α x,0),y = (α y,0),z = (0,0),pc 1 =true 4 if(x<y) (0 <0) 5 z=x; 6 else x = (α x,0),y = (α y,0),z = (0,0),pc 1 = (α x < α y ) 7 z=y; x = (α x,0),y = (α y,0),z = (α y,0),pc 1 = (α x < α y ) 8 return z; returnz= (α y,0),pc 1 = (α x < α y ) 9 } 41/44

145 TU Graz Bsp. Min: Iteration 1 1 int min(int x,int y) x = (α x,0),y = (α y,0),pc 1 =true 2 { 3 int z; x = (α x,0),y = (α y,0),z = (0,0),pc 1 =true 4 if(x<y) (0 <0) 5 z=x; 6 else x = (α x,0),y = (α y,0),z = (0,0),pc 1 = (α x < α y ) 7 z=y; x = (α x,0),y = (α y,0),z = (α y,0),pc 1 = (α x < α y ) 8 return z; returnz= (α y,0),pc 1 = (α x < α y ) 9 } FindnewinputvaluessuchthatC = pc 1 = α x < α y holdsand execute again, e.g. min(0, 1) 41/44

146 TU Graz Bsp. Min: Iteration 2 1 int min(int x,int y) x = (α x,0),y = (α y,1),pc 2 =true 2 { 3 int z; 4 if(x<y) 5 z=x; 6 else 7 z=y; 8 return z; 9 } 42/44

147 TU Graz Bsp. Min: Iteration 2 1 int min(int x,int y) x = (α x,0),y = (α y,1),pc 2 =true 2 { 3 int z; x = (α x,0),y = (α y,1),z = (0,0),pc 2 =true 4 if(x<y) 5 z=x; 6 else 7 z=y; 8 return z; 9 } 42/44

148 TU Graz Bsp. Min: Iteration 2 1 int min(int x,int y) x = (α x,0),y = (α y,1),pc 2 =true 2 { 3 int z; x = (α x,0),y = (α y,1),z = (0,0),pc 2 =true 4 if(x<y) (0 <1): 5 z=x; 6 else 7 z=y; 8 return z; 9 } 42/44

149 TU Graz Bsp. Min: Iteration 2 1 int min(int x,int y) x = (α x,0),y = (α y,1),pc 2 =true 2 { 3 int z; x = (α x,0),y = (α y,1),z = (0,0),pc 2 =true 4 if(x<y) (0 <1): pc 2 = α x < α y 5 z=x; 6 else 7 z=y; 8 return z; 9 } 42/44

150 TU Graz Bsp. Min: Iteration 2 1 int min(int x,int y) x = (α x,0),y = (α y,1),pc 2 =true 2 { 3 int z; x = (α x,0),y = (α y,1),z = (0,0),pc 2 =true 4 if(x<y) (0 <1): pc 2 = α x < α y 5 z=x; x = (α x,0),y = (α y,1),z = (α x,0),pc 2 = α x < α y 6 else 7 z=y; 8 return z; 9 } 42/44

151 TU Graz Bsp. Min: Iteration 2 1 int min(int x,int y) x = (α x,0),y = (α y,1),pc 2 =true 2 { 3 int z; x = (α x,0),y = (α y,1),z = (0,0),pc 2 =true 4 if(x<y) (0 <1): pc 2 = α x < α y 5 z=x; x = (α x,0),y = (α y,1),z = (α x,0),pc 2 = α x < α y 6 else 7 z=y; 8 return z; returnz= (α x,0),pc 2 = α x < α y 9 } 42/44

152 TU Graz Bsp. Min: Iteration 2 1 int min(int x,int y) x = (α x,0),y = (α y,1),pc 2 =true 2 { 3 int z; x = (α x,0),y = (α y,1),z = (0,0),pc 2 =true 4 if(x<y) (0 <1): pc 2 = α x < α y 5 z=x; x = (α x,0),y = (α y,1),z = (α x,0),pc 2 = α x < α y 6 else 7 z=y; 8 return z; returnz= (α x,0),pc 2 = α x < α y 9 } FindingnewinputvaluessuchthatC = pc 1 pc 2 =falseisnot possible, hence terminate. 42/44

153 TU Graz Limitations infinite number of paths: upper bounds limitations of solvers and theorem provers for single-threaded programs 43/44

154 TU Graz Tools Cute, jcute Pex(see exercise) PathCrawler 44/44

155 Quality Assurance in Software Development Qualitätssicherung in der Softwareentwicklung A.o.Univ.-Prof. Dipl.-Ing. Dr. Bernhard Aichernig Institute for Software Technology Graz University of Technology Austria Summer Term / 23

156 Agenda 1 Spezifikationen Beispiel Eigenschaften von Spezifikationen Contracts 2 Invarianten 2 / 23

157 Oracle Definition: any means used to predict the outcome of a test. Kind of oracles: specification existing test cases (regression testing) previous version of the program prototype domain expert actual program (dangerous!) 3 / 23

158 Specifications fix the criteria to accept an implementation. are a basis for a contract between user and supplier. define what should be achieved, but not how. greatest possible freedom for developer. programs are not suitable as specifications! 4 / 23

159 Specification vs. Implementation Specification vs. Implementations abstract concrete what how 5 / 23

160 Specification Implementation 1 The specification serves as starting point for the design of the implementation. 2 The specification is the bases for any verification: 1 for proving that an implementation satisfies a specification 2 for the generation of test cases 3 as a test oracle 6 / 23

161 Programs as Specifications? public static double P (double tfactorial) { double x1=0; double x0=tfactorial/2; double a=tfactorial; boolean finished=false; while (finished==false) { x1=(x0+(a/x0))/2; if (x1>x0){ if ((x1-x0)< ){ finished=true;} } else if (x0>x1){ if ((x0-x1)< ){ finished=true;} } x0=x1; } return x1; } 7 / 23

162 Spezifikation Vers. 1 Example (Postcondition) Postcondition (Nachbedingung): post P (in, out) = df out 2 = in 8 / 23

163 Spezifikation Vers. 2 Not all inputs are valid! Example (Pre-Postcondition) Postcondition: post P (in, out) = df out 2 = in Precondition (Vorbedingung): pre P (in) = df in 0 9 / 23

164 Spezifikation Vers. 3 We are only interested in positive results! Example (Pre-Postcondition) Postcondition: post P (in, out) = df out 2 = in out 0 Precondition: pre P (in) = df in 0 10 / 23

165 Spezifikation Vers. 4 In practice, the result is an approximations! Example (Pre-Postcondition) Postcondition: post P (in, out) = df abs(in out 2 ) ǫ in out 0 Precondition: ǫ... error constant abs... absolute value pre P (in) = df in 0 11 / 23

166 Satisfiability In order to be a meaningful specification, one solution must exist. Thus, one program/implementation must exist that satisfies the specification. We also say that there is a model of the specification. Or, the specification is realisable (implementable). 12 / 23

167 Underspecification Specifications need not to determine unique results. Examples are flight reservation systems, compilers. Underspecification is a form of abstraction. 13 / 23

168 Preciseness Specifications must be precise. That means that the set of solutions (models) must be defined. Underspecified does not mean imprecise. Distinguish between abstract and imprecise! 14 / 23

169 Design by Contract A method for software development Pre- and postconditions als contracts between classes and clients Pre- and postconditions are executable. History: Hoare-Logik (1969), VDM (70-ies), Eiffel (1992), JML (1999), C# Code Contracts (2008) 15 / 23

170 C# Code Contracts Example (sqrt) public static double sqrt2(double x) { Contract.Requires(x >= 0); Contract.Ensures( Math.Abs(x - (Contract.Result<double>() * Contract.Result<double>())) < ); return Math.Sqrt(x); } Vorbedingung: Requires (normale) Nachbedingung: Ensures 16 / 23

171 Java Modeling Language (JML) Example (sqrt) public final static double eps = ; requires x >= 0.0; ensures JMLDouble.approximatelyEqualTo(x, \result * \result, eps); public static double sqrt(double x) { return Math.sqrt(x); } Vorbedingung: requires (normale) Nachbedingung: ensures 17 / 23

172 Vorteile von Contracts zur Dokumentation (Spezifikation) von Klassen und Interfaces Kontrakt abstrakter als Code (kein Algorithmus!) Kontrakt kann maschinell geprüft werden (im Gegensatz zu informalen Kommentaren) Fehlersuche (debugging) Wartung Schuldzuweisung bei Fehlern: Client vs. Class Effizienz: Keine ineffizienten defensiven Überprüfungen (checks) im Code 18 / 23

173 Beispiel Effizienz Example (Binärsuche) public static int search(int[] a, int x) { Contract.Requires(a!= null); Contract.Requires(Contract.ForAll(1, a.length, i => a[i-1] <= a[i])); return Array.BinarySearch(a, x); } All-quantor ( ): ForAll Endliche Bereichsangabe f"ur i macht den Ausdruck ausf"uhrbar. Eine alternative Prüfschleife im Endprogramm wäre ineffizient: Laufzeit: logarithmisch linear 19 / 23

174 Modularität des Programmverstehens Wie kann man solchen Code verstehen? Example (Typischer OO Code)... source.close(); target.close(); getfile.setlastmodified( loc.modtime().gettime()); return modtime(); Möglichkeit 1: Lesen des Codes aller Methoden, lesen des Codes aller Methoden in den Methoden, lesen des... Erschwernis: Subtypen polymorphismus (dynamische Bindung) Möglichkeit 2: Lesen der Kontrakte aller Methoden 20 / 23

175 Invarianten Definition (Klasseninvariante, allgemein) Eine Klasseninvariante ist eine Eigenschaft des Klassenzustandes (Variablen, Felder), die immer nach Ausführen eines jeden Konstruktors, sowie vor und nach Ausführen einer jeden Methode gelten mu s. auch Dateninvariante oder kurz Invariante genannt. 21 / 23

176 Rolle von Invarianten Typeninvariante: schränkt einen Datentyp ein (z.b. Natürliche Zahlen) Repräsentationsinvariante: dokumentiert Designentscheidung der Datenrepräsentation (z.b. Sortierung einer Liste) Dokumentieren Beziehungen zwischen Variablen (Feldern). Invarianten werden durch Kapselung geschützt. (Repräsentations-)invarianten sind oft für die Korrektheit eines Algorithmus notwendig. 22 / 23

177 C# Beispiel Purse: Translated from JML [Burdy et al. 2005] 23 / 23

178 Quality Assurance in Software Development Qualitätssicherung in der Softwareentwicklung A.o.Univ.-Prof. Dipl.-Ing. Dr. Bernhard Aichernig Institute for Software Technology Graz University of Technology Austria Summer Term / 10

179 Agenda 1 Model-based Specifications Example: Alarm System 2 / 10

180 Model-based Specifications Motivation: Observation: pre- and postconditions are closely coupled to the inner data representation of a class. Advantage: Pecise description of the inner behaviour of the implementation. Precise documentation of the representation invariant. Disadvantage: When changing a variable of a class, also the specifications must be changed. The variables of a class must be declared before the specification. Not suitable for specifying abstract classes and interfaces! Conclusion We need abstractions of data representations models 3 / 10

181 Model-based Specification in C# Some restrictions apply: Unlike JML, the Code Contracts framework does not know model variables. Model variables are abstractions of one or more concrete variables Invariants cannot be checked on interfaces In C# we can use generic abstract classes for modelling the abstract state with (collections of) interfaces Using sets (ISet), sequences (IList), maps (IDictionary) to abstract from efficient implementation representations. 4 / 10

182 Anforderungen für ein Alarmsystem Eine chemische Fabrik ist mit einer Anzahl von Überwachungseinrichtungen ausgerüstet, welche abhängig vom Zustand der Fabrik Alarme auslösen können. Wird ein Alarm ausgelöst, mu s ein Experte angefordert werden. Experten haben verschiedene Qualifikationen um mit verschiedenen Alarmen umgehen zu können. 5 / 10

183 Anforderungen für ein Alarmsystem (cont.) R1 Es ist ein computerbasiertes System zu entwickeln, welches die Alarme in dieser Fabrik managed. R2 Vier Arten von Qualifikationen werden gebraucht um mit den Alarmen umzugehen. Diese sind elektrische, mechanische, biologische und chemische Qualifikationen. R3 Es müssen während allen festgelegten Zeitperioden Experten abrufbereit sein. R4 Jeder Experte hat eine Liste von Qualifikationen. 6 / 10

184 Anforderungen für ein Alarmsystem (cont.) R5 Jeder Alarm, der an das System gemeldet wird hat eine zugeordnete Qualifikation und eine Beschreibung des Alarms, die ein Experte verstehen kann. R6 Wann immer ein Alarm vom System empfangen wird, soll ein Experte mit der richtigen Qualifikation gefunden werden, so da s er angepaged werden kann. R7 Die Experten sollen die Systemdatenbank abfragen können, um zu überprüfen wann sie abrufbereit sein werden/müssen. R8 Es mu s möglich sein die Anzahl der abrufbaren Experten abzufragen. 7 / 10

185 Mögliche Klassen und Methoden Klassen (Typen) IAlarmSystem IQualification IAlarm IPeriod IExpert (Description) Methoden ExpertToPage ExpertIsOnDuty NumberOfExperts 8 / 10

186 Deklaration der Methoden R6: public int NumberOfExperts(IPeriod p); R7: public IPeriod[] ExpertIsOnDuty(IExpert e); R8: public IExpert ExpertToPage(IAlarm a, IPeriod p); 9 / 10

187 C# Example AlarmSystem.cs 10 / 10

188 Quality Assurance in Software Development Qualitätssicherung in der Softwareentwicklung A.o.Univ.-Prof. Dipl.-Ing. Dr. Bernhard Aichernig Institute for Software Technology Graz University of Technology Austria Summer Term / 27

189 Agenda 1 Äquivalenzklassen aus Disjunktiver Normalform 2 Bildung von Testfallfolgen 3 Testabdeckungen Testabdeckung von Finiten Zustandsmaschinen 2 / 27

190 DNF-Partitionierung Gegeben eine formale Spezifikation, z.b. pre(in) post(in, out) Rekursives Umschreiben durch Anwenden der Regel: a b (a b) ( a b) (a b) das ergibt eine Disjunktive Normalform (DNF). Logische Äquivalenzen ergeben neue Disjunktionen: a b a b Spezialwerte v eines Typs T können auch behandelt werden : x : T (x = v x v) x : T 3 / 27

191 DNF-Partitionierung (Forts.) Normale Äquivalenzklassen durch umschreiben von pre(in) post(in, out) Robustheits-Äquivalenzklassen durch umschreiben von pre(in) post(in, out) Achtung! Es wird die In-Output-Relation partitioniert, nicht nur der Eingabebereich. 4 / 27

192 Maximum Beispiel Ein Programm soll das Maximum von zwei ganzen Zahlen ausgeben. Wir können in Pseudonotation spezifizieren: max: Int Int Int max(x,y) as r post (r = x r = y) r x r y 5 / 27

193 Maximum Beispiel (Forts.) Umschreiben der Disjunktion ergibt die Postcondition: post ((r = x r = y) (r x r = y) (r = x r y)) r x r y 6 / 27

194 Maximum Beispiel (Forts.) Distributivgesetz auf zweimal ergibt post (r = x r = y r x r y) (r x r = y r x r y ) (r = x r y r x r y ) 7 / 27

195 Maximum Beispiel (Forts.) Nach Vereinfachung erhalten wir: post (r = x r = y) (r > x r = y) (r = x r > y) Das repräsentiert drei Äquivalenzklassen von Testfallen: 1 {(x, y, r) r = x r = y} 2 {(x, y, r) r > x r = y} 3 {(x, y, r) r = x r > y} 8 / 27

196 Testfallfolgen Für komplexe zustandsbasierte Systeme (OO), Eingabe-Ausgabestrategien sind nicht ausreichend: interner Zustand kann nicht direkt als Eingabe gesetzt werden (Kapselung), sondern nur durch Interaktionen. der Tester muss eine adäquate Folge von Aktionen finden, die den internen Zustand setzen, von dem eine Methode (Funktion) getestet werden soll. jede Funktion mag einfach sein, aber ihre Interaktionen sind es nicht. Typische Anwendungen: Unit-Testen von Objekt-Orientierten Systemen. Systemtesten von interaktiven Systemen. 9 / 27

197 Formale Technik 1 Wähle ein Modell für den internen Zustand. 2 Spezifikation jeder Methode durch Vor-Nachbedingungen 3 DNF Partitionierung jeder Methode 4 Konstruktion eines endlichen Zustandsautomaten (finite state machine): Extrahiere pro Methode zwei Constraints: Zustand davor und danach. Partitionierung des Zustands Transitionen sind partitionierte Sub-Methoden 5 Wähle Folgen, die den Zustandsautomaten abdecken 10 / 27

198 Scheduler Beispiel Ein Prozessscheduler behandelt Prozesse, die in drei möglichen Zuständen sind: ready um am Prozessor zu laufen, waiting um ready zu werden, optional, einen einzigen Prozess der active ist. Diese Prozesse werden durch eine eindeutige Pid gekennzeichnet. Ein Prozess kann nicht gleichzeitig ready und waiting sein. Der aktive Prozess ist weder ready noch waiting Es muss einen active Prozess geben, wann immer es einen ready Prozess gibt, der laufen kann. 11 / 27

199 Zustandsmodell scheme scheduler = class type Pid = Nat value nil: Nat variable active : Pid, ready : Pid-set, waiting: Pid-set 12 / 27

200 Zustandsinvariante axiom ready waiting = {}, active (ready waiting), nil (ready waiting), active = nil ready = {} 13 / 27

201 Schedule value schedule: Pid-set Pid schedule(ps) as p post p ps pre ps {}, 14 / 27

202 Init Init: Unit write any Unit Init() post active = nil ready waiting = {}, 15 / 27

203 New New: Pid write waiting Unit New(p) post waiting = waiting old {p} pre p active p (ready waiting), 16 / 27

204 Ready Ready: Pid write active, waiting, ready Unit Ready(q) post waiting = waiting old \ {q} active old = nil (ready = ready old active = q) active old nil (ready = ready old {q} active = active old ) pre q waiting, 17 / 27

205 Swap Swap: Unit write any Unit Swap() post waiting = waiting old {active old } ready old = {} (active = nil ready = {}) ready old {} (active = schedule(ready old ) ready = ready old \ {active}) pre active nil, 18 / 27

206 Konstruktion des Automaten 1 Analysiere die Äquivalenzklassen an allen Methoden (Funktionen) und leite Sub-methoden anhand dieser Klassen ab. Hier unterscheiden wir ob active = nil und ob ready oder waiting leer sind. 2 Von jeder partitionierten Methode erhält man die Vorher- Nachherzustände durch existientielles Quantifizieren der Variablen, die nicht in Betracht gezogen werden. 3 DNF Partitionierung der Zustände aus Schritt 2. Das ergibt die Zustände des Automaten. 4 Konstruiere den Automaten durch Auflösen der Transitionen gegen die Zustände. 19 / 27

207 Partitioned New New1: Pid write waiting Unit New1(p) post waiting = waiting old {p} pre active = nil ready = {} p waiting, New2: Pid write waiting Unit New2(p) post waiting = waiting old {p} pre p active active nil p (ready waiting), 20 / 27

208 Partitioned Ready Ready1: Pid write active, waiting, ready Unit Ready1(q) post waiting = waiting old \ {q} ready = ready old active = q pre q waiting active = nil, Ready2: Pid write active, waiting, ready Unit Ready2(q) post waiting = waiting old \ {q} ready = ready old {q} active = active old pre q waiting active nil, 21 / 27

209 Partitioned Swap Swap1: Unit write any Unit Swap1() post waiting = waiting old {active old } active = nil ready = {} pre active nil ready = {}, Swap2: Unit write any Unit Swap2() post waiting = waiting old {active old } active ready old ready = ready old \ {active} pre active nil ready {}, 22 / 27

210 Berechnung der Zustände DNF Analyse der Invariante Existentielles Quantifizieren (hiding) ergibt den 1 Vorherzustand: variable 1,...,variable n pre post 2 Nachherzustand: variable1 old,...,variablen old pre post Danach DNF-Analyse aller Zustände ergibt Äquivalenzklassen der Zustände. 23 / 27

211 Zustände des Automaten Variables State 1 State 2 State 3 State 4 State 5 State 6 active = nil = nil nil nil nil nil ready = {} = {} = {} = {} = {} = {} waiting = {} = {} = {} = {} = {} = {} 24 / 27

212 Konstruktion der Transitionen Aus den Zuständen und partitionierten Methoden, kann man die Transitionen konstruieren. Eine Transition existiert von Zustand St1 nach St2 genau dann, wenn die folgende Bedingung wahr ist: s, s old, in, out : St1(s old ) pre i (s old, in) post i (s old, in, s, out) St2(s) wobei die Zustandsvariablen s und s old den gesamten Variablenvektor repr asentieren. s entspricht \old(s) in JML 25 / 27

213 Der endliche Automat für den Scheduler New1 Swap1 New2 Swap2 Swap2 Init Swap2 1 2 Ready1 3 4 Ready2 5 6 New1 New2 New2 Swap1 Swap2 Ready2 Ready1 Ready2 New2 26 / 27

214 Testabdeckung des Zustandsautomaten Einige mögliche Abdeckungen: 1 Alle Zustände abdecken. 2 Alle Methoden abdecken. 3 Alle Transitionen abdecken. 4 Alle aufeinanderfolgenden Paare von Transitionen. 5 Alle mögliche Pfade mit n Wiederholungen. 27 / 27

215 Quality Assurance in Software Development Qualitätssicherung in der Softwareentwicklung A.o.Univ.-Prof. Dipl.-Ing. Dr. Bernhard Aichernig Institute for Software Technology Graz University of Technology Austria Summer Term / 31

216 Agenda 1 Conformance Testing Properties 2 Labelled Transition Systems Equivalence Preorder ioco Examples 3 Test generation 2 / 31

217 Input/Output Conformance Relation Jan Tretmans Based on IO-Labeled Transition Systems 3 / 31

218 Input/Output Conformance Relation Jan Tretmans Based on IO-Labeled Transition Systems 3 / 31

219 Conformance Testing 4 / 31

220 Conformance Testing - Soundness Test suite is sound: conformance all tests pass 5 / 31

221 Conformance Testing - Soundness Test suite is sound: conformance all tests pass 5 / 31

222 Conformance Testing - Exhaustiveness Test suite is exhaustive: conformance all tests pass 6 / 31

223 Conformance Testing - Exhaustiveness Test suite is exhaustive: conformance all tests pass 6 / 31

224 Conformance Testing - Completeness Test suite is complete: conformance all tests pass 7 / 31

225 Conformance Testing - Completeness Test suite is complete: conformance all tests pass 7 / 31

226 Conformance Testing with ioco System (Implementation) is modeled as IOTS weakly input enabled Specification is an IOLTS possibly incomplete possible non-deterministic τ!g1 τ!g1?g2?g2!g3!g3!g1 8 / 31

227 Conformance Testing with ioco System (Implementation) is modeled as IOTS weakly input enabled Specification is an IOLTS possibly incomplete possible non-deterministic τ!g1 τ!g1?g2?g2!g3!g3!g1 8 / 31

228 Input Output Labeled Transition Systems Input Output Labeled Transition System An IOLTS is an LTS M = (Q M, A M, M, q0 M) with Q M a finite set of states A M = A M I A M O {τ} where A M I and A M O are input and output alphabets τ A M I A M O is an unobservable, internal action M Q M A M Q M is the transition relation q0 M QM is the initial state. QUESTION? What means: The implementation conforms-to the specification? 9 / 31

229 Input Output Labeled Transition Systems Input Output Labeled Transition System An IOLTS is an LTS M = (Q M, A M, M, q0 M) with Q M a finite set of states A M = A M I A M O {τ} where A M I and A M O are input and output alphabets τ A M I A M O is an unobservable, internal action M Q M A M Q M is the transition relation q0 M QM is the initial state. QUESTION? What means: The implementation conforms-to the specification? 9 / 31

230 How to relate 2 LTSs? Equivalence Relations (=) Bisimulation Trace Equivalence Testing Equivalence... Preorder Relations ( ) Trace Preorder Testing Preorder... Input-Output Relations... ioconf ioco / 31

231 (Weak) Bisimulation Two states are bisimilar iff they simulate each other and go to states which are bisimilar Bisimulation is not suited for testing! 11 / 31

232 Trace Equivalence A trace is an observable sequence of actions Two states are trace equivalent iff they have the same traces Trace equivalence is the weakest notion of conformance 12 / 31

233 Equivalence vs. Preorder Relations Equivalence Relation (R) reflexive (srs) symmetric: irs sri transitive: irs srt irt Preorder Relations ( ) NOT necessarily antisymmetric: irs i s s i simplifies testing e.g.: Trace Preorder i tr s traces(i) traces(s) 13 / 31

234 Some Notations: Transitions q a M q = df (q, a, q ) M q ǫ q = df (q = q ) (q τ M q 1 q n 1 τ M q ) q a q = df q 1, q 2 : q ǫ M q 1 a M q 2 ǫ M q 14 / 31

235 Some Notations: Quiescence δ is used to represent quiescence q δ q = df q is a quiescent state. Quiescent state = no edge labeled with an output or an internal action 15 / 31

236 Some Notations: Quiescence δ is used to represent quiescence q δ q = df q is a quiescent state. Quiescent state = no edge labeled with an output or an internal action 15 / 31

237 Some Notations: Suspension Automaton (M) = (Q M, A (M), (M), q M 0 ) where: A (M) = A M {δ} with δ A (M) O (M) is obtained from M by adding loops q δ q for each quiescent state 16 / 31

238 Some Notations: After q after M σ = df {q q σ M q } Q after M σ = df q Q (q after M σ). 17 / 31

239 Some Notations: Out Out M (q) = df {a A M O q a M } Out M (Q) = df q Q (Out M(q)) 18 / 31

240 ioco Definition: ioco Let IUT = (Q IUT, A IUT, IUT, q0 IUT ) be weakly input enabled with A IUT = A IUT I A IUT O {τ} and S = ( Q S, A S, S, q0) S be strongly responsive with A S = A S I A S O {τ}. Then: IUT ioco S = df σ traces( (S)) : Out IUT ( (IUT) after IUT σ) Out S ( (S) after S σ). IUT ioco S iff outputs (and quiescences) of the IUT are possible in S after an arbitrary suspension trace of S. 19 / 31

241 ioco Definition: ioco Let IUT = (Q IUT, A IUT, IUT, q0 IUT ) be weakly input enabled with A IUT = A IUT I A IUT O {τ} and S = ( Q S, A S, S, q0) S be strongly responsive with A S = A S I A S O {τ}. Then: IUT ioco S = df σ traces( (S)) : Out IUT ( (IUT) after IUT σ) Out S ( (S) after S σ). IUT ioco S iff outputs (and quiescences) of the IUT are possible in S after an arbitrary suspension trace of S. 19 / 31

242 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 20 / 31

243 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 20 / 31

244 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 21 / 31

245 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 21 / 31

246 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 22 / 31

247 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 22 / 31

248 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 23 / 31

249 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 23 / 31

250 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 24 / 31

251 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 24 / 31

252 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 25 / 31

253 P ioco S? P ioco S = df σ traces( (S)) : Out IUT ( (P) after IUT σ) Out S ( (S) after S σ). 25 / 31

Test Case Design Techniques

Test Case Design Techniques Summary of Test Case Design Techniques Brian Nielsen, Arne Skou {bnielsen ask}@cs.auc.dk Development of Test Cases Complete testing is impossible Testing cannot guarantee the absence of faults How to select

More information

Test case design techniques II: Blackbox testing CISS

Test case design techniques II: Blackbox testing CISS Test case design techniques II: Blackbox testing Overview Black-box testing (or functional testing): Equivalence partitioning Boundary value analysis Domain analysis Cause-effect graphing Behavioural testing

More information

Software testing. Objectives

Software testing. Objectives Software testing cmsc435-1 Objectives To discuss the distinctions between validation testing and defect testing To describe the principles of system and component testing To describe strategies for generating

More information

Test case design techniques I: Whitebox testing CISS

Test case design techniques I: Whitebox testing CISS Test case design techniques I: Whitebox testing Overview What is a test case Sources for test case derivation Test case execution White box testing Flowgraphs Test criteria/coverage Statement / branch

More information

Specification and Analysis of Contracts Lecture 1 Introduction

Specification and Analysis of Contracts Lecture 1 Introduction Specification and Analysis of Contracts Lecture 1 Introduction Gerardo Schneider gerardo@ifi.uio.no http://folk.uio.no/gerardo/ Department of Informatics, University of Oslo SEFM School, Oct. 27 - Nov.

More information

Software Testing & Analysis (F22ST3): Static Analysis Techniques 2. Andrew Ireland

Software Testing & Analysis (F22ST3): Static Analysis Techniques 2. Andrew Ireland Software Testing & Analysis (F22ST3) Static Analysis Techniques Andrew Ireland School of Mathematical and Computer Science Heriot-Watt University Edinburgh Software Testing & Analysis (F22ST3): Static

More information

11-QA 1 11-QA. Increased cost to distribute modifications Technical reviews. Increased customer support Document reviews

11-QA 1 11-QA. Increased cost to distribute modifications Technical reviews. Increased customer support Document reviews Software Qualities Quality Assurance Maintainer Go Documentation Readable Ce Go Design Functionality Ease of use Ease of learning User Reliability Correctness Efficiency Low Cost Portability Increased

More information

Contents. What is Wirtschaftsmathematik?

Contents. What is Wirtschaftsmathematik? Contents. Introduction Modeling cycle SchokoLeb example Graphical procedure Standard-Form of Linear Program Vorlesung, Lineare Optimierung, Sommersemester 04 Page What is Wirtschaftsmathematik? Using mathematical

More information

APPROACHES TO SOFTWARE TESTING PROGRAM VERIFICATION AND VALIDATION

APPROACHES TO SOFTWARE TESTING PROGRAM VERIFICATION AND VALIDATION 1 APPROACHES TO SOFTWARE TESTING PROGRAM VERIFICATION AND VALIDATION Validation: Are we building the right product? Does program meet expectations of user? Verification: Are we building the product right?

More information

Introduction to Computers and Programming. Testing

Introduction to Computers and Programming. Testing Introduction to Computers and Programming Prof. I. K. Lundqvist Lecture 13 April 16 2004 Testing Goals of Testing Classification Test Coverage Test Technique Blackbox vs Whitebox Real bugs and software

More information

Software Testing. Quality & Testing. Software Testing

Software Testing. Quality & Testing. Software Testing Software Testing Software Testing Error: mistake made by the programmer/developer Fault: a incorrect piece of code/document (i.e., bug) Failure: result of a fault Goal of software testing: Cause failures

More information

SPICE auf der Überholspur. Vergleich von ISO (TR) 15504 und Automotive SPICE

SPICE auf der Überholspur. Vergleich von ISO (TR) 15504 und Automotive SPICE SPICE auf der Überholspur Vergleich von ISO (TR) 15504 und Automotive SPICE Historie Software Process Improvement and Capability determination 1994 1995 ISO 15504 Draft SPICE wird als Projekt der ISO zur

More information

Die wichtigsten Use Cases für MISRA, HIS, SQO, IEC, ISO und Co. - Warum Polyspace DIE Embedded Code-Verifikationslösung ist.

Die wichtigsten Use Cases für MISRA, HIS, SQO, IEC, ISO und Co. - Warum Polyspace DIE Embedded Code-Verifikationslösung ist. Die wichtigsten Use Cases für MISRA, HIS, SQO, IEC, ISO und Co. - Warum Polyspace DIE Embedded Code-Verifikationslösung ist. Christian Guß Application Engineer The MathWorks GmbH 2015 The MathWorks, Inc.

More information

Kapitel 2 Unternehmensarchitektur III

Kapitel 2 Unternehmensarchitektur III Kapitel 2 Unternehmensarchitektur III Software Architecture, Quality, and Testing FS 2015 Prof. Dr. Jana Köhler jana.koehler@hslu.ch IT Strategie Entwicklung "Foundation for Execution" "Because experts

More information

Standard for Software Component Testing

Standard for Software Component Testing Standard for Software Component Testing Working Draft 3.4 Date: 27 April 2001 produced by the British Computer Society Specialist Interest Group in Software Testing (BCS SIGIST) Copyright Notice This document

More information

Module 10. Coding and Testing. Version 2 CSE IIT, Kharagpur

Module 10. Coding and Testing. Version 2 CSE IIT, Kharagpur Module 10 Coding and Testing Lesson 23 Code Review Specific Instructional Objectives At the end of this lesson the student would be able to: Identify the necessity of coding standards. Differentiate between

More information

Software Testing. Definition: Testing is a process of executing a program with data, with the sole intention of finding errors in the program.

Software Testing. Definition: Testing is a process of executing a program with data, with the sole intention of finding errors in the program. Software Testing Definition: Testing is a process of executing a program with data, with the sole intention of finding errors in the program. Testing can only reveal the presence of errors and not the

More information

Basic Testing Concepts and Terminology

Basic Testing Concepts and Terminology T-76.5613 Software Testing and Quality Assurance Lecture 2, 13.9.2006 Basic Testing Concepts and Terminology Juha Itkonen SoberIT Contents Realities and principles of Testing terminology and basic concepts

More information

Regression Verification: Status Report

Regression Verification: Status Report Regression Verification: Status Report Presentation by Dennis Felsing within the Projektgruppe Formale Methoden der Softwareentwicklung 2013-12-11 1/22 Introduction How to prevent regressions in software

More information

Lecture 03 (04.11.2013) Quality of the Software Development Process

Lecture 03 (04.11.2013) Quality of the Software Development Process Systeme hoher Qualität und Sicherheit Universität Bremen, WS 2013/14 Lecture 03 (04.11.2013) Quality of the Software Development Process Christoph Lüth Christian Liguda Your Daily Menu Models of Software

More information

Module 10. Coding and Testing. Version 2 CSE IIT, Kharagpur

Module 10. Coding and Testing. Version 2 CSE IIT, Kharagpur Module 10 Coding and Testing Lesson 26 Debugging, Integration and System Testing Specific Instructional Objectives At the end of this lesson the student would be able to: Explain why debugging is needed.

More information

Manufacturing View. User View. Product View. User View Models. Product View Models

Manufacturing View. User View. Product View. User View Models. Product View Models Why SQA Activities Pay Off? Software Quality & Metrics Sources: 1. Roger S. Pressman, Software Engineering A Practitioner s Approach, 5 th Edition, ISBN 0-07- 365578-3, McGraw-Hill, 2001 (Chapters 8 &

More information

Random vs. Structure-Based Testing of Answer-Set Programs: An Experimental Comparison

Random vs. Structure-Based Testing of Answer-Set Programs: An Experimental Comparison Random vs. Structure-Based Testing of Answer-Set Programs: An Experimental Comparison Tomi Janhunen 1, Ilkka Niemelä 1, Johannes Oetsch 2, Jörg Pührer 2, and Hans Tompits 2 1 Aalto University, Department

More information

Mit einem Auge auf den mathema/schen Horizont: Was der Lehrer braucht für die Zukun= seiner Schüler

Mit einem Auge auf den mathema/schen Horizont: Was der Lehrer braucht für die Zukun= seiner Schüler Mit einem Auge auf den mathema/schen Horizont: Was der Lehrer braucht für die Zukun= seiner Schüler Deborah Löwenberg Ball und Hyman Bass University of Michigan U.S.A. 43. Jahrestagung für DidakEk der

More information

Why? A central concept in Computer Science. Algorithms are ubiquitous.

Why? A central concept in Computer Science. Algorithms are ubiquitous. Analysis of Algorithms: A Brief Introduction Why? A central concept in Computer Science. Algorithms are ubiquitous. Using the Internet (sending email, transferring files, use of search engines, online

More information

Introducing Formal Methods. Software Engineering and Formal Methods

Introducing Formal Methods. Software Engineering and Formal Methods Introducing Formal Methods Formal Methods for Software Specification and Analysis: An Overview 1 Software Engineering and Formal Methods Every Software engineering methodology is based on a recommended

More information

AUTOMATED TEST GENERATION FOR SOFTWARE COMPONENTS

AUTOMATED TEST GENERATION FOR SOFTWARE COMPONENTS TKK Reports in Information and Computer Science Espoo 2009 TKK-ICS-R26 AUTOMATED TEST GENERATION FOR SOFTWARE COMPONENTS Kari Kähkönen ABTEKNILLINEN KORKEAKOULU TEKNISKA HÖGSKOLAN HELSINKI UNIVERSITY OF

More information

TIn 1: Lecture 3: Lernziele. Lecture 3 The Belly of the Architect. Basic internal components of the 8086. Pointers and data storage in memory

TIn 1: Lecture 3: Lernziele. Lecture 3 The Belly of the Architect. Basic internal components of the 8086. Pointers and data storage in memory Mitglied der Zürcher Fachhochschule TIn 1: Lecture 3 The Belly of the Architect. Lecture 3: Lernziele Basic internal components of the 8086 Pointers and data storage in memory Architektur 8086 Besteht

More information

Search Engines Chapter 2 Architecture. 14.4.2011 Felix Naumann

Search Engines Chapter 2 Architecture. 14.4.2011 Felix Naumann Search Engines Chapter 2 Architecture 14.4.2011 Felix Naumann Overview 2 Basic Building Blocks Indexing Text Acquisition Text Transformation Index Creation Querying User Interaction Ranking Evaluation

More information

A Static Analyzer for Large Safety-Critical Software. Considered Programs and Semantics. Automatic Program Verification by Abstract Interpretation

A Static Analyzer for Large Safety-Critical Software. Considered Programs and Semantics. Automatic Program Verification by Abstract Interpretation PLDI 03 A Static Analyzer for Large Safety-Critical Software B. Blanchet, P. Cousot, R. Cousot, J. Feret L. Mauborgne, A. Miné, D. Monniaux,. Rival CNRS École normale supérieure École polytechnique Paris

More information

ida.com excellence in dependable automation

ida.com excellence in dependable automation IEC 61508 Maintenance Status IEC 61508 Maintenance Projekt ist aus dem zulässigen Zeitrahmen gelaufen Viele Baustellen auch durch neue Mitglieder (Frankreich, USA, IEC 61511 Team) Bestehende Anforderungen,

More information

Chapter 24 - Quality Management. Lecture 1. Chapter 24 Quality management

Chapter 24 - Quality Management. Lecture 1. Chapter 24 Quality management Chapter 24 - Quality Management Lecture 1 1 Topics covered Software quality Software standards Reviews and inspections Software measurement and metrics 2 Software quality management Concerned with ensuring

More information

Testing of safety-critical software some principles

Testing of safety-critical software some principles 1(60) Testing of safety-critical software some principles Emerging Trends in Software Testing: autumn 2012 Matti Vuori, Tampere University of Technology 27.11.2012 Contents 1/4 Topics of this lecture 6

More information

Embedded Software Development and Test in 2011 using a mini- HIL approach

Embedded Software Development and Test in 2011 using a mini- HIL approach Primoz Alic, isystem, Slovenia Erol Simsek, isystem, Munich Embedded Software Development and Test in 2011 using a mini- HIL approach Kurzfassung Dieser Artikel beschreibt den grundsätzlichen Aufbau des

More information

Über die Semantik von Modellierungssprachen

Über die Semantik von Modellierungssprachen Über die Semantik von Modellierungssprachen und des UML-Standards Prof. Dr. Bernhard Rumpe Technische Universität Braunschweig http://www.sse.cs.tu-bs.de/ Seite 2 What is a model? And why do we need modeling

More information

Testing LTL Formula Translation into Büchi Automata

Testing LTL Formula Translation into Büchi Automata Testing LTL Formula Translation into Büchi Automata Heikki Tauriainen and Keijo Heljanko Helsinki University of Technology, Laboratory for Theoretical Computer Science, P. O. Box 5400, FIN-02015 HUT, Finland

More information

Model Checking: An Introduction

Model Checking: An Introduction Announcements Model Checking: An Introduction Meeting 2 Office hours M 1:30pm-2:30pm W 5:30pm-6:30pm (after class) and by appointment ECOT 621 Moodle problems? Fundamentals of Programming Languages CSCI

More information

136 CHAPTER 4. INDUCTION, GRAPHS AND TREES

136 CHAPTER 4. INDUCTION, GRAPHS AND TREES 136 TER 4. INDUCTION, GRHS ND TREES 4.3 Graphs In this chapter we introduce a fundamental structural idea of discrete mathematics, that of a graph. Many situations in the applications of discrete mathematics

More information

Software Engineering Introduction & Background. Complaints. General Problems. Department of Computer Science Kent State University

Software Engineering Introduction & Background. Complaints. General Problems. Department of Computer Science Kent State University Software Engineering Introduction & Background Department of Computer Science Kent State University Complaints Software production is often done by amateurs Software development is done by tinkering or

More information

VDM vs. Programming Language Extensions or their Integration

VDM vs. Programming Language Extensions or their Integration VDM vs. Programming Language Extensions or their Integration Alexander A. Koptelov and Alexander K. Petrenko Institute for System Programming of Russian Academy of Sciences (ISPRAS), B. Communisticheskaya,

More information

Building an Architecture Model 1. 1. Entwerfen Sie mit AxiomSys ein Kontextdiagramm, das folgendermaßen aussieht:

Building an Architecture Model 1. 1. Entwerfen Sie mit AxiomSys ein Kontextdiagramm, das folgendermaßen aussieht: Building an Architecture Model 1 1. Entwerfen Sie mit AxiomSys ein Kontextdiagramm, das folgendermaßen aussieht: Wie Ihnen aus der vergangenen Lehrveranstaltung bekannt ist, bedeuten Sterne neben den Bezeichnungen,

More information

A: Ein ganz normaler Prozess B: Best Practices in BPMN 1.x. ITAB / IT Architekturbüro Rüdiger Molle März 2009

A: Ein ganz normaler Prozess B: Best Practices in BPMN 1.x. ITAB / IT Architekturbüro Rüdiger Molle März 2009 A: Ein ganz normaler Prozess B: Best Practices in BPMN 1.x ITAB / IT Architekturbüro Rüdiger Molle März 2009 März 2009 I T A B 2 Lessons learned Beschreibung eines GP durch das Business läßt Fragen der

More information

Verteilte Systeme 3. Dienstevermittlung

Verteilte Systeme 3. Dienstevermittlung VS32 Slide 1 Verteilte Systeme 3. Dienstevermittlung 3.2 Prinzipien einer serviceorientierten Architektur (SOA) Sebastian Iwanowski FH Wedel VS32 Slide 2 Prinzipien einer SOA 1. Definitionen und Merkmale

More information

Rigorous Software Engineering Hoare Logic and Design by Contracts

Rigorous Software Engineering Hoare Logic and Design by Contracts Rigorous Software Engineering Hoare Logic and Design by Contracts Simão Melo de Sousa RELEASE (UBI), LIACC (Porto) Computer Science Department University of Beira Interior, Portugal 2010-2011 S. Melo de

More information

Allgemein Nutzung Call-by-Value / Call-by-Reference Struct Typen Typische Fehler. Umgang mit Pointern. Seminar Effiziente Programmierung

Allgemein Nutzung Call-by-Value / Call-by-Reference Struct Typen Typische Fehler. Umgang mit Pointern. Seminar Effiziente Programmierung Alexander Lambertz 1/ 37 Umgang mit Pointern Seminar Effiziente Programmierung Alexander Lambertz Informatik Universität Hamburg 08. November 2012 Alexander Lambertz 2/ 37 Gliederung 1 Allgemein Definition

More information

SYSM 6304: Risk and Decision Analysis Lecture 5: Methods of Risk Analysis

SYSM 6304: Risk and Decision Analysis Lecture 5: Methods of Risk Analysis SYSM 6304: Risk and Decision Analysis Lecture 5: Methods of Risk Analysis M. Vidyasagar Cecil & Ida Green Chair The University of Texas at Dallas Email: M.Vidyasagar@utdallas.edu October 17, 2015 Outline

More information

SC207 Software Engineering. Review Report: Producing More Reliable Software

SC207 Software Engineering. Review Report: Producing More Reliable Software SC207 Software Engineering Review Report: Producing More Reliable Software Guo Zaiyi (SA1) Lecturer: Dr. Edmond C. Prakash School of Computer Engineering Nanyang Technological University Abstract This

More information

Jeffrey D. Ullman Anfang von: CS145 - Herbst 2004 - Stanford University Online unter: www.odbms.org. Folien mit weißem Hintergrund wurden hinzugefügt!

Jeffrey D. Ullman Anfang von: CS145 - Herbst 2004 - Stanford University Online unter: www.odbms.org. Folien mit weißem Hintergrund wurden hinzugefügt! Jeffrey D. Ullman Anfang von: CS145 - Herbst 2004 - Stanford University Online unter: www.odbms.org Folien mit weißem Hintergrund wurden hinzugefügt! Real SQL Programming Embedded SQL Call-Level Interface

More information

Software Engineering. Software Development Process Models. Lecturer: Giuseppe Santucci

Software Engineering. Software Development Process Models. Lecturer: Giuseppe Santucci Software Engineering Software Development Process Models Lecturer: Giuseppe Santucci Summary Modeling the Software Process Generic Software Process Models Waterfall model Process Iteration Incremental

More information

Coverability for Parallel Programs

Coverability for Parallel Programs 2015 http://excel.fit.vutbr.cz Coverability for Parallel Programs Lenka Turoňová* Abstract We improve existing method for the automatic verification of systems with parallel running processes. The technique

More information

Intelligent Systems: Three Practical Questions. Carsten Rother

Intelligent Systems: Three Practical Questions. Carsten Rother Intelligent Systems: Three Practical Questions Carsten Rother 04/02/2015 Prüfungsfragen Nur vom zweiten Teil der Vorlesung (Dimitri Schlesinger, Carsten Rother) Drei Typen von Aufgaben: 1) Algorithmen

More information

Sample Exam. 2011 Syllabus

Sample Exam. 2011 Syllabus ISTQ Foundation Level 2011 Syllabus Version 2.3 Qualifications oard Release ate: 13 June 2015 ertified Tester Foundation Level Qualifications oard opyright 2015 Qualifications oard (hereinafter called

More information

there s a renewed focus on real-world return on investment interdependencies and collaboration are key time is the critical metric

there s a renewed focus on real-world return on investment interdependencies and collaboration are key time is the critical metric !!! 1 trends implications the requirement is for responsible spending and IT governance there s a renewed focus on real-world return on investment IT consolidation Process automation collaboration between

More information

Chapter 1. Computation theory

Chapter 1. Computation theory Chapter 1. Computation theory In this chapter we will describe computation logic for the machines. This topic is a wide interdisciplinary field, so that the students can work in an interdisciplinary context.

More information

APPLICATION SETUP DOCUMENT

APPLICATION SETUP DOCUMENT APPLICATION SETUP DOCUMENT HeiTek Software Development GmbH Add-Ons Oracle Application Change Layout in Receiving Personalisation Example Ref Prepared by HeiTek Software Development GmbH Author: : Georg

More information

Introduction to Static Analysis for Assurance

Introduction to Static Analysis for Assurance Introduction to Static Analysis for Assurance John Rushby Computer Science Laboratory SRI International Menlo Park CA USA John Rushby Static Analysis for Assurance: 1 Overview What is static analysis?

More information

The Software Development Process

The Software Development Process Systeme hoher Qualität und Sicherheit Universität Bremen WS 2015/2016 Lecture 03 (26.10.2015) The Software Development Process Christoph Lüth Jan Peleska Dieter Hutter Your Daily Menu Models of software

More information

Zabin Visram Room CS115 CS126 Searching. Binary Search

Zabin Visram Room CS115 CS126 Searching. Binary Search Zabin Visram Room CS115 CS126 Searching Binary Search Binary Search Sequential search is not efficient for large lists as it searches half the list, on average Another search algorithm Binary search Very

More information

Outline. 1 Denitions. 2 Principles. 4 Implementation and Evaluation. 5 Debugging. 6 References

Outline. 1 Denitions. 2 Principles. 4 Implementation and Evaluation. 5 Debugging. 6 References Outline Computer Science 331 Introduction to Testing of Programs Mike Jacobson Department of Computer Science University of Calgary Lecture #3-4 1 Denitions 2 3 4 Implementation and Evaluation 5 Debugging

More information

Software Testing. Massimo Felici JCMB BP-G

Software Testing. Massimo Felici JCMB BP-G Software Testing Massimo Felici JCMB-1402 0131 650 5899 1BP-G04 0131 650 4408 mfelici@inf.ed.ac.uk What is Software Testing? Software Testing is the design and implementation of a special kind of software

More information

National University of Ireland, Maynooth MAYNOOTH, CO. KILDARE, IRELAND. Testing Guidelines for Student Projects

National University of Ireland, Maynooth MAYNOOTH, CO. KILDARE, IRELAND. Testing Guidelines for Student Projects National University of Ireland, Maynooth MAYNOOTH, CO. KILDARE, IRELAND. DEPARTMENT OF COMPUTER SCIENCE, TECHNICAL REPORT SERIES Testing Guidelines for Student Projects Stephen Brown and Rosemary Monahan

More information

I-Q SCHACHT & KOLLEGEN QUALITÄTSKONSTRUKTION GMBH ISO 26262:2011. Liste der Work Products aus der Norm

I-Q SCHACHT & KOLLEGEN QUALITÄTSKONSTRUKTION GMBH ISO 26262:2011. Liste der Work Products aus der Norm I-Q SCHACHT & KOLLEGEN QUALITÄTSKONSTRUKTION GMBH ISO 26262:2011 Liste der Work Products aus der Norm 1. Work Products in der ISO 26262:2011 1.1 Liste ISO-26262:2011, part 1 - Vocabulary no relevant work

More information

Computing Concepts with Java Essentials

Computing Concepts with Java Essentials 2008 AGI-Information Management Consultants May be used for personal purporses only or by libraries associated to dandelon.com network. Computing Concepts with Java Essentials 3rd Edition Cay Horstmann

More information

Experimental Comparison of Concolic and Random Testing for Java Card Applets

Experimental Comparison of Concolic and Random Testing for Java Card Applets Experimental Comparison of Concolic and Random Testing for Java Card Applets Kari Kähkönen, Roland Kindermann, Keijo Heljanko, and Ilkka Niemelä Aalto University, Department of Information and Computer

More information

Know or Go Practical Quest for Reliable Software

Know or Go Practical Quest for Reliable Software Know or Go Practical Quest for Reliable Software Dr.-Ing. Jörg Barrho Dr.-Ing. Ulrich Wünsche AVACS Project meeting 25.09.2014 2014 Rolls-Royce Power Systems AG The information in this document is the

More information

Microsoft Certified IT Professional (MCITP) MCTS: Windows 7, Configuration (070-680)

Microsoft Certified IT Professional (MCITP) MCTS: Windows 7, Configuration (070-680) Microsoft Office Specialist Office 2010 Specialist Expert Master Eines dieser Examen/One of these exams: Eines dieser Examen/One of these exams: Pflichtexamen/Compulsory exam: Word Core (Exam 077-881)

More information

Software Testing and Maintenance 1

Software Testing and Maintenance 1 Input Space Partitioning Introduction Equivalence Partitioning Boundary-Value Analysis Summary Software Testing and Maintenance 1 The Test Selection Problem The input domain of a program consists of all

More information

New quality management system

New quality management system Start PIE 27813 - EFQMEXAM New quality management system 06/2000, Page: 1(35) 1. approach: harmonisation of quality evaluation methods. EFQM is the basic method, because it provides the most complete coverage

More information

Upgrading Your Skills to MCSA Windows Server 2012 MOC 20417

Upgrading Your Skills to MCSA Windows Server 2012 MOC 20417 Upgrading Your Skills to MCSA Windows Server 2012 MOC 20417 In dieser Schulung lernen Sie neue Features und Funktionalitäten in Windows Server 2012 in Bezug auf das Management, die Netzwerkinfrastruktur,

More information

The Model Checker SPIN

The Model Checker SPIN The Model Checker SPIN Author: Gerard J. Holzmann Presented By: Maulik Patel Outline Introduction Structure Foundation Algorithms Memory management Example/Demo SPIN-Introduction Introduction SPIN (Simple(

More information

Formal Software Testing. Terri Grenda, CSTE IV&V Testing Solutions, LLC www.ivvts.com

Formal Software Testing. Terri Grenda, CSTE IV&V Testing Solutions, LLC www.ivvts.com Formal Software Testing Terri Grenda, CSTE IV&V Testing Solutions, LLC www.ivvts.com Scope of Testing Find defects early Remove defects prior to production Identify Risks Unbiased opinion When Should Testing

More information

ISTQB Certified Tester. Foundation Level. Sample Exam 1

ISTQB Certified Tester. Foundation Level. Sample Exam 1 ISTQB Certified Tester Foundation Level Version 2015 American Copyright Notice This document may be copied in its entirety, or extracts made, if the source is acknowledged. #1 When test cases are designed

More information

AP Computer Science AB Syllabus 1

AP Computer Science AB Syllabus 1 AP Computer Science AB Syllabus 1 Course Resources Java Software Solutions for AP Computer Science, J. Lewis, W. Loftus, and C. Cocking, First Edition, 2004, Prentice Hall. Video: Sorting Out Sorting,

More information

Lecture 03 (26.10.2015) The Software Development Process. Software Development Models. Where are we? Your Daily Menu.

Lecture 03 (26.10.2015) The Software Development Process. Software Development Models. Where are we? Your Daily Menu. Your Daily Menu Systeme hoher Qualität und Sicherheit Universität Bremen WS 2015/2016 Lecture 03 (26.10.2015) The Software Development Process Christoph Lüth Jan Peleska Dieter Hutter Models of software

More information

Produktfamilienentwicklung

Produktfamilienentwicklung Produktfamilienentwicklung Bericht über die ITEA-Projekte ESAPS, CAFÉ und Families Günter Böckle Siemens CT SE 3 Motivation Drei große ITEA Projekte über Produktfamilien- Engineering: ESAPS (1.7.99 30.6.01),

More information

(Refer Slide Time: 01:52)

(Refer Slide Time: 01:52) Software Engineering Prof. N. L. Sarda Computer Science & Engineering Indian Institute of Technology, Bombay Lecture - 2 Introduction to Software Engineering Challenges, Process Models etc (Part 2) This

More information

Rigorous Software Development CSCI-GA 3033-009

Rigorous Software Development CSCI-GA 3033-009 Rigorous Software Development CSCI-GA 3033-009 Instructor: Thomas Wies Spring 2013 Lecture 11 Semantics of Programming Languages Denotational Semantics Meaning of a program is defined as the mathematical

More information

Static Analysis. Find the Bug! 15-654: Analysis of Software Artifacts. Jonathan Aldrich. disable interrupts. ERROR: returning with interrupts disabled

Static Analysis. Find the Bug! 15-654: Analysis of Software Artifacts. Jonathan Aldrich. disable interrupts. ERROR: returning with interrupts disabled Static Analysis 15-654: Analysis of Software Artifacts Jonathan Aldrich 1 Find the Bug! Source: Engler et al., Checking System Rules Using System-Specific, Programmer-Written Compiler Extensions, OSDI

More information

CSCE 747 Software Testing and Quality Assurance

CSCE 747 Software Testing and Quality Assurance CSCE 747 Software Testing and Quality Assurance Lecture 04 Decision Tables Based Testing 1 Lec 04 Decision Tables 1 9/9/2013 Last Time Equivalence Class Testing Revision Control Systems Git intro Today

More information

Timebox Planning View der agile Ansatz für die visuelle Planung von System Engineering Projekt Portfolios

Timebox Planning View der agile Ansatz für die visuelle Planung von System Engineering Projekt Portfolios Agile Leadership Day 2015 Markus Giacomuzzi - Siemens Building Technologies Headquarters Zug Timebox Planning View der agile Ansatz für die visuelle Planung von System Engineering Projekt Portfolios structure

More information

Different Approaches to White Box Testing Technique for Finding Errors

Different Approaches to White Box Testing Technique for Finding Errors Different Approaches to White Box Testing Technique for Finding Errors Mohd. Ehmer Khan Department of Information Technology Al Musanna College of Technology, Sultanate of Oman ehmerkhan@gmail.com Abstract

More information

Testing, Debugging, and Verification

Testing, Debugging, and Verification Testing, Debugging, and Verification Testing, Part II Moa Johansson 10 November 2014 TDV: Testing /GU 141110 1 / 42 Admin Make sure you are registered for the course. Otherwise your marks cannot be recorded.

More information

Software Engineering

Software Engineering Software Engineering Lecture 04: The B Specification Method Peter Thiemann University of Freiburg, Germany SS 2013 Peter Thiemann (Univ. Freiburg) Software Engineering SWT 1 / 50 The B specification method

More information

Berufsakademie Mannheim University of Co-operative Education Department of Information Technology (International)

Berufsakademie Mannheim University of Co-operative Education Department of Information Technology (International) Berufsakademie Mannheim University of Co-operative Education Department of Information Technology (International) Guidelines for the Conduct of Independent (Research) Projects 5th/6th Semester 1.) Objective:

More information

Information Systems 2

Information Systems 2 Information Systems 2 Prof. Dr. Dr. L. Schmidt-Thieme MSc. André Busche Übung 9 0. Allerlei 1. Übung 2. Hands on some things 2.1 Saxon 2.2 Corba 28.06.10 2/ 0. Allerlei 1. Übung 2. Hands on some things

More information

Static Analysis of Dynamic Properties - Automatic Program Verification to Prove the Absence of Dynamic Runtime Errors

Static Analysis of Dynamic Properties - Automatic Program Verification to Prove the Absence of Dynamic Runtime Errors Static Analysis of Dynamic Properties - Automatic Program Verification to Prove the Absence of Dynamic Runtime Errors Klaus Wissing PolySpace Technologies GmbH Argelsrieder Feld 22 82234 Wessling-Oberpfaffenhofen

More information

Mitgliederversammlung OdA ICT Bern. Kurzreferat Cloud Computing. 26. April 2012 Markus Nufer

Mitgliederversammlung OdA ICT Bern. Kurzreferat Cloud Computing. 26. April 2012 Markus Nufer Mitgliederversammlung OdA ICT Bern Kurzreferat Cloud Computing 26. April 2012 Markus Nufer 1 Cloud Computing ist die grosse Veränderung in der ICT Cloud Computing ist in aller Munde: als neuartige Technologie,

More information

2) Write in detail the issues in the design of code generator.

2) Write in detail the issues in the design of code generator. COMPUTER SCIENCE AND ENGINEERING VI SEM CSE Principles of Compiler Design Unit-IV Question and answers UNIT IV CODE GENERATION 9 Issues in the design of code generator The target machine Runtime Storage

More information

Design by Contract beyond class modelling

Design by Contract beyond class modelling Design by Contract beyond class modelling Introduction Design by Contract (DbC) or Programming by Contract is an approach to designing software. It says that designers should define precise and verifiable

More information

Form follows Climate. CFD-Simulation. Mi 6.4.2011 15.00-18.00 Uhr

Form follows Climate. CFD-Simulation. Mi 6.4.2011 15.00-18.00 Uhr cfd analyse eines flugzeugflügels bei mach 7 Zum Start der Entwicklung unseres Entwurfswerkzeuges werden wir die Ergebnisse des Workshops Speculative Patterns aus dem Entwurfsprojekt 51,9 N mit der Software

More information

Social Skills für Software Experten

Social Skills für Software Experten Corporate Technology Social Skills für Experten Erfahrungsbericht vom Siemens Curriculum für Senior Architekten / Architekten Matthias Singer Siemens AG Learning Campus Copyright 2010. All rights reserved.

More information

Elementary Number Theory and Methods of Proof. CSE 215, Foundations of Computer Science Stony Brook University http://www.cs.stonybrook.

Elementary Number Theory and Methods of Proof. CSE 215, Foundations of Computer Science Stony Brook University http://www.cs.stonybrook. Elementary Number Theory and Methods of Proof CSE 215, Foundations of Computer Science Stony Brook University http://www.cs.stonybrook.edu/~cse215 1 Number theory Properties: 2 Properties of integers (whole

More information

NEW MEXICO Grade 6 MATHEMATICS STANDARDS

NEW MEXICO Grade 6 MATHEMATICS STANDARDS PROCESS STANDARDS To help New Mexico students achieve the Content Standards enumerated below, teachers are encouraged to base instruction on the following Process Standards: Problem Solving Build new mathematical

More information

Verification and Validation of Software Components and Component Based Software Systems

Verification and Validation of Software Components and Component Based Software Systems Chapter 5 29 Verification and Validation of Software Components and Component Based Christina Wallin Industrial Information Technology Software Engineering Processes ABB Corporate Research christina.wallin@mdh.se

More information

Verification of Imperative Programs in Theorema

Verification of Imperative Programs in Theorema Verification of Imperative Programs in Theorema Laura Ildikó Kovács, Nikolaj Popov, Tudor Jebelean 1 Research Institute for Symbolic Computation, Johannes Kepler University, A-4040 Linz, Austria Institute

More information

1 White-Box Testing by Stubs and Drivers

1 White-Box Testing by Stubs and Drivers White-box testing is a verification technique software engineers can use to examine if their code works as expected. In this chapter, we will explain the following: a method for writing a set of white-box

More information

Mathematical Induction

Mathematical Induction Mathematical Induction In logic, we often want to prove that every member of an infinite set has some feature. E.g., we would like to show: N 1 : is a number 1 : has the feature Φ ( x)(n 1 x! 1 x) How

More information

Software Engineering Reference Framework

Software Engineering Reference Framework Software Engineering Reference Framework Michel Chaudron, Jan Friso Groote, Kees van Hee, Kees Hemerik, Lou Somers, Tom Verhoeff. Department of Mathematics and Computer Science Eindhoven University of

More information

Extended Static Checking for Java

Extended Static Checking for Java Lukas TU München - Seminar Verification 14. Juli 2011 Outline 1 Motivation 2 ESC/Java example 3 ESC/JAVA architecture VC generator Simplify 4 JML + ESC/Java annotation language JML What ESC/Java checks

More information

TECH. Requirements. Why are requirements important? The Requirements Process REQUIREMENTS ELICITATION AND ANALYSIS. Requirements vs.

TECH. Requirements. Why are requirements important? The Requirements Process REQUIREMENTS ELICITATION AND ANALYSIS. Requirements vs. CH04 Capturing the Requirements Understanding what the customers and users expect the system to do * The Requirements Process * Types of Requirements * Characteristics of Requirements * How to Express

More information