SOFTWARE TESTING. A Craftsmcm's Approach THIRD EDITION. Paul C. Jorgensen. Auerbach Publications. Taylor &. Francis Croup. Boca Raton New York

Transcription

1 SOFTWARE TESTING A Craftsmcm's Approach THIRD EDITION Paul C. Jorgensen A Auerbach Publications Taylor &. Francis Croup Boca Raton New York Auerbach Publications is an imprint of the Taylor & Francis Group, an informa business

2 Contents Preface to the Third Edirion Preface to the Second Edition Preface to the First Edition The Author xvii xix xxi xxiii PART I: A MATHEMATICAL CONTEXT 1 A Perspective on Testing Basic Definitions Test Cases Insights from a Venn Diagram Identifying Test Cases Functional Testing Structural Testing The Functional versus Structural Debate Error and Fault Taxonomies Levels of Testing 12 References 13 Exercises 14 2 Examples Generalized Pseudocode The Triangle Problem Problem Statement Discussion Traditional Implementation Structured Implementation The NextDate Function Problem Statement Discussion Implementation The Commission Problem Problem Statement Discussion Implementation 26 vii

3 viii Contents 2.5 The SATM System Problem Statement Discussion The Currency Converter Saturn Windshield Wiper Controller 31 References 31 Exercises 31 3 Discrete Math for Testers Set Theory Set Membership Set Definition The Empty Set Venn Diagrams Set Operations Set Relations Set Partitions Set Identities Functions Domain and Range Function Types Function Composition Relations Relations among Sets Relations on a Single Set., Propositional Logic Logical Operators Logical Expressions Logical Equivalence Probability Theory 47 Reference 49 Exercises 49 4 Graph Theory for Testers Graphs Degree of a Node Incidence Matrices Adjacency Matrices Paths Connectedness Condensation Graphs Cyclomatic Number Directed Graphs Indegrees and Outdegrees Types ofnodes Adjacency Matrix of a Directed Graph Paths and Semipaths Reachability Matrix 59

4 Contents ix n-connectedness Strong Components Graphs for Testing Program Graphs Finite State Machines Petri Nets Event-Driven Petri Nets StateCharts 69 References 72 Exercises 72 PART II: FUNCTIONAL TESTING 5 Boundary Value Testing Boundary Value Analysis Generalizing Boundary Value Analysis Limitations of Boundary Value Analysis Robustness Testing Worst-Case Testing Special Value Testing Examples Test Cases for the Triangle Problem Test Cases for the NextDate Function Test Cases for the Commission Problem Random Testing Guidelines for Boundary Value Testing 86 Exercises 87 6 Equivalence Class Testing Equivalence Classes Weak Normal Equivalence Class Testing Strong Normal Equivalence Class Testing Weak Robust Equivalence Class Testing Strong Robust Equivalence Class Testing Equivalence Class Test Cases for the Triangle Problem Equivalence Class Test Cases for the NextDate Function Equivalence Class Test Cases Equivalence Class Test Cases for the Commission Problem Output Range Equivalence Class Test Cases Guidelines and Observations 101 References 101 Exercises Decision Table-Based Testing Decision Tables Technique Test Cases for the Triangle Problem 108

5 x Contents 7.3 Test Cases for the NextDate Function First Try SecondTry Third Try Test Cases for the Commission Problem Guidelines and Observations 114 References 115 Exercises Retrospective on Functional Testing Testing Effort Testing Efficiency Testing Effectiveness Guidelines Case Study 123 PART III: STRUCTURAL TESTING 9 Path Testing DD-Paths Test Coverage Metrics Metric-Based Testing Test Coverage Analyzers Basis Path Testing McCabe's Basis Path Method Observations on McCabe's Basis Path Method Essential Complexity Guidelines and Observations 146 References 148 Exercises Dataflow Testing Define/Use Testing Example du-paths for Stocks du-paths for Locks du-paths for totallocks du-paths for Sales du-paths for Commission du-path Test Coverage Metrics Slice-Based Testing Example Style and Technique Guidelines and Observations 166 References 167 Exercises 167

6 Contents xi 11 Retrospective on Structural Testing Gaps and Redundancies Metrics for Method Evaluation Case Study Revisited Path-Based Testing Dataflow Testing Slice Testing 177 References 177 Exercises 178 PART IV: INTEGRATION AND SYSTEM TESTING 12 Levels of Testing Traditional View of Testing Levels Alternative Life Cycle Models Waterfall Spin-Offs Specification-Based Life Cycle Models Tfie SATM System Separating Integration and System Testing Structural Insights Behavioral Insights 198 References Integration Testing A Closer Look at the SATM System Decomposition-Based Integration Top-Down Integration Bottom-Up Integration Sandwich Integration Pros and Cons Call Graph-Based Integration Pairwise Integration Neighborhood Integration Pros and Cons Path-Based Integration New and Extended Concepts MM-Paths in the SATM System MM-Path Complexity Pros and Cons Case Study Decomposition-Based Integration Call Graph-Based Integration MM-Path-Based Integration 226 References 227 Exercises 227

7 xii Contents 14 System Testing Threads Thread Possibilities Thread Definitions Basis Concepts for Requirements Specification Data Actions Devices Events Threads Relationships among Basis Concepts Modeling with Basis Concepts Finding Threads Structural Strategies for Thread Testing Bottom-Up Threads Node and Edge Coverage Metrics Functional Strategies for Thread Testing Event-Based Thread Testing Port-Based Thread Testing Data-Based Thread Testing SATM Test Threads System Testing Guidelines Pseudostructural System Testing Operational Profiles Progression vs. Regression Testing ASF Testing Example 257 References 259 Exercises Interaction Testing Context of Interaction ATaxonomy of Interactions Static Interactions in a Single Processor Static Interactions in Multiple Processors Dynamic Interactions in a Single Processor Dynamic Interactions in Multiple Processors Interaction, Composition, and Determinism Client/Server Testing 280 References 281 Exercises 282 PART V: OBJECT-ORIENTED TESTING 16 Issues in Object-Oriented Testing Units for Object-Oriented Testing Implications of Composition and Encapsulation 286

8 Contents xiii 16.3 Implications of Inheritance Implications of Polymorphism Levels of Object-Oriented Testing GUI Testing Dataflow Testing for Object-Oriented Software Examples for Part V The Object-Oriented Calendar The Currency Conversion Application 291 References 296 Exercises Class Testing Methods as Units Pseudocode for o-ocalendar Class: CalendarUnit Class: testlt Class: Date Class: Day Class: Month Class: Year Unit Testing for Date.increment Classes as Units Pseudocode for the windshieldwiper Class Unit Testing for the windshieldwiper Class Object-Oriented Integration Testing UML Support for Integration Testing MM-Paths for Object-Oriented Software Pseudocode for o-ocalendar A Framework for Object-Oriented Dataflow Integration Testing Event- and Message-Driven Petri Nets Inheritance-Induced Dataflow Message-Induced Dataflow Slices? 323 Reference 324 Exercises GUI Testing The Currency Conversion Program Unit Testing for the Currency Conversion Program Integration Testing for the Currency Conversion Program System Testing for the Currency Conversion Program 330 Exercises Object-Oriented System Testing Currency Converter UML Description Problem Statement System Functions 338

9 xiv Contents Presentation Layer High-Level Use Cases Essential Use Cases Detailed GUI Definition Expanded Essential Use Cases Real Use Cases UML-Based System Testing StateChart-Based System Testing 349 References 349 PART VI: MILLENNIUM TESTING 21 Exploratory Testing The Context-Driven School Exploring Exploratory Testing Exploring a Familiär Example Exploratory and Context-Driven Testing Observations 358 References 358 Exercises Model-Based Testing Testing Based on Models Appropriate Models Peterson's Lattice Expressive Capabilities of Mainline Models Making Appropriate Choices Use Case-Based Testing Deriving Test Cases from Use Cases Interacting Use Cases How Many Use Cases? Commercial Tool Support for Model-Based Testing 367 References Test-Driven Development Test-Then-Code Cycles Automated Test Execution (Testing Frameworks) Java andjunit Example Java Source Code JUnit Test Code Remaining Questions Specification Based or Code Based? Configuration Management? Granularity? Pros, Cons, and Open Questions of TDD Retrospective on MDD versus TDD 386

10 Contents xv 24 A Closer Look at All Pairs Testing The All Pairs Technique Program Inputs Independent Variables Input Order Failures Due Only to Pairs of Inputs A Closer Look at the NIST Study Appropriate Applications for All Pairs Testing Recommendations for All Pairs Testing 401 References Epilogue: Software Testing Excellence Craftsmanship Best Practices of Software Testing Top 10 Best Practices for Software Testing Excellence Model-Driven Development Careful Definition and Identification of Levels of Testing System-Level Model-Based Testing System Testing Extensions Incidence Matrices to Guide Regression Testing Use of MM-Paths for Integration Testing Intelligent Combination of Specification-Based and Code-Based Unit-Level Testing Code Coverage Metrics Based on the Nature of Individual Units Exploratory Testing during Maintenance Test-Driven Development Mapping Best Practices to Diverse Projects A Mission-Critical Project A Time-Critical Project Corrective Maintenance of Legacy Code 408 Reference 408 Index, 409