End-to-end testing in the public domain door Rik Marselis. 22 september voordracht georganiseerd door. met de steun van

Size: px

Start display at page:

Download "End-to-end testing in the public domain door Rik Marselis. 22 september 2011. voordracht georganiseerd door. met de steun van"

Merry Peters
8 years ago
Views:

1 End-to-end testing in the public domain door Rik Marselis 22 september 2011 voordracht georganiseerd door Discussiegroep Software Testing met de steun van Ingenieurshuis, Antwerpen

2 End-to-end testing in the public domain TI/KVIV 22 September 2011 Rik Marselis To buy a book? 2 1

3 Introductions: Who are you? Type of function? Testing / Development / Management / Userorg. / Experience with Testing? Active tester / Have been involved / Testing?? / 3 Introducing: Rik Marselis Senior Test consultant (& teststrategist) Over 30 years in IT Over 15 years in testing & quality assurance Involved in TestNet and BNTQB Advisor at various organizations Leader expert group E2E-testing Author of books and articles TMap & ISTQB certified tester & accredited trainer Prince2 practitioner, CMMI en CISA 4 2

4 From IT to BT We are on the eve of Business Technology (BT) IT is a commodity, computers are a fact of life How you use computers in your business process that is the key difference between success and failure Business processes keep growing more and more complex What s the impact of BT on testing? End-to-end testing the business processes Testing of various systems must be linked Every test must have a comparable maturity level 5 Example of a chain: webshop Place order Receive invoice Pay invoice Receive item Receive order Create invoice Receive payment Send item Webshop Portal Order System Client mgt system External Bank system Mail System Warehouse Online Banking Invoice System Financial System Stock System Stock System Purchasing System 6 3

5 Business process supported by IT-systems Business process Place order Receive invoice Pay invoice Receive item Receive order Create invoice Receive payment Send item IT systems 7 A chain is only as strong as the weakest link Chains grow across organisations Damage inflicts all chain partners , SOA and the internet make data exchange simple Increasing customer demands reduce tolerance for failures 8 4

6 Testing one system in a project wish, legislation, policy, opportunity, problem operation & management requirements acceptance tests functional design technical design realisation system tests development tests 9 So many test levels! Do we need E2E? 10 5

7 E2E test: testing the process! Every part is tested and approved, but the car isn t! 11 End-to-end testing activities in a project Interface evaluation with system B Part of the project assignment Participate in the (over a consecutive series of systems ) Interface test with system B ts -tes E2E Participate in an project Part of the project assignment SIT 12 6

8 An?? 13 Definition of ing The dynamic test intended to demonstrate that the consecutive series of systems supports the (business) process according to specifications Actor Actor 14 7

9 What is an Electronic Patient File? PC at the general practitioner? Informationsystem in hospital? Central information exchange? Dedicated chipcard? 15 EPF a chain of systems GP Region. Hospital Etcetera Fysio. Central Information Exchange Academ. Hospital Apotheek DigiD UZI 16 8

10 Chance of failure and impact in a chain Chance of failure Ambulance Dependency Chance of failure De Chance of Dependency failure G.P. Apotheek Hospital Chance of Impact failure Dependency Chances of failure Chance of Dependency everywhere in the chain failure Chance of Impact at the end Central Dependency failure Dependencies in the chain Information UZI Chance of ncy y failureexchange denc ende17 n p e e p D De pe nd en cy Types of chains; dependencies Tertiary chain L M N O K P H Primary chain A Secundary chain B C D E F G 18 9

11 More than a chain; a network A Networks of informationsystems are also called a mesh 19 The Chain-game!! 10

12 Integral E2E Test Plan (IETP) Chain Risk Analysis Feasibility analysis 21 Risk based testing 22 11

13 Chain Risk Analysis 1. Determine participants and scope 2. Initial session 3. Analysis of Chance of failure 4. Determining chain risks 5. Completeness check 23 Chain risk analysis Impact Project board Project manager Line management System admin. Accountant Product riskuser department Designer analysis Functional maintenance Requirements manager Developer Test manager DBA Chance of failure 24 12

14 Feasibility analysis Knowledge & accessibility of knowledge 1/0 0/0 Beyond scope of the organization /1 Within scope of the organization Within scope of the department In project = number of systems 1 5/0 7 1/0 0/1 0/0 3/0 Test environments 4/0 1/0 0/1 0/0 Project environment 4/0 0/0 1/0 2/1 Complicating / obstructing factors in systems 0/0 Organisation & management 25 End-to-end testing not feasible? Test sub-chains! Business process Sub-processes Chain of systems 26 13

15 Natural development while maturity grows ate Str inc rea Test Policy Test Policy l tic a cts oje al on rat i Pr me lop ve De Test specification and Test execution ce Test Method Op e Test Plan an Test engineer Test Management en int Test co-ordinator Test Strategy / Master Test Plan Ma nt wh ile Test manager Ta c ma tu rity Test director se s gic Mission / Vision / Organisation strategy 27 End-to-end test process Adaptive Analysing chain risks and determining feasibility Determining -Scope -Sub-chains -Strategy -Infrastructure Planning and Estimating Creating the IETP Detailing dataflows Determining limitations and measures Determining -Strategy -Planning -Estimate Creating the SETP GO NO GO Specifying test infrastructure build and setup Creating the infrastructure design document GO NO GO Realising the infrastructure and intake Preparation Specifying logical cases Filling basic files and creating physical endto-end test cases Creating the scenario Inventorying processes and systems E2E Test Management E2E Infrastructure Coordination E2E Test Direction E2E Test Execution GO NO GO GO NO GO Executing the end-to-end test Completing the

16 The decision to execute an E2E test Considering benefits against the feasibility Benefits Risk reduction Results Quality of the chain Chain maintenance Chain test automation Feasibility Costs and lead time Complicating factors End-to-end test exceeds scope of project; the challenge of the E-to-E test director Business process Chain of systems Involvement Project Maintenance: Maintenance: Stable No activities system: release no testset available COTSsoftware: no documentation available Note: systems often in various organisations 30 15

17 Thesis There should be distinct requirements at the level of chains of systems The chain must be evaluated against these requirements 31 End-to-end test process Adaptive Analysing chain risks and determining feasibility Determining -Scope -Sub-chains -Strategy -Infrastructure Planning and Estimating Creating the IETP Detailing dataflows Determining limitations and measures Determining -Strategy -Planning -Estimate Creating the SETP GO NO GO Specifying test infrastructure build and setup Creating the infrastructure design document GO NO GO Realising the infrastructure and intake Preparation Specifying logical cases Filling basic files and creating physical endto-end test cases Creating the scenario Inventorying processes and systems E2E Test Management E2E Infrastructure Coordination E2E Test Direction E2E Test Execution GO NO GO GO NO GO Executing the end-to-end test Completing the

18 Example how to make E2E-test-cases Work in the opposite direction of the processtream Order in order system Invoice in financial system Picklist in logistics system IT-products 33 End-to-end testing with TMap NEXT Find faults across and beyond projects Find faults in business processes Builds confidence in business processes Excellent business case (it prevents damage, especially to the image of the organization) End-to-end testing with TMap NEXT provides the method, activities, tools and examples to handle the complexity of ing

19 End-to-end testing with TMap NEXT Actor Actor Sogeti

through Globally accepted Software Testing Certification standards

through Globally accepted Software Testing Certification standards Chris Van Bael Test consultant ps_testware Board KVIV SW Testing Board BNTQB ISEB Practitioner certified ISTQB Accredited trainer Book: Advanced Test Management 1 Enabling Test Professionals, through Globally