Risks & Trust. A different view on testing Erik Boelen

Risks & Trust A different view on testing Erik Boelen

Risks & Trust Objectives

Objectives Why do we test? What is all this commotion about risks? What is testing then for real? How do we place these ideas in the modern world? I will try to show you that testing is about common sense!

NO RISK NO TEST NO TEST NO TRUST

Risks & Trust Why do we test? And who is involved?

www.vodafone.nl Why do we test?

Scarlet Trio Why do we test?

KBC Online Why do we test?

Euroclear Why do we test?

European Commission Why do we test?

Developers Analysts Business owners Clients Marketing Operational people System testers Who is involved in this project?

Developers Analysts Business owners Clients Marketing Operational people System testers Who are the testers in this project?

How do they each test in this project? Developers Build code Test code for errors Test from a coding point of view Does my code give errors when I run it? Analysts Write specifications documents Test from a specifications point of view Did the developer do what I expect him to do?

How do they each test in this project? Business owners Decide on business requirements Test from a business point of view Is this a marketable product? Clients Are represented by business owners Test from a client point of view Can I use this product? Does it do what I expect it to do?

How do they each test in this project? Marketing Test from a marketing point of view Are all our marketing rules OK for this application? Operational people Test from an operational point of view Does this application fit in our monitoring tools? What happens if the application crashes? Is there an automatic restart?

System Testers How do they each test in this project? Test from a system point of view Does this application perform the functions as speficied in the requirements? Can this application handle the required amount of concurrent users? Does the system fit in the existing system architecture?...

Risks & Trust How to organise your testing?

How do you handle these roles in testing? Possibility 1 Let s bring them all together when the product is finished Show the product and have them test all at once Possibility 2 Get things organised in an effective way Which one would you choose?

Step 1 - The V-Model Business owners Clients Marketing Operational people System testers Analysts Developers How to prevent testing from becoming a bottleneck in your project?

Step 2 Context driven How do we adapt the V-Model principles in our context? Can be translated to More on this with the part of the challenges in the modern world

Step 3 Determine your test approach What kind of testing do we need? How much of this testing do we need? What kind of test techniques will we use? Who will perform these tests? When will we perform these tests? RISK

Risks & Trust Risks and the test approach

What is a risk? A risk is something that can become a problem, because of the change that the project is doing

What is all this commotion about risk? NO RISK NO TEST

Our first project together! Replacing the flaps of an airplane wing No functional impact What to test? Unit testing Unit integration testing System testing System integration testing User acceptance testing Operational acceptance testing

Documents? User requirements? Risks? Own experience? Intuition? What did we base ourselves on? Do we need some more information in order to set up our tests? If yes? What kind of information? If no? What are our following activities in testing?...or did we miss something in this story?

Option 1 The real requirements Replacing the flaps of an airplane wing No functional impact Unit testing Unit integration testing System testing System integration testing User acceptance testing Operational acceptance testing

Option 1 The real requirements Replacing the flaps of an airplane wing No functional impact User acceptance testing

Option 2 The real requirements Replacing the flaps of an airplane wing No functional impact Unit testing Unit integration testing System testing System integration testing User acceptance testing Operational acceptance testing

Option 2 The real requirements Replacing the flaps of an airplane wing No functional impact System integration testing User acceptance testing

Option 3 The real requirements Replacing the flaps of an airplane wing No functional impact Unit testing Unit integration testing System testing System integration testing User acceptance testing Operational acceptance testing

Option 3 The real requirements Replacing the flaps of an airplane wing No functional impact Extensive testing Unit testing Unit integration testing System testing System integration testing User acceptance testing Operational acceptance testing

Difference? What is the difference between option 1,2 and 3?

Difference? What is the difference between option 1,2 and 3? RISK

Our second project together! Upgrading the database to a higher version No functional impact What to test? Unit testing Unit integration testing System testing System integration testing User acceptance testing Operational acceptance testing

Our second project together! Upgrading the database to a higher version What to test? No functional impact RISK Unit testing Unit integration testing System testing System integration testing User acceptance testing Operational acceptance testing

Step 3 Determine your test approach What kind of testing do we need? How much of this testing do we need? What kind of test techniques will we use? Who will perform these tests? When will we perform these tests? We have to make a good assessment of our risks in our context and adapt our approach to this This is where your testing really starts!

What is the nicest part about risks? Risks keep on changing...... so we need to anticipate Luckily we are used to anticipating in real life!

Let s play some poker

What do we do? The Flop

The Flop What do we do? Based on the given situation we check whether we want to change cards

What do we do? The Turn

The Turn What do we do? Based on the given situation we check whether we want to change cards

The River Our cards are adapted as much as possible according to the situation

Conclusion on risks Testing makes sense...... but testing based on risks makes even more sense

Risks & Trust What is testing?

Software Testing What is testing then for real? Testing is the process consisting all life cycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects Risk Based Testing An approach to testing to reduce the level of product risks and inform stakeholders of their status, starting in the initial stages of a project. It involves the identification of product risks and the use of risk levels to guide the test process. ISTQB Glossary of Software Testing

What is testing then for real? Evaluation of software products Satisfy requirements Fit for purpose Defect detection Reduce the level of product risks NO TEST NO TRUST

Where do we build trust? We build trust in preparation & execution!

Risks & Trust Testing in the modern world Mobile Applications

Mobile testing - General approach Always start from Risk Based Testing approach Assess the subject under test What is the exact subject that we have to test? What is the scope of the test activities given the test subject? What are the requirements that we need to test against? Determine the product risks given the subject and the context Given the subject under test, what are the risks that we need to cover with tests? Define the approach based on these product risks

Mobile testing - Mobile App Risks Functional product risks Compatibility risks - Large number of Devices + Large number of OS Typical non-functional mobile risks Interruptions SMS, calls, network outage, battery removal,... Portrait vs landscape Motion sensors Status of the device (e.g. Flight mode) Memory allocation App interactions + interactions with SIM Cards Installation procedure + interruptions Battery status energy modes Localisation Security risks - Network, application, OS Userfriendlyness in combination with target groups

Mobile testing - Mobile App Test Approach Functional testing similar to non-mobile Compatibility testing Emulators & simulators SDK emulators from Android & Apple Simulation lools Monkeytalk for ios and Android Genymotion for Android Actual device testing 3 sets of devices 2 client site, one internal Non-functional testing Requirements check with the client Risk based assessment of the subject under test Test matrix based on the previous items Security risks We work together with highly specialized partner in security testing

Mobile testing - Test Automation Main question is What to automate? Functional test automation Not to be applied on the devices Automation principles applied to the emulators Simulation of the devices through drivers for functional testing of back office To be applied on the devices Functional testing tools currently investigating Test Studio Compatibility test automation Investigating solutions in the Cloud Offshoring to our Poland office Non-functional test automation Performance testing on the device itself Investigating solutions in the Cloud Performance testing of the back office of the app Devices are being simulated through drivers

Mobile app example Group exercise Let s do one more project together Mobile App Sports betting application Front End Graphical User Interface App Website with similar functionality Back End Database in the cloud Billing system Payment system

What are our requirements? Registration form Spots selection screen Betting screen Billing overview Payments overview Sports Betting App

Who is involved in testing? Sports Betting App How do we organise our testing based on the development lifecycle model? What are the risks that we see in this application? What would be the best test approach in this situation?