Application Modelling

Size: px

Start display at page:

Download "Application Modelling"

Lizbeth Barrett
8 years ago
Views:

1 Seminar on Model-Based Testing as a Service Application Modelling Author: Antti Heimola Date: Dec-4, 2008 Time: Thu Dec- 4, 2008 Location: Tampere: TUT/Department of Software Systems, Tietotalo, class room TC Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

TUT/Department of Software Systems, Tietotalo, class room TC131 1 2008 Nokia

2 Content Model based Testing Case: KENDO - Testing modelling/automation needs - Time it takes 2 HR2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

3 Model based Testing James Bach s Blog: The Unbearable Lightness of Model-Based Testing Question: How do you suggest that you apply this technique with a complex application, something like MS Word for instance. I m not sure what technique you are referring to. If you are talking about using state models to describe a system, then it s interesting that you ask that, because I asked Harry Robinson that same question (it was even about applying it to Word) after seeing an otherwise fascinating talk about statebased testing that he gave, years ago. As I recall, he wasn t ready to answer that question. But I am. My answer is: ask Harry. Seriously, he has actually worked with model-based testing tools at Microsoft, in the years since I challenged him on this point. 3 HR2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

If you are talking about using state models to describe a system, then it s interesting that you ask that, because I asked Harry Robinson that same question (it was even about applying it to Word)

4 Model based Testing Wolfgang Grieskamp 4 HR2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

5 Application Modelling: Case KENDO GUI modelling tool flow design tool Specification tool Test case generation tool NOT linked to UML or other SW design schemes Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

or other SW design schemes 5 2008 Nokia AppModelling.

6 6 HR2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

7 System Success Factor of Model based Testing - technical z : System Modelling Other tools SSF = 3x+2y+z * Test Execution API y : Test Execution Control Other tools Test Service API Communication PC Successful model based testing requires support from test execution control tool and SUT test services testability built in x : Bluetooth, USB, WLAN, IP Communication System Under Test Test Support (testability) Target * Not any real formula, just a notion Test Services 7 HR2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

test services testability built in x : Bluetooth, USB, WLAN, IP Communication System Under Test Test Support (testability) Target * Not any real

8 Test Service example Model, i.e. Specification of a system event: Left soft key is Options Options menu contains following items: Sort, Instructions,, Exit Specification converted to test scripts: vw_verifytext vw_verifymenu Options Sort/Instructions/ /Exit Test scripts driving SUT using provided Test Services: Question of test automation needs. Many test tool vendors are happy with press button support Test Service must support Capture screen minimum while Test Execution Control takes care of text verification Capture screen support is NOT enough for verifying the whole menu content more sophisticated Test Support required 8 HR2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

Many test tool vendors are happy with press button support Test Service must support Capture screen minimum while Test Execution Control takes care of text verification Capture screen

9 Levels enabling cross Platform Modelling & Testing System Modelling + Test Mode CPP NFT: Performance, Stress, Endurance.. Probability Priority (test data) Test Execution API Test Execution Control Advanced Features Memory Usage CPU Linux S60 PC Etc. Test Service API Communication Bluetooth, USB, WLAN, IP Communication System Under Test Test Support (testability) Test Services Test Service API Communication Bluetooth, USB, WLAN, IP Communication System Under Test Test Support (testability) Test Services Test Service API Communication Bluetooth, USB, WLAN, IP Communication System Under Test Test Support (testability) Test Services In case Application flow (functionality) is the same, no modifications needed, the model can be executed on any platform supported by Test Execution Control and Test Services 9 HR2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

Test Service API Communication Bluetooth, USB, WLAN, IP Communication System Under Test Test Support (testability) Test Services Test Service API Communication Bluetooth, USB, WLAN, IP Communication

10 History 1. Ross Collard: Test Design: Developing Test Cases from Use Cases, KeyWord based test automation system 2001 Idea of integrating Use Cases & KWs 3. Mikko Nikkanen: KENDO I, 2005: Use Case based (test) modelling 4. Mikko Mäkinen: KENDO II, 2007: Application modelling Requests for productizing the approach 5. Productized (internal) version HR 2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

Mikko Nikkanen: KENDO I, 2005: Use Case based (test) modelling 4.

11 History 1999 * ** *) Ross Collard: Developing test cases from Use Cases, Software Testing&Quality Engineering, July/August 1999 **) Mikko Mäkinen: Model based approach to Software testing 2007, page HR 2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

Model based approach to Software testing 2007, page 28 11 HR 2008 Nokia

12 Some Principles UI level Application Modelling UI flow Design Target: Model all paths, not only some parts of them In case a model is too complex, most likely the GUI is too complex to use User Roles: Model Designer, Test Designer, Tester Clearly separated events: User Actions, System Responses Use of KeyWord based approach was logical: Press a button, Tap, Open Menu Managing complex models System Model, Application Models, Sub-Models, Use/Test Cases 12 HR 2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

System Responses Use of KeyWord based approach was logical: Press a button, Tap, Open Menu Managing complex models System Model, Application

13 KENDO User Roles & Key features Model Simulation. Model Designer, Test Designer, Tester Specification generation 13 HR 2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

generation 13 HR 2008 Nokia AppModelling.

14 14 HR 2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

15 Concept Reference: Implementing Lean Software Development Mary and Tom Poppendieck, Write executable specifications instead of requirements - Myth: The job of testing is to find defects : The job of tests is to PREVENT defects, not to find them 15 HR 2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

find defects : The job of tests is to PREVENT defects, not to find them 15 HR 2008 Nokia

16 KENDO time it takes S60 and Maemo Modeling time Minutes Minutes Minutes Minutes Minutes Hours Hours Model Designer Test Designer (ASTE) Execution Total * Total ** Application Specification Testing Phonebook ,3 7,5 Calendar ,2 11,2 Notepad ,7 8,2 Messaging ,8 8,6 Gallery ,5 8,5 Average ,7 8,8 Median ,5 8,5 * Excluding * before specification test execution type time support, (column ** after F) the specification type support ** Estimated with new auto-complete feature (75% of column C) Maemo (810) application modeling times with Kendo Application Model Designer Specification Test Designer Testing Execution Total [min] Total [h] Notes ,0 Clock ,4 Bookmarks ,8 Contacts ,5 Average [h] 1,8 2,6 1,1 1,1 1,9 8,4 8,4 Median [h] 1,8 2,8 1,0 1,0 2,0 8,4 8,4 16 HR 2008 Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

10,7 8,8 Median 90 345 60 65 21 10,5 8,5 * Excluding * before specification test execution type time support, (column ** after F) the specification type support ** Estimated with new auto-complete

17 Application Modelling or Model based Testing? Thank You P.S. SW testers are still needed, e.g. exploratory testing Nokia AppModelling.ppt / Antti Heimola Seminar on Model-Based Testing as a Service Tampere Dec-4, 2008

exploratory testing 17 2008 Nokia AppModelling.

How To Test For Performance

How To Test For Performance : Roles, Activities, and QA Inclusion Michael Lawler NueVista Group 1 Today s Agenda Outline the components of a performance test and considerations Discuss various roles, tasks, and activities Review