Keyword driven testing

Transcription

1 Keyword driven testing in theory & practice Levente Nemeth /11/2015 Nokia 2014

2 Agenda keyword driven testing - concept keyword driven testing robot framework how we do it? Nokia example 2 19/11/2015 Nokia 2014

3 keyword driven testing 3 19/11/2015 Nokia 2014

4 Keyword driven testing Concept The basic idea behind the keyword-driven testing approach in test automation is to separate the automation from the test case design source: /11/2015 Nokia 2014

5 Keyword driven testing Concept 5 19/11/2015 Nokia 2014 Automation engineer Verification engineer

6 Keyword driven testing Call Center example How would you test a new CRM website? Action Input I need to provide Keyword Test data (Parameters) I enter my username Sindy Enter username ${UNAME} I enter my password cute Translate Enter password ${PSWD} I click on Login button Press login button Verify if I logged in Welcome in CRM Verify login ${WELCOME } 6 19/11/2015 Nokia 2014

7 report library keywords Keyword driven approach Actions Test data Login ${username} ${password} Verify ${username} ${welcome_text} def type_uname(self, fieldid, username): ( ) def type_passwd(self, fieldid, password): ( ) def click_login (self, buttonid): ( ) driver script results 7 19/11/2015 Nokia 2014

8 Keyword driven testing Layers of abstraction Encapsulate your test code! Robot test case Keywords Robot custom library lib. #2 Ordinary test case Test case Robot telnet library Modify or replace business logic without changing test cases! 8 19/11/2015 Nokia 2014 Python telnet library Python Standard C++ library Linux Standard C library C Assembly Application OS

9 Keyword driven testing What is good? Easy to use (common test language for VE and TA) Enables writing tests in a more abstract manner Easy to maintain High level of reusability Automation can be started earlier in the SDLC (TDD) 9 19/11/2015 Nokia 2014

10 Keyword driven testing What could be better? High initial investment required Might hide too much of the business logic It takes time to adopt keyword driven practicies 10 19/11/2015 Nokia 2014

11 robot framework 11 19/11/2015 Nokia 2014

12 What is robot? Robot Framework is a generic, low level test automation framework that utilizes keyword driven approach many built-in functionalities can be easily extended with custom libraries defines a language to write generic keyword driven test cases defines an API to write keyword libraries in python and Java relies on external tools Hierarchical test suite setup & teardown logic 12 19/11/2015 Nokia 2014

13 How a robot test case look like? Actions Actions Test data Test data Test suite file: contains list of test cases Resource files: encloses library references and keyword definitions (linking) Libraries: implements testing logic behind keywords 13 19/11/2015 Nokia 2014 driver script results Variable files: contains parameter values that can be defined in several ways such as key value pairs (dictionary style) custom classes

14 Keyword driven testing Simple robot framework example Actual method names with input parameters defined in Selenium 2 library 14 19/11/2015 Nokia 2014

15 How a robot test execution look like? Levels of setup & teardown Suite setup TC1 setup TC1 execution TC1 teardown Test Connected step 1 Connect Deploy Test step conf. 21 Reset Disconnect Configure Undeploy Test step n2 TC2 setup TC2 execution Test step FAILED! TC2 teardown Suite teardown 15 19/11/2015 Nokia 2014

16 How we do it? Automation as a service 16 19/11/2015 Nokia 2014

17 Automation as a service E2E verification Verification team #1 Verification team #2 Automation team Verification team # /11/2015 Nokia 2014

18 Automation as a service Lifecycle Engineer request(s) Report issue in Jira TDD like approach with unit tests Prioritize and put request to backlog Discuss Deliver to repository 18 19/11/2015 Nokia 2014 Review and testing Impelmentation

19 19 19/11/2015 Nokia 2014

20 How robot framework (RF) look like in practice? Dependency on external tools RIDE RED jenkins Quality Center robot framework core built-in libraries telnet ssh ftp custom libraries custom protocols SUT 20 19/11/2015 Nokia 2014

21 Robot Framework functionalities Built-in robot library examples Many built-in libraries - Swing library for Java UI testing - Selenium library for web testing - Rammbock is a protocol tester (define custom messages) - etc. Possibility to develop custom libraries - Python 2.7, 3.5* - Java 21 19/11/2015 Nokia 2014