TE TER. Continuous testing and delivery. number 21. SUBSCRIBE It s FREE for testers. Including articles by: Roy de Kleijn Polteq.

Transcription

1 TE TER SUBSCRIBE It s FREE for testers Essential for software Continuous testing and delivery Including articles by: Bogdan Bereza VictO Roy de Kleijn Polteq Mark Lehky Jessica Schiffmann Prism Informatics Eric M.S.P. Veith Wilhelm Büchner Hochschule/TU Bergakademie Freiberg testers June / 5 v2.0 number 21 THIS ISSUE OF PROFESSIONAL TESTER IS SPONSORED BY

2 Behave yourself by Roy de Kleijn Acceptance-test-driven development In this article I will explain an approach to preventing the second and third types of error. Acceptance-test-first Behaviour-driven development grew in the mind of its inventor Dan North (see dannorth.net/introducing-bdd) from testdriven development. However whereas TDD is a development activity dealing with unit testing, BDD is very definitely a testing approach: it aims to deliver effective acceptance testing based on defined requirements, but in an agile way compatible with lifecycles that develop and deliver business value in short iterations. Roy de Kleijn s tutorial and get-started guide to behaviour-driven development for testers Defects in software are caused by one of three kinds of human error. In the first, the developer understands what is required and attempts to achieve it but fails. In the second, the developer misunderstands the requirement and so succeeds in achieving the wrong thing. That misunderstanding also causes the third kind of error: working to achieve something that is not required and adds no business value. This also introduces a defect, but typically not one with the potential to cause product failure. More importantly, it wastes effort making project failure more likely. Put another way, TDD works from the inside of software creation, out. BDD works outside-in, assuring that everything done at lower levels is correct according to the high level. So, BDD is an application of the test-first principle used in TDD, but it changes the meaning of that term from unit-test-first to the greatly superior acceptance-test-first. This makes it very suitable for maintenance of legacy software as well as new-build, because knowledge of existing internal design and structure is not needed to do BDD. BDD in practice An agile iteration ( sprint or equivalent) begins with a short meeting defining its objectives, that is the requirements it will deliver. It s important that all involved contribute to defining these and their underlying requirements: but even then, the understanding of different people and roles tends to vary because of their different viewpoints and the different languages they use to communicate. BDD aims to solve this problem by having everyone work together also on acceptance criteria, called scenarios. Steve Watson discussed how to do this in the February 2013 issue of Professional Tester. 12 PT - June professionaltester.com

3 Gothenburg, Sweden 4-7 November 2013 Early Bird & Group Discounts Available NOW Europe s Biggest Gathering of Software Testers 6 Worldclass Keynotes 7 TuTorials 40+ presentations 50 Exhibitors, 1000 attendees LEarn, network, EngagE & Discuss. First time Delegate connection session Welcome Drinks Speed Networking The Test Lab Taste of Gothenburg Community Dinner active workshops Special Interest Discussion Tables Test Clinic & Tip Exchange The Official EuroSTAR Party and Much More!

4 In order to <receive benefit> As a <role> I want to <goal/desire> Figure 1: syntax of a BDD story Examples: input1 input2 outcome yes 20 accepted yes 13 not accepted Figure 3: example table Scenario: <scenario title> Given <pre-condition> And <optional additional pre-condition> When <action> And <optional additional action> Then <post-condition> And <optional additional post-condition> Figure 2: syntax of a BDD scenario The syntax of a BDD story is shown in figure 1. Obviously its creation is driven mainly by business representatives. The (usually many) acceptance criteria for the story are defined as BDD scenarios in the syntax shown in figure 2, surrounded if necessary by examples set out in a table as shown in figure 3. Notice here that each entry in the table is either a test data item or an expected outcome, and each line in the table is a test. This defined, uniform syntax makes the acceptance criteria easy to read and understand so that everyone involved can define them together. It also means they can be parsed and managed using tools. The testers present must ensure that they are clear and unambiguous. If everyone agrees that has been achieved, there should be no nasty surprises in store for anyone. This process is repeated for all scenarios against all acceptance criteria, so in theory minimum code required to pass them should be written. Whether that is true in practice depends of course on how well the code is written and that is where TDD comes in. scenario 7 5 step org) the stories can be executed immediately. Only the actual test code need be implemented. This is based upon and analogous to the way developers use TDD frameworks. Traceability of course must work in both directions. When a story is executed, the results map back to the scenarios and examples: every line is viewed in red or green type depending on the results of tests associated with it. The exact current state of the product is immediately apparent. In theory, any other documentation describing its functionality becomes superfluous. This information can be interfaced to continuous integration/delivery servers. It provides immediate feedback on the effect on existing functionality of every commit or deployment, in a form that is easily accessible and comprehensible to everyone. After the meeting the testers can get started on writing functional, executable (and therefore automatable) acceptance tests to implement the scenarios according to the acceptance criteria. Each scenario and each of the test steps of which it consists is taken through a series of states as shown in figure 4. Initially, the scenario fails (state 2) because there is no step. Now the step is written but it too fails because there is no test object (3). Now just enough code is created to make the step pass (4) and if necessary previously-created code is refactored (5). When all steps pass, the scenario passes (6) and if necessary code created for other scenarios is refactored (7). Figure 4: sequence of scenario and step states (after P. Creux, see Executable specifications and living documentation BDD s most important advantage is that it maps acceptance criteria, expressed as scenarios, directly to code. This can be seen as the ultimate in traceability. Using a BDD framework such as JBehave (also originally invented by Dan North, see Getting started with BDD I hope all this will make those PT readers who have not done so already want to try out BDD. To help, I have set up an example project which you can use to test web applications. It supports parallel test execution and takes a screenshot whenever the actual and expected outcome differs. To set it up, please follow the instructions in figure 5 Roy de Kleijn (roy.dekleijn@polteq.com) is a technical test consultant at Polteq where he works with clients on test process and automation, mainly in agile teams, and develops and presents training in Selenium/WebDriver. He is especially interested in web technology, new programming languages and knowledge transfer methods and publishes Selenium-related tutorials at and test-related articles at This article is based on one originally published in Dutch by TestNet (see 14 PT - June professionaltester.com

5 First install the following software: 1 Eclipse IDE for Java Developers from 2 Maven. In Eclipse, choose Help > Eclipse Marketplace and search for Maven Integration for Eclipse 3 JBehave Plugin. Follow the instructions at Now import the example project: 1 Download the sourcecode from 2 Extract the ZIP file to your chosen folder 3 In Eclipse, right-click in the Package Explorer panel and choose Import > Existing Maven Projects 4 Choose the folder where you stored the files as root Directory. 5 Complete the wizard and click Finish The project is in three parts (see figure 5a): the package org.google.pages contains an abstraction of the application under test the package org.google.steps contains the mapping between textual sentences and code the folder org/google/web contains the textual stories To execute the stories: Choose Run > Run Configurations (figure 5b) Right-click on Maven Build and choose New Select the project using Browse Workspace In the Goals field, enter the command integration-test -Dgroup=google Dbrowser=firefox Figure 5: setting up the example project Figure 5b Figure 5a PT - June professionaltester.com 15