www.hcltech.com SpecFlow Bridge between Business and Development Business Assurance & Testing SpecFlow An Opensource or Paid Tool AuthOrs: Anurag Sharma Sharma-an@hcl.com Is a software engineer at HCL Technologies India. Currently responsible for automating the test cases at API Level and providing the API Automation solution to End-Users. Samarth Gupta samarth.g@hcl.com WHITEPAPER April 2015
TABLE OF CONTENTS INTRODUCTION 3 PROBLEM FACED/ DURING IMPLEMENTATION OF GOOD PRACTICES 3 SOLUTION APPROACH 3 BENEFITS 7 LEARNING/IMPROVEMENTS 8 APPLICABILITY TO OTHER PROJECTS 8 REFERENCES 8 APPRECIATIONS RECEIVED 8 ABBREVIATIONS 8 ABOUT HCL 9 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 2
INTRODUCTION Behavior Driven Development (BDD) is an agile software development technique focused on improving a key factor in the successful development of any software product. BDD comes with the idea of filling the gap between the business and the development techniques. BDD centers on obtaining a clear understanding of desired software behavior through discussion with stakeholders. PROBLEM FACED/ DURING IMPLEMENTATION OF GOOD PRACTICES SpecFlow is an open-source.net tool that lets you write specifications using 100% Cucumber-compatible Gherkin syntax. The tool has a number of advantages such as: y It integrates with Visual Studio (VS), which means you get File New templates for creating new feature files and step definitions. y It gives complete VS debugger support, so you can set breakpoints on Given/ When/Then lines in your.feature files and step through their execution. y You can implement your step definitions in any.net language. y When you compile a project containing SpecFlow feature files, the output is a NUnit test assembly. So, you can use your favorite NUnit-compatible test runner or existing CI infrastructure to run the specifications with no additional configuration. CONSTRAINTS y As for the IDE, SpecFlow has full integration with Visual Studio 2008, 2010 and 2012, but all these IDEs use paid licenses. y To make automation more effective, we use TeamCity as our Continuous Integration platform with SpecFlow, which also requires yearly subscription. y Visual Studio Express Edition can be used free of charge even for commercial purposes. The limitations on the edition s functionality are borderline ridiculous. For example, it doesn t support Microsoft s own unit testing framework, and all add-ins are banned (like SpecFlow and ReSharper). It also doesn t work with Microsoft s own version control system (TFS). SOLUTION APPROACH INTEGRATING WITH VISUAL STUDIO EXPRESS Install SpecFlow and NUnit Before we get started, we need to install VS2010 Express, SpecFlow and NUnit. 1. Install VS2010 C# Express if you don t already have it 2. Install the latest SpecFlow version from the SpecFlow 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 3
3. Install the latest NUnit version (Nunit 2.6.3) from the NUnit download page 4. Add the SpecFlow directory (C:\Program Files\TechTalk\SpecFlow or similar) and the NUnit directory (C:\Program Files\NUnit 2.5.9\bin\net-2.0 or similar) to your Windows path. Adding SpecFlow templates to Visual Studio 2010 Express 1. While installing SpecFlow to VS2010 (non-express), it creates templates that can be accessed through the New Item menu. Fortunately, it is fairly trivial to add these to VS2010 Express 2. Just download the templates, unzip the directory and place the.zip files 3. (SpecFlowFeature.zip, SpecFlowStepDefinition.zip, etc.) into your Visual Studio 2010 Express C# templates directory (usual pathname C:\Users\username\ Documents\Visual Studio 2010\Templates\ItemTemplates\Visual C# or similar) This means you now have access to the SpecFlow templates while adding a new item to your C# project. Generating SpecFlow NUnit tests from Feature Files When you add a SpecFlow feature to the full version of VS2010, it automatically generates a unit test file behind the feature file, so that it s always up to date. This doesn t work in the Express edition, so you have to use the specflow.exe file to do this for yourself. Fortunately, this can be done in a batch file and then added as an external tool to VS2010 Express, so you get the same functionality but not quite as neat. 1. Enable Expert Settings in VS2010 Express (if not already enabled), by choosing Tools->Settings->Expert Settings. 2. Create a batch file in your project directory called generatespecflowtests.bat with the following content: Create an external tool setting the same as we did for the generate unit tests but with the parameters defined on this dialog: Command: Link to your batch file Arguments: $(ProjectFileName) Initial Directory: $(ProjectDir) Use Output Window: checked 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 4
Running SpecFlow NUnit tests from Feature Files To automatically run SpecFlow NUnit tests and capture a pretty HTML result, we have to create another batch file. 1. Create a batch file in your project directory named runspecflowtests.bat or similar. 2. Include below content in your batch file. Create an external tool setting the same as we did for the generate unit tests but with the parameters defined on this dialog: Command: Link to your batch file Arguments: $(TargetName)$(TargetExt)$(ProjectDir)$(ProjectFileName)$(Bi ndir)testresult.xml Initial Directory: $(BinDir) Use Output Window: checked Result: Result will be published in HTML format, which will include all the test execution steps of each scenario in the Given/When/Then format (detailed). It will show the result summary-wise, feature-wise and scenario-wise: 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 5
Integrating with Jenkins It is possible to Integrate SpecFlow with Jenkins to create a detailed HTML report. 1. Set up a new project in Jenkins 2. Add the NUnit Plugin and HTML Publisher Plugin to Jenkins 3. In your project, go to Configure -> Build -> Add a build step 4. In the dropdown, scroll down to -> Execute Windows Batch Command 5. Ensure this step is placed after your MSBuild step 6. Add the following, replacing the variables: 7. Click on Add post post-build action and select Publish HTML reports. Fill the path of HTML file created locally and the title with which you want to show the report 8. Click on Add post post-build action and select Publish Nunit test result report. In thet test report XMLs field, put TestResult.xml 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 6
9. Whenever you build the project, we will get the result in Jenkins as follows: BENEFITS QUANTITATIVE BENEFITS WORTH USD 29,302 35000 30000 25000 20000 Approach with out 'SpecFlow' i.e. via GUI Automation Testing 15000 10000 Approach with 'SpecFlow' i.e. via API Testing 5000 0 Total Cost involved in 1 year (In $, effort cost) Lumpsum Tool Cost involved (In $, per year), For QTP version 10 by HP Grand cost involved (In $, per year) QUALITATIVE BENEFITS y Tools used are Open Source Around 80% reduction in test execution time as compared to manual testing Exposure to Continuous Integration Servers helps in scheduling the overnight batch execution 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 7
BDD testing is not restricted to any domain, Feature file can be easily understood by any stakeholders Produces more detailed, easily understood test execution report SpecFlow can be integrated with existing tools like Selenium, TestComplete and QTP SpecFlow supports many languages SpecFlow dependencies are openly available, which can be easily integrated with any project LEARNING/IMPROVEMENTS It is easier to adopt, and it is best suited for the agile testing project; however, it can be equally used in other types of projects as well. It can be integrated with other tools as well. APPLICABILITY TO OTHER PROJECTS Having no dependency on the domain type, it can be used in any type of projects. It can easily be implemented. REFERENCES http://www.specflow.org/ APPRECIATIONS RECEIVED Highly appreciated from the client, and it has suggested to implement the same for other projects in the same account. ABBREVIATIONS Abbreviation BDD Expansion Behavior Driven Development 2015, HCL TECHNOLOGIES. REPRODUCTION PROHIBITED. THIS DOCUMENT IS PROTECTED UNDER COPYRIGHT BY THE AUTHOR, ALL RIGHTS RESERVED. 8
ABOUT HCL About HCL Technologies HCL Technologies is a leading global IT services company working with clients in the areas that impact and redefine the core of their businesses. Since its emergence on the global landscape, and after its IPO in 1999, HCL has focused on transformational outsourcing, underlined by innovation and value creation, offering an integrated portfolio of services including software-led IT solutions, remote infrastructure management, engineering and R&D services and business services. HCL leverages its extensive global offshore infrastructure and network of offices in 31 countries to provide holistic, multi-service delivery in key industry verticals including Financial Services, Manufacturing, Consumer Services, Public Services and Healthcare & Life sciences. HCL takes pride in its philosophy of Employees First, Customers Second which empowers its 100,240 transformers to create real value for customers. HCL Technologies, along with its subsidiaries, had consolidated revenues of US$ 5.7 billion, for the Financial Year ended as on 31 st December 2014 (on LTM basis). For more information, please visit www.hcltech.com About HCL Enterprise HCL is a $6.8 billion leading global technology and IT enterprise comprising two companies listed in India HCL Technologies and HCL Infosystems. Founded in 1976, HCL is one of India s original IT garage start-ups. A pioneer of modern computing, HCL is a global transformational enterprise today. Its range of offerings includes product engineering, custom & package applications, BPO, IT infrastructure services, IT hardware, systems integration, and distribution of information and communications technology (ICT) products across a wide range of focused industry verticals. The HCL team consists of over 105,699 professionals of diverse nationalities, who operate from 31 countries including over 505 points of presence in India. HCL has partnerships with several leading global 1000 firms, including leading IT and technology firms. For more information, please visit www.hcl.com Hello there! I am an Ideapreneur. I believe that sustainable business outcomes are driven by relationships nurtured through values like trust, transparency and flexibility. I respect the contract, but believe in going beyond through collaboration, applied innovation and new generation partnership models that put your interest above everything else. Right now 105,000 Ideapreneurs are in a Relationship Beyond the Contract with 500 customers in 31 countries. How can I help you?