VERIFICATION AND VALIDATION AUTOMATED TESTING TOOLS CLAUDIU ADAM
ABOUT ISDC ISDC... in a nutshell: - founded in 1990 in the Netherlands (Hilversum), 1999 in Romania (Cluj) - with a current pool more than 200 talented and performance-driven people - dedicated delivery models for end customers, IT services companies, and IT product companies - knowledgeable and experienced in technology stacks like Microsoft.NET, Java, Oracle; - core provided solutions: BI, Application Integration, Application Modernization 2
ABOUT ME 1.Bachelor degree - Stefan cel Mare University from Suceava - Computer Science department 2. Master degree at Technical University - Cluj Napoca Software Engineering department 3. Working since 2011 in ISDC, 3.5 years in IT as a Test Automation Engineer 3
CONTENT 1.Overview of a System Under Test 2.Why to automate things? 3.Testing levels 4.Testing types 5.Test execution (manual vs. automated path) 6.Functional testing 7.Non functional testing 8.Q & A 9.Conclusions 4
OVERVIEW OF A SYSTEM UNDER TEST Component 1 Component 3 SUT Other System Component 2 OS Hardware File System SUT = System Under Test OS = Operating System 5
TESTING LEVELS IN VER/VAL COMPONENT TESTING INTEGRATION TESTING VERIFICATION SYSTEM TESTING ACCEPTANCE TESTING VALIDATION 6
TESTING TYPES - TESTING LEVELS COMPONENT TESTING INTEGRATION TESTING SYSTEM TESTING ACCEPTANCE TESTING Functional Testing Non Functional Testing Structural Testing 7
FUNCTIONAL TESTING Followed principles: 1. We verify the compliance of the system with the functional requirements, and we use test design techniques to assure optimal functional coverage with our test cases 2. Test cases are documented and managed within the test management tools. We validate the test cases with the customer. 3. We keep the traceability to the requirements (Requirement -> Test Cases -> Issues) 4. Functional test cases represent a very important input for the future automated tests 8
NON FUNCTIONAL TESTING Endurance testing Load testing Performance testing Recovery testing Security testing Scalability testing Stress testing Usability testing 9
TEST TYPES ON ENVIRONMENTS Environment Test Types DEVELOPMENT Functional testing, Non functional testing TEST Functional testing, Non functional testing PREPRODUCTION Acceptance testing, Non functional testing PRODUCTION - 10
MANUAL PATH Requirements Design Test Cases Implement Test Cases Report Results Execute Test Cases 11
AUTOMATED PATH Requirements Design / Customize Test Framework and Cases Implement Test Cases Report Results Execute Test Cases in CI CI = Continuous Integration 12
TOOLS FOR FUNCTIONAL TESTING Technology Back-end Front-end Java Junit, TestNG, SoapUI Selenium IDE, RC, WebDriver Java, Watij, Sahi, Test Complete.NET MSTEST, NUnit, WCFStorm Selenium IDE, RC, WebDriverC#, Watin, Telerik, Test Complete BI In house built tools, Data Generators - 13
AUTOMATED FRONT-END FLOW IE Firefox Test Cases (Methods) Chrome Application Opera Safari Generate report 14
DEMO WORK WITH ECLIPSE, MAVEN AND SELENIUM WEBDRIVER JAVA 1. What you need for creating and running a test 2. @Before, @Test, @After 3. Execute test method and see the results
DEMO WORK WITH VISUAL STUDIO PREMIUM 2012, TFS AND SELENIUM WEBDRIVER C# 1. What you need for creating and running a test 2. How the [TestClass] and [TestMethod] look like 3. Execute test method and see the results
TOOLS FOR PERFORMANCE TESTING Technology Tools Java Load UI, SoapUI, JMeter.NET Visual Studio BI In house built tools, JMeter, Visual Studio 17
PERFORMANCE LEVEL AUTOMATION request response Load Agent 1 Load Agent 2 request response Application Server 1 Database Server 1 response request Web Server Load Agent 3 Application Server 2 Database Server 2 response request Load Agent 4 18
DEMO WORK WITH JMETER FOR MEASURING THE PERFORMANCE 1. What you need 2. How to create a Thread and Requests to server 3. Run tests and see/collect the results
CONTINUOUS INTEGRATION 20
DEMO WORK WITH JENKINS FOR CONTINUOUS INTEGRATION 1. What you need 2. How to create a jobs 3. Run job and see/send the results
TOOLS LIST FUNCTIONAL TESTING 1. Junit (Java) - http://junit.org/ 2. TestNG (Java) - http://testng.org 3. SoapUI (SOAP Exposed Services) - http://www.soapui.org/ 4. Selenium WebDriver (many languages) - http://docs.seleniumhq.org/ 5. MSTEST (.NET) - http://msdn.microsoft.com/en-us/library/ms182486.aspx 6. NUnit (.NET) - http://www.nunit.org/ 7. WCFStorm (WCF Exposed Services) - http://www.wcfstorm.com 8. Telerik (.NET) - http://www.telerik.com/ or free library Web AII http://www.telerik.com/automated-testing-tools/free-testing-framework.aspx 22
TOOLS LIST NON FUNCTIONAL TESTING 1. SoapUI (SOAP Exposed Services) - http://www.soapui.org/ 2. LoadUI - http://www.loadui.org/ 3. Jmeter - http://jmeter.apache.org/ 4. Microsoft Visual Studio Ultimate - http://msdn.microsoft.com/enus/library/vstudio/dd293540.aspx 23
TOOLS LIST TEST / ISSUE MANAGEMENT 1. TestLink - http://www.teamst.org/ 2. Jira - http://www.atlassian.com/software/jira/ 3. Microsoft Test Manager - http://msdn.microsoft.com/enus/library/vstudio/dd380763.aspx 24
TOOLS LIST CONTINUOUS INTEGRATION 1. Jenkins - http://jenkins-ci.org/ 2. TFS - http://tfs.visualstudio.com/ 3. Team City - http://www.jetbrains.com/teamcity/ 25
Q & A 26
CONCLUSIONS 27
CONTACT http://www.linkedin.com/in/adamclaudiu claudiu.adam@isdc.eu adam.claudiu86@gmail.com 28
THANK YOU 29