Neotys NeoLoad Test Automation Study. Study. Version: 1.0 Status from: :40:00 Status: Released Author: Ing.

Transcription

1 Neotys NeoLoad Test Automation Study Study Version: 1.0 Status from: :40:00 Status: Released Author: Ing. Jaroslav Klimes Document-ID: Classification: Internal Software Quality Lab. Alle Rechte vorbehalten. Druckdatum:

2 Contents DOCUMENT INFORMATION... II General... II Document-History... II Dokument-Qualitätssicherung... II Purpose and contents of this document... II 1. GENERAL About the developer Product portfolio Program parts Supported interfaces SPECIFICATION Test design Data-driven tests Verifications Reusability Robustness EXECUTION Planning Interfaces Error handling Protocolling ADMINISTRATION CONCLUSION ATTACHEMENT Picture list Version: 1.0 vom Author: Ing. Jaroslav Klimes Status: Released I / II

3 Document information General Connections to other documents: This analysis is a part of Test Automation Study by Software Quality Lab. Receivers: - Document-History Version Status Date Responsible Person Reason for change / Notes Dokument-Qualitätssicherung Role Name Availability: (Org.+Tel.+ ) Activity Date Signature Author Jarosalv Klimes created Reviewer Bernhard Groiß reviewed Reviewer Release Johannes Hochrainer Johannes Hochrainer reviewed released Purpose and contents of this document This document contains an objective analysis of a test automation tool and it should be available as a decision base for the people interested in the selection of a new tool. Version: 1.0 vom Author: Ing. Jaroslav Klimes Status: Released II / II

4 1. General 1.1. About the developer Neotys is a company operating on the market since Its headquarter is located in France, and it has two subsidiaries in USA and Germany. Neotys specializes exclusively on load testing and the tool described here, NeoLoad, is its only product. NeoLoad aims to provide a comprehensive load testing tool that is simple to use at the same time Product portfolio NeoLoad is the only product developed by Neotys Program parts NeoLoad is designed as a modular application, primarily because of the license policy. The license can be bought either just for the main application (Controller), which allows to perform load tests on webs using a number of virtual users limited by the licence, or for the additional protocol and monitoring modules. The protocol modules include Adobe Flex, GWT, Oracle Forms and SOAP, the monitoring modules allow performance monitoring of many server types and are sold in packages that are priced according to the number of monitoring modules enabled by the package. There is also a supplementary application for remote testing that is included in the price of the main application named Load Generator. It is a simple application allowing to launch the tests on a remote machine, while the main application controls the test execution and gathers results. This way the effective load on the test object can be substantially increased. NeoLoad can be installed on Windows, Linux or Solaris platforms, but the main application can only be installed on 32-bit systems. The remote machines can already run on 64-bit systems however, as there is a version of Load Generator designed for these systems. One of the most notable advantages of commercial tools such as NeoLoad in comparison with open source products is usually a better product support documentation. In the case of NeoLoad there is a very comprehensive documentation available both online and offline. This is supplemented by several online video-tutorials explaining the first steps with the tool and main usage scenarios. As for the support, Neotys offers two levels of it: The basic support via , and the gold support via and phone with new versions included. The basic level is free for six months after purchase, otherwise the support must be paid for. Speaking about the supported languages, the tool is available in english and french Supported interfaces NeoLoad is primarily focused on HTTP interface, but it can also cooperate with many other web-related technologies like PHP, J2EE, SOAP,.NET, ASP, AJAX, Flex, Flash, Air, GWT or Silverlight. Even the ERP technologies like SAP Web or Oracle Forms are supported. As for the interfaces to other test tools, these are not present. Version: 1.0 vom Autor: Ing. Jaroslav Klimes Status: Released 1 / 16

5 2. Specification The test specification is quite intuitive in NeoLoad. The basic idea is to record scenarios directly from the browser and check and adjust them afterwards if necessary. This way of test design works quite well here and there is no need to know any computer language you just add objects into an object tree. It is also possible to add requests manually of course but that can often be quite difficult, as it is not that easy to discover all data that are passed between the pages. Neotys 1: Main application interface After opening the main application and selecting New project the GUI shown on the Neotys 1 picture appears. It has the following main elements: Top bar menu and buttons that control the basic functions and allow setting application preferences. Version: 1.0 vom Autor: Ing. Jaroslav Klimes Status: Released 2 / 16

6 Three big buttons just under the top bar: Design, Runtime and Results. Using these buttons the user chooses which test phase he or she wants to work with. The cards below the three buttons that are dependent on the selected button. The card that is active on the Neotys 1 picture controls the scenario recording and editing Test design The basic and simplest way to design a test in NeoLoad begins with recording scenarios. The main application is supplied with a proxy server, which all network traffic from a selected browser is directed through. The tool also records browser settings together with the requests, which enables an exact reproduction of the requests during the test execution, with an exception of client-side Javascripts. Neotys 2: Virtual users The recorded browser traffic is saved under so-called Virtual users (See picture Neotys 2). Later during the test execution multiple Virtual Users can be used, composed into Populations. This way more realistic tests can be performed, assuming that not all users behave the same way on the website. After the recording dynamic data used by recorded pages are automatically detected including logins, cookies, or data passed between the pages. Once such usage of dynamic data is detected, the necessary data is extracted from the previous pages. Here it is also possible to set a random extraction of such value this may be helpful in cases when there is for example a menu on a page. The random dynamic value extraction can extract a different value related to a selection from the menu everytime. Version: 1.0 vom Autor: Ing. Jaroslav Klimes Status: Released 3 / 16

7 Making the Virtual Users more variable can also be performed in other ways: It is for example possible to extract a variable from any page manually. A way how this is done is quite interesting: normally you just have to fill the fields starting with and ending with (see picture Neotys 3), but there is also an Advanced mode as well, which allows extracting content using regular expressions. A value gained this way is stored into a variable that can be passed to another page anytime. Neotys 3: Variable extraction dialog (simple mode) (screenshot is cut on top) As a next step the Virtual User can be equipped with some control elements called Actions. These offer features like loops, delays, containers or branching. The following Actions that can be added to the scenario: Delay adds a break. Loop executes subordinate objects multiple times (can be a variable). While executes subordinate objects as long as a specified condition is true. If condition Version: 1.0 vom Autor: Ing. Jaroslav Klimes Status: Released 4 / 16

8 Variable modifier allows to adjust variables used in the pages. Container can contain multiple requests. The advantage of this is that execution results are also provided for the container then. Random Container can execute subordinate items at random. Try-catch Stop virtual User stops the execution of the scenario instantly. Fork Script allows an execution of a JavaScript. Rendez-vous can create load surges, synchronizing HTTP requests of multiple virtual users. Finally, a functional check of the results can be specified for the requests under a Virtual User through Assertions this topic is closer described in detail in chapter Verifications. When all adjustments of Virtual Users are completed, these can be checked. The check can reveal if all variables are passed correctly and all pages work as expected. Checking process can be done either just automatically or both automatically and manually. The automatic check can reveal just HTTP error codes and failed Assertions. With the manual check it is also possible to see the rendered HTML pages and determine if the server responses look as expected. Neotys 4: Populations card Version: 1.0 vom Autor: Ing. Jaroslav Klimes Status: Released 5 / 16

9 Thank you very much for downloading a public preview of our well known and highly appreciated tool studies. If you re interested in reading the whole document just apply here and request the document you want to read from our comprehensive set of studies already available. For further information on our tool studies visit our web site. Or, you can always use the following form to request the studies you want to read. Seite 1 von 2

10 Bestellformular für Tool-Studien/Order form for tool studies An/to Software Quality Lab GmbH Gewerbepark Urfahr Linz AUSTRIA Bitte ausgefüllt per an/ Please fill out and send to [email protected] Name/full name: Adresse/address: Firma/company: / Funktion/role: Telefon/phone: / [bitte füllen Sie die Daten vollständig und korrekt aus/please supply correct and complete data] Bestellliste/Order list [bitte kreuzen Sie die gewünschten Tools an/please select the tool studies] ALM, RM IBM Rational Requirements Composer Jama Contour 3.61 NEU! Micro Focus Borland Caliber Microsoft Team Foundation Server 2013 Polarion ALM 2014 HP ALM 11 VersioneOne 13.0 Visure Solutions IRQA 4 Test Management Imbus TestBench Micro Focus SilkCentral 2008 Microsoft VSTS 2010 Mozilla Testopia 2.1 Orcanos QPack 5 Polarion ALM NEU! Siemens SiTEMPPO 5.8 TestLink Seapine TestTrack 2012 Static Code Analysis SonarQube 4.5 GUI Test Automation Appium Bredex GUIdancer 4.2 HP UFT Professional 10 IBM Functional Tester Microsoft VSTS 2010 QF-Test NEU! Ranorex 3.3 SAP ecatt Selenium 2.0 SmartBear TestComplete 9.3 SmartBear SoapUI 5.1 Weitere Informationen auf unserer Website Embedded Systems LDRA 8.0 Razorcat Tessy 2.6 Load/Performance Apache JMeter 2.8 Grinder 3.4 HP LoadRunner 9.5 Neotys Neoload SmartBear LoadUI Unit Tests Google Test 1.5 JUnit 4.11 NUnit Die Toolnamen und eingetragenen Warenzeichen sind Eigentum der jeweiligen Toolhersteller/Tool names are property of tool vendors. Die Studien sind grundsätzlich auf Anfrage frei erhältlich. Software Quality Lab behält sich jedoch vor, die Aufwände für gewisse Branchen und Privatpersonen in Rechnung zu stellen (z.b. Beratungsfirmen, Toolhersteller usw.) Studenten erhalten die Studie zum halben Preis/All studies are basically free, except for certain branches of industry or private persons, for whom we invoice at cost. Students get a discount of 50%. Seite 2 von 2

11 Tool Evaluation Center Efficient, goal-oriented and professional tool evaluation Unique in Europe What is TEC? The Tool Evaluation Center (TEC) is a neutral environment for evaluating software tools related to software development, through which you will receive a detailed, vendorindependent overview of the suitability of different tools for your software projects. Your Partner for software quality and testing Consulting Implementation Training Automation Evaluation Software Quality Lab is your competent partner in software quality and testing and one of the leading independent consulting companies in Austria. We specialize ourselves in the improvement of efficiency, quality and security in the fields of software development and system processes. All tools in TEC are up to date, installed and configured, furnished with examples and ready for demonstration. Practical work and getting to know the tools in a handson manner is also possible. Your Benefit EETime-saving: Tool selection in a few days. No time-consuming research and product requests, no installation and testing. EEPreparation of information for different target groups from technicians to the managing board. EESupport provided by specialists, ranging from requirements through to a proof of concept or a pilot project. Our services around the tool selection Tool categories in the TEC EEApplication Life Cycle Management EERequirements Management EETest Management EEGUI Test Automation EETest Data Management EEComprehensible decision criteria: Using special techniques we evaluate the tools together with you, basing on your requirements as a foundation for your investment decision. EELarge selection and direct comparison of products ranging from market leaders through newcomers to open source software and freeware. Linz / Vienna / Graz / Lustenau / Munich Requirements elicitation, Creating a list of criteria Tool review, Pre-selection of tools, Practical getting to know the tools in the TEC Workshop with 3-5 suitable tools, Proof of Concept, Planning and implementing the pilot project Integration with other tools Tools in TEC Our TEC includes more then 40 installed tools from reputable manufacturer (e.g. IBM, HP, Micro Focus, Microsoft, Polarion, Siemens) as well as interesting newcomer products and alternatives from the open source area (e.g. JMeter, Selenium). Further details can be found at V Software Quality Lab GmbH [email protected]