Functional and LoadTest Strategies

Transcription

1 Test Automation Functional and LoadTest Strategies Presented by: Courtney Wilmott April 29, 2013 UTD CS6367 Software Testing and Validation

2 Definitions / Overview Software is a set of programs, procedures, algorithms and its documentation concerned with the operation of a data processing system. A software bug is an error, flaw, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways due to Data, Systems or Processing issues. Software testing is a process of verifying and validating that a software application or program: 1. Meets the business technical requirements that guided its design and development. 2. Works as expected. It is not possible to prove there are no faults in the software using testing. Goal of testing finding faults in the software demonstrating that there are no faults in the software (for the test cases that has been used during testing).

3 Failure Economics A study conducted by NIST in 2002 reports that software bugs cost the U.S. economy $59.5 billion annually. More than a third of this cost could be avoided if better software testing was performed. [10] It is commonly believed that the earlier a defect is found, the cheaper it is to fix it. The following table shows the cost of fixing the defect depending on the stage it was found. [11] For example, if a problem in the requirements is found only postrelease, then it would cost times more to fix than if it had already been found by the requirements review. With the advent of modern continuous deployment practices and cloud-based services, the cost of re-deployment and maintenance may lessen over time.

4 Failure Resolution Motivation for Production Systems: Recover System vs. Fixing the Bug Fault Localization Priority and Level of effort. Work-Around, Release code fix. Downtime/Validation/ Installation

5 Types of Testing Unit (Module) testing testing of a single module in an isolated environment Integration testing testing parts of the system by combining the modules System testing testing of the system as a whole after the integration phase Acceptance testing testing the system as a whole to find out if it satisfies the requirements specifications

6

7 Test Data Management Understanding the Challenges Ongoing administration of data in test environments continues to be a major challenge for QA teams even when test data setup has been completed successfully. Teams traditionally face challenges on three major fronts: - Managing reference data - Streamlining the data provisioning process and - Ensuring data quality in test environments To what degree is testing required and at what costs does it remain affordable? Are functional unit tests sufficient or is it necessary to conduct extensive stress tests of new releases of your applications before placing them into production? At the highest levels involving system integration tests, the application has to be challenged with test cases that are comparable to the real production data.

8 Functional and Regression Testing Why QTP Works Best? for your Web Services Applications

9 QucikTest Professional 8.2 Tutorial

10 Automated Testing QTP Introduction Preparation Record Enhance Run & Debug Reference Outline

11 Drawbacks of manual testing Time-consuming and tedious Impossible to test each feature thoroughly

12 Benefits of AT Fast ( 快 速 ) Reliable ( 可 靠 ) Repeatable ( 可 重 复 ) Programmable Comprehensive ( 全 面 ) Reusable( 可 重 用 )

13 QTP Introduction What is QTP? Mercury QuickTest Professional is functional test and regression test automation solution that address every major software application and environment. Basic purpose: to test standard web object, ActiveX controls and Visual Basic controls Structure: B/S and C/S

14 Testing process Preparation Record Enhance Run & Debug Analyze

15 Preparation Testing Plan: Computer: QTP: What process and web? What s the expected? When to begin and end? Close unneeded software (anti-virus, etc.) Internet Explorer settings QTP 8.2 or newer Required add-in

16 QuickTest Window Hop to record? Record

17

18 Enhance your test Add comment Creating checkpoints Parameterizing Tests Suggestions: Playback of it first

19 Add Comment Browser("Browser").Page("Page_2").WebEdit("#hobby").Set "Reading novels" Browser("Browser").Page("Page_2").WebButton("Submit").Click Rem Rem <<<<<<<<<<<<<check the output Browser("Browser").Page("Page_3").WebElement("input_text_1").Check CheckPoint("input_text_1") Browser("Browser").Page("Page_3").WebEdit("#hobby").Set "Watching TV" Browser("Browser").Page("Page_3").WebButton("Submit").Click

20 Creating checkpoints A checkpoint is a verification point that compares a current value for a specified property with the expected value for that property. Effect: it enables you to identify whether your Web site or application is functioning correctly. Pre-defined checkpoint User-defined checkpoint

21

22 User-defined checkpoint Dim Style, index Style = Browser("Browser").Page("Page_22").WebEdit("#Aim").GetROProperty("outer HTML") index = InStr (Style, "color=read") If index <>0 Then Reporter.ReportEvent micpass, "HTML compared", "OK" Else Reporter.ReportEvent micfail, "HTML compared", "Trouble" End If

23 Parameterizing Tests A parameter is a variable that is assigned a value from an external data source or generator. It greatly increases the power and flexibility of your test or component.

24 Parameters types Test, action or component parameters enable you to use values passed from your test or component, or values from other actions in your test. Data Table parameters enable you to create a datadriven test (or action) that runs several times using the data you supply.

25 Parameters types (Continued) Environment variable parameters enable you to use variable values from other sources during the run session. Random number parameters enable you to insert random numbers as values in your test or component.

26 Two types of Data sheets Global Sheet Usage: it will be available to all actions and control the number of test iterations. Local Sheet Usage : it will be available in that action only and control the number of action iterations.

27 Advanced configuration Regular expression Use Data Table formula

28 Wait() and sync Msgbox() Object Spy Debug toolbar

29 Analyze Read test report Why it fails if it exits

30 Active Screen Features of QTP Keyword View & Expert View Auto-Documentation Generator : Step & Function Check Point :Standard, Table, Text, etc.. DataTable: Global & Local Parameterizing Object Spy Object Repository

31 What s not mentioned Split Action Insert Action Find objects VBScript Descriptive Programming

32 Plus Keyword driven Testing It deploys the concept of Keyword-driven testing to radically simplify test creation and maintenance. When to use QTP? When the system is relatively stable Functional testing & Regression testing How long and how frequent

33 Qualification Be familiar with the system Have a better understanding of QTP Others

34 Load / Stress Testing The case for LoadRunner Why LoadRunner Works Best? For Web-Applications

35 Case study: SuperMedia May 2009

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51 Virtualized Load & Stress Testing

52 Summary QTP is one Keyword-driven testing tool suitable for functional testing and regression testing. Testing Process Preparation Record Enhance Run & Debug Analyze QTP Resources:

53 Load testing key benefits Cut costs by reducing your application performance issues in production Load test faster with the modern and powerful virtual user generator and robust correlation studio Speed scripting to accelerate time to market with patented HP TruClient technology Increase efficiency with measurable and repeatable load generation Isolate performance bottlenecks with system and end-user monitoring and a wide range of analysis components Use integrated HP Diagnostic software to pinpoint root causes of application issues LoadRunner resources: