Test Automation Functional and LoadTest Strategies Presented by: Courtney Wilmott April 29, 2013 UTD CS6367 Software Testing and Validation
Definitions / Overview Software is a set of programs, procedures, algorithms and its documentation concerned with the operation of a data processing system. A software bug is an error, flaw, failure, or fault in a computer program or system that produces an incorrect or unexpected result, or causes it to behave in unintended ways due to Data, Systems or Processing issues. Software testing is a process of verifying and validating that a software application or program: 1. Meets the business technical requirements that guided its design and development. 2. Works as expected. It is not possible to prove there are no faults in the software using testing. Goal of testing finding faults in the software demonstrating that there are no faults in the software (for the test cases that has been used during testing).
Failure Economics A study conducted by NIST in 2002 reports that software bugs cost the U.S. economy $59.5 billion annually. More than a third of this cost could be avoided if better software testing was performed. [10] It is commonly believed that the earlier a defect is found, the cheaper it is to fix it. The following table shows the cost of fixing the defect depending on the stage it was found. [11] For example, if a problem in the requirements is found only postrelease, then it would cost 10 100 times more to fix than if it had already been found by the requirements review. With the advent of modern continuous deployment practices and cloud-based services, the cost of re-deployment and maintenance may lessen over time.
Failure Resolution Motivation for Production Systems: Recover System vs. Fixing the Bug Fault Localization Priority and Level of effort. Work-Around, Release code fix. Downtime/Validation/ Installation
Types of Testing Unit (Module) testing testing of a single module in an isolated environment Integration testing testing parts of the system by combining the modules System testing testing of the system as a whole after the integration phase Acceptance testing testing the system as a whole to find out if it satisfies the requirements specifications
Test Data Management Understanding the Challenges Ongoing administration of data in test environments continues to be a major challenge for QA teams even when test data setup has been completed successfully. Teams traditionally face challenges on three major fronts: - Managing reference data - Streamlining the data provisioning process and - Ensuring data quality in test environments To what degree is testing required and at what costs does it remain affordable? Are functional unit tests sufficient or is it necessary to conduct extensive stress tests of new releases of your applications before placing them into production? At the highest levels involving system integration tests, the application has to be challenged with test cases that are comparable to the real production data.
Functional and Regression Testing Why QTP Works Best? for your Web Services Applications
QucikTest Professional 8.2 Tutorial
Automated Testing QTP Introduction Preparation Record Enhance Run & Debug Reference Outline
Drawbacks of manual testing Time-consuming and tedious Impossible to test each feature thoroughly
Benefits of AT Fast ( 快 速 ) Reliable ( 可 靠 ) Repeatable ( 可 重 复 ) Programmable Comprehensive ( 全 面 ) Reusable( 可 重 用 )
QTP Introduction What is QTP? Mercury QuickTest Professional is functional test and regression test automation solution that address every major software application and environment. Basic purpose: to test standard web object, ActiveX controls and Visual Basic controls Structure: B/S and C/S
Testing process Preparation Record Enhance Run & Debug Analyze
Preparation Testing Plan: Computer: QTP: What process and web? What s the expected? When to begin and end? Close unneeded software (anti-virus, etc.) Internet Explorer settings QTP 8.2 or newer Required add-in
QuickTest Window Hop to record? Record
Enhance your test Add comment Creating checkpoints Parameterizing Tests Suggestions: Playback of it first
Add Comment Browser("Browser").Page("Page_2").WebEdit("#hobby").Set "Reading novels" Browser("Browser").Page("Page_2").WebButton("Submit").Click Rem Rem <<<<<<<<<<<<<check the output Browser("Browser").Page("Page_3").WebElement("input_text_1").Check CheckPoint("input_text_1") Browser("Browser").Page("Page_3").WebEdit("#hobby").Set "Watching TV" Browser("Browser").Page("Page_3").WebButton("Submit").Click
Creating checkpoints A checkpoint is a verification point that compares a current value for a specified property with the expected value for that property. Effect: it enables you to identify whether your Web site or application is functioning correctly. Pre-defined checkpoint User-defined checkpoint
User-defined checkpoint Dim Style, index Style = Browser("Browser").Page("Page_22").WebEdit("#Aim").GetROProperty("outer HTML") index = InStr (Style, "color=read") If index <>0 Then Reporter.ReportEvent micpass, "HTML compared", "OK" Else Reporter.ReportEvent micfail, "HTML compared", "Trouble" End If
Parameterizing Tests A parameter is a variable that is assigned a value from an external data source or generator. It greatly increases the power and flexibility of your test or component.
Parameters types Test, action or component parameters enable you to use values passed from your test or component, or values from other actions in your test. Data Table parameters enable you to create a datadriven test (or action) that runs several times using the data you supply.
Parameters types (Continued) Environment variable parameters enable you to use variable values from other sources during the run session. Random number parameters enable you to insert random numbers as values in your test or component.
Two types of Data sheets Global Sheet Usage: it will be available to all actions and control the number of test iterations. Local Sheet Usage : it will be available in that action only and control the number of action iterations.
Advanced configuration Regular expression Use Data Table formula
Wait() and sync Msgbox() Object Spy Debug toolbar
Analyze Read test report Why it fails if it exits
Active Screen Features of QTP Keyword View & Expert View Auto-Documentation Generator : Step & Function Check Point :Standard, Table, Text, etc.. DataTable: Global & Local Parameterizing Object Spy Object Repository
What s not mentioned Split Action Insert Action Find objects VBScript Descriptive Programming
Plus Keyword driven Testing It deploys the concept of Keyword-driven testing to radically simplify test creation and maintenance. When to use QTP? When the system is relatively stable Functional testing & Regression testing How long and how frequent
Qualification Be familiar with the system Have a better understanding of QTP Others
Load / Stress Testing The case for LoadRunner Why LoadRunner Works Best? For Web-Applications
Case study: SuperMedia May 2009
Virtualized Load & Stress Testing
Summary QTP is one Keyword-driven testing tool suitable for functional testing and regression testing. Testing Process Preparation Record Enhance Run & Debug Analyze QTP Resources: http://www8.hp.com/us/en/software/enterprisesoftware.html
Load testing key benefits Cut costs by reducing your application performance issues in production Load test faster with the modern and powerful virtual user generator and robust correlation studio Speed scripting to accelerate time to market with patented HP TruClient technology Increase efficiency with measurable and repeatable load generation Isolate performance bottlenecks with system and end-user monitoring and a wide range of analysis components Use integrated HP Diagnostic software to pinpoint root causes of application issues LoadRunner resources: http://www8.hp.com/us/en/software/enterprise-software.html