Developing a Load Testing Strategy

Size: px
Start display at page:

Download "Developing a Load Testing Strategy"

Transcription

1 Developing a Load Testing Strategy Michele Ruel St.George Bank CMGA 2005 Page 1

2 Overview... 3 What is load testing?... 4 Scalability Test... 4 Sustainability/Soak Test... 4 Comparison Test... 4 Worst Case... 4 Spike Test/Break Point... 4 Back-end Slowdown Test... 4 Performance Regression/Benchmark Test... 4 Why do we need Load Testing?... 5 How do we start Load Testing?... 6 Where does it fit in the organisation?... 6 Who runs the tests?... 6 Who decides if the test is successful?... 6 Who gathers information?... 6 When do you test?... 7 Software... 7 Hardware... 8 Anatomy of a Load Test... 8 Planning... 8 Reporting Testing Archive And how's it going at St.George Bank? Where we are in designing And how did your first big test go? But we do already have some lessons we've learned Conclusion Page 2

3 Overview The purpose of this paper is to describe how St.George Bank is creating a load test strategy to fit into its corporate framework. It will define load testing, suggest when to do load testing, highlight the considerations around load testing, and describe our experience during our first big load test. Page 3

4 What is load testing? Load testing is a method of applying a (usually synthetic) transaction load to an application system to test various performance and stability aspects of code and architecture. Here is a partial list of types of tests: Scalability Test These ramp-up tests determine the extent to which the application scales before reaching a choking point. This test will give us the bottleneck or limiting factor to performance. This could be CPU, memory or other system resources of the server, frontend or backend throughput limitation, or application limitations (e.g. Queue size, connection max, etc.). Sustainability/Soak Test This is a long running-test under low, medium and high load that determines if the application exhibits problems over an extended period of time (12-48 hours). It will tell us if there are any problems like memory leaks or other resource usage problems. Comparison Test Establish the impact/overhead of the new version of the application over the existing version. This test is done only if there is an existing version of the application. However, we have to make sure that we are comparing apples to apples as newer versions could have a different implementation from the old and/or the behavior of the application could differ. Worst Case This tests behaviour of a worst case scenario of the application under load. All of the parameters are at their maximum values. All other tests should be done with a vanilla scenario. This test could bring to light problems with the application that may have not been caught. This is basically boundary condition testing. Spike Test/Break Point This test helps to view the impact of a spiking load on the application behaviour/performance. It determines if a sudden increase in load causes any functional/performance problems. Back-end Slowdown Test Determine the impact of a back-end slowdown on the application performance. The back end server is artificially limited. These can be short ramp-up tests or long running tests. This test could bring out code defects that would not show up under normal conditions. Performance Regression/Benchmark Test This is particularly important for critical scenarios of the application that should be run as a performance regression mix to see the impact on overall performance and capacity. This test will tell us the overhead of the new release over the old one. Also this test could be run when doing optimization or tuning. Page 4

5 Why do we need Load Testing? Load testing done well will greatly improve the end product of your applications. If it is incorporated into the development framework, users will see a marked improvement in the systems that they work on every day. Those errors and system disasters that are always feared as an application moves to productions will no longer show up, because production loads and hardware will have been provend. Often organisations come to load testing after a very painful implementation occurred or has witnessed some other organisation s very painful implementation. Incorporating load testing is a lot of work to undertake to be sure of your code and environment, but consider the time it takes to constantly fix and explain to your customers why their application is so slow or unstable. The next section has a list of times when this type of automated testing would be userful. Page 5

6 How do we start Load Testing? Where does it fit in the organisation? Here we are talking about management of load testing activities. This could be the most important decision that you make. It will greatly influence the success of the testing. Think about the structure of your organisation and the politics that are involved. Most types of testing take place within the development area of the IT hierarchy, but most studies of this placement point to problems with conflicting goals. The project s success is measured by the on time, under budget rule. This goal usually limits the amount of testing allowed and skews the results of any testing done to positive. When the application then goes to production, all the load- and stress-related problems are solved there, with attendant bad experience for the users. The alternative is to place this function in an area that is separated from the development management, for instance, in production operations, the testing organisation, or capacity planning. If this is done, testing can be more objective and, if major performance or stabiliy problems are encountered, production implementation delay can be debated rather than allowed to continue even to meet budget or dates. Who runs the tests? Development Development could be a good choice because they have the best access to application architecture and can easily create the scripts that run the load tests. This way, load testing procedures can be developed as the application is developed and used for unit and system testing. The testing organisation This would seem to be the perfect area for load testing management, unless they are report to development. They will understand the importance of different types of tests and will have a methodology to structure and track their testing. Who decides if the test is successful? This is probably a committee decision. There should be a group composed of interested parties. project management, production support, capacity planning, and the testing organisation. This will be discussed later under exit criteria. Who gathers information? Once again, you must look to your organisation chart for the answer to this question. The type of data that is needed is business usage metrics, and then they must be translated to system load. Places in the typical IT organisation where people are used to dealing with these questions are capacity planning and performance analysis. Your production support organisation may also have the business contacts and the experience to translate these numbers to system load. Page 6

7 When do you test? During development We have developed a Risk Matrix that helps us to estimate the time needed for load testing and the types of load tests to run. If we know a particular bit of the system is prone to cause performance problems, or we are doing something like running an unpredictable query on our mainframe, we will plan on using these synthetic transactions to drive development tests. Before implementation This is the traditional time to do load testing. Our plan is to do several iterations and tests in a preliminary phase to shake out the scripts and determine code readiness for the final test. The next step would be to do a final load test at the end of customer acceptance testing. This test will have a go-no go decision on implementation; the passing (exit) criteria are set up at the beginning of load test planning; a group of concerned people from different areas will make this decision, so that development s bias for implementing and fixing in production will not be a factor. Before peak periods After the development testing requirements are finished, you can then move on to planning load tests to run before each important systems peak times, for instance, before the Christmas peak, to be sure that it will work smoothly during these times. At specific intervals For some systems that are fairly stable in terms of not much development going on, it might be useful to just schedule a yearly test to make sure that they re not about to break under increased load or different usage. This is also useful in that the planning of the test will ensure at least yearly contact with the business to see if there are any changes in usage metrics planned. Software There are a number of products that are available to assist you in running a load test; it is outside the charter of this paper to discuss specific software. Suffice to say that these products are of a level of sophistication now that they can not only control the injection of load into the system, but also collect system metrics during the run and collate the response times of each of the transactions. The software goes through a learning phase in which screen movements and choices are captured. This is then translated into a script that should be modified to vary input and transaction choices. If the software you choose doesn t have built-in metric measurement, then you must consider how you are going to monitor the test systems for load; perhaps your production organisation has tools that you can use in test. When you re choosing software, you must consider what types of systems you will be testing: web-based, mainframe-based, or other platforms. You will probably have to buy a certain number of licences for virtual users, so make sure you ll have enough to apply the peak load to the systems. Software varies widely in cost, so that will also be a big factor in your decision. Page 7

8 Hardware Hardware, ah hardware this is pivot on which all load testing turns. If your test system hardware is inadequate, or unstable, you will not be able to trust your test numbers. There are two main test hardware configurations to consider: A system exactly like the production system, duplicating network and system hardware and software, using production-size databases A test system that can easily be extrapolated to the production size, for instance, a complete system at half or a quarter the size of production. Keep in mind that the further away from production configuration the system gets, the harder to draw conclusions it will be. For this you might consider capacity planning modelling software to compare the test to the production performance. Be creative about where you get hardware from; perhaps you have older hardware that you can borrow to set up the system; if you re implementing a new system on new hardware, ask to bring in the production hardware earlier and test on it. Just keep in mind that this hardware will have to be readied for production, so build that into your schedule. Anatomy of a Load Test Planning Here are the minimal topics that you must think about before you run a load test: Each one is discussed in the following sections. Business metrics and translation to application load Software Hardware Test strategy: o What to test o What types of tests to run Entry criteria (what is needed before you start the formal testing) Exit criteria (how to know the load testing is over) Reporting Test archive The hard part: business metrics Usage Modelling is a the estimation of an application s production load, in terms of user volumes and transaction rates. These numbers are required to ensure load test scenarios apply a realistic work mix, running at realistic transaction rates. In addition, Exit Criteria are specified at loads determined by Usage Modelling this ensures that the response times we measure as a result can be compared against those specified in our Exit Criteria. Page 8

9 Capacity Planning is the logical organisation to take ownership of this phase and responsibility for its delivery to both load testing and other appropriate customer. They are the people who will probably have the data from the production systems, and they will also probably have the contacts in the business organisation. Who do you get the metrics from? Well, the business, of course? No? The customers should have these numbers. They should have been part of the justification of the project, shouldn t they? The answer here is get the best figures you can from the best people available. Start with the project sponsors. Find out how they justified the project. If this is a replacement application, then someone probably has these statistics somewhere. Maybe capacity planning has them, maybe production support has them, and maybe there are some key customers who know them. In the end, though, you may have to just make an educated guess. Remember to document the process and the outcomes. Give each metric a confidence value, so you can refine how you estimate in the future, and keep notes on where you got them. How do you verify them? Huh? Verify them? I had enough trouble getting them together. This has to be part of this fact-finding process; it s you chance to be a detective. Apply reasonability tests to the final figures. If it s an internal application, and the number of users is more than the number of employees, maybe there s something wrong! Really, dig a little further; ask another person or different questions; think about the relationship of the transaction numbers and evaluate if they seem reasonable. And, most important of all, have your customers, development, and capacity planning sign off on them. What are the minimal metrics needed? Seasonal peak date(s) and usage This is usually a percentage over average use, and maybe there are several peaks, so make sure you have all of them. There may be peaks with different usage criteria. Daily peak time(s) and usage Number of users Batch job schedule and usage System interfaces This is important from the user standpoint and also from the application designer standpoint. It will give you early warning of the extent of your test environment and the scope of your test. Transaction numbers for all the important transactions Response time expectation. It may be necessary to break this down by transaction type, like logon, and short medium, and long transactions. (Since customers only experience response times as comparative, it might be useful to have the technical people come up with standard response times by type of application, and then those numbers can be the basis for that discussion.) If you have a current system, this is easier. Page 9

10 Software Check to see you have the correct types of user platforms for this project and enough virtual users for your peaks. If you don t have enough users, plan on purchasing more or devise a strategy to use the ones you have in some creative way. Hardware Plan the hardware you need; start early to be sure that it s available when you need it and be adamant about the time you need to exclusively use it. Make sure that you have the technical people available to set it up and test it before you plan to do the formal test. Test Planning What to test Each project is different. The place to start would be assessing the risks in the project. Make sure you test the really risky pieces of the application, which would be new architecture, new programs, or any new or complicated system interfaces. After that, look at the high volume or resource-intensive transactions, and then everything else. Make sure to prioritise your tests. You probably won t have time for all of them. What types of tests to do Once again, this should be a product of your risk assessment. If you re concerned about memory leaks in a windows web server environment, for instance, you would do soak tests. If this is a customer-facing web application, you ll want to do extensive break and peak load testing Entry Criteria Entry Criteria are the conditions needed to start the formal testing. There is a standard set that you will probably use is: Stable code Stable hardware Data availability and quality Completion of metric collection This is an area where you have to be firm. You will probably be pushed to start without stable code and hardware, and you may ease into what other people think is formal load testing by helping with some functional testing. Make a point of reminding people that you are not in formal load testing if you haven t had these criteria met. It would be a good idea to establish this with each test team early in the project; get signoff on these criteria from all interested parties business, production, development, and capacity planning. Exit Criteria Formal load testing passes or fails depending on whether or not Exit (or Success) Criteria are satisfied. It is therefore crucial to seek agreement before commencing load testing. Page 10

11 Typical Exit Criteria specifications include: Acceptable response times slowest acceptable response times (expressed as 90 th percentile figures) for a range of transactions at peak load Acceptable server resource limits typically CPU limits at peak load. Exit Criteria Specification requires: Completion of the Metrics exercise a clear picture of Peak Load is required Stakeholder involvement with participation from Business, Project, Production support and load testing. Refinement with successive project experience. Schedule Load testing must be planned into the project schedule, so it is necessary to have a tool that will allow the project planners to gauge how long load testing will take. We have developed a risk matrix tool that will allow the planners to estimate how long testing will take. It takes into account various elements of risk that should be considered in planning a project schedule. This is still in its formative stages, and we expect there will be much more refinement as we do more load tests. Page 11

12 Reporting Reporting should be done daily to assess progress. Reports should go to all of the load test committee. At end of the load test schedule, a full report should be sent to all of the concerned project participants detailing the tests done and their outcomes. If the exit committee decides that the project has failed testing, then detailed reasons for the failure must be given as well as the tests which have to be repeated to pass load testing. Testing Archive Consideration should be given as to what data should be saved during the tests and beyond. If this information is saved, it can be used for regression testing and release load testing. Scripts While it will probably be necessary to regenerate the scripts for each new round of load testing, notes should be kept on adjustments and procedures used to accomplish various script items, such as how unique new records were generated, and how problems were solved Reports. Reports and data from major tests should be saved so they can be compared with the next round of test. And how's it going at St.George Bank? Where we are in designing Our processes keep changing, so we're aiming at a moving target. We re also trying to keep up with organisation changes, so we remain flexible. We have created a risk matrix tool for the estimators and are working with our risk planners to incorporate all of our concerns into one tool. And how did your first big test go? We are still immersed in a large CRM implementation. Our load testing core workgroup consisted of : One experienced load test planner One or two experienced load test scripter(s) One capacity planner One or two vendor software experts Production support people for our platforms, the network, and web operations. One problem we had with personnel was that because this project was so long, contract people kept changing and personal plans took people away from the team at crucial times. We have been testing for three months, first on the production platform, and then moving to a staging platform. This also turned out to be quite a problem Page 12

13 because we moved from the production platform to start the pilot. The stagining platform had not been used before and took forever to configure because all of our experts were off fixing pilot problems. But we do already have some lessons we've learned Business metrics: do them early, do them often and VERIFY them Try to insure that LT is separate from development Be tough about your entry and exit criteria Make sure technical experts are available to fix the hardware and application Check with your technical experts to see what they use as monitoring tools. Conclusion Load Testing can be of great value to an organisation in terms of guaranteed implementation success, stability and performance of an application. As expected in this type of outcome, this guarantee comes only from rigorous adherence to certain requirements: entry and exit criteria, and well-designed and executed tests. Page 13

Application Performance Testing Basics

Application Performance Testing Basics Application Performance Testing Basics ABSTRACT Todays the web is playing a critical role in all the business domains such as entertainment, finance, healthcare etc. It is much important to ensure hassle-free

More information

Recommendations for Performance Benchmarking

Recommendations for Performance Benchmarking Recommendations for Performance Benchmarking Shikhar Puri Abstract Performance benchmarking of applications is increasingly becoming essential before deployment. This paper covers recommendations and best

More information

Performance Testing for Managers. Presented by Stuart Moncrieff at SIGiST Melbourne on June 15 th, 2011

Performance Testing for Managers. Presented by Stuart Moncrieff at SIGiST Melbourne on June 15 th, 2011 Performance Testing for Managers Presented by Stuart Moncrieff at SIGiST Melbourne on June 15 th, 2011 What will be covered? Performance Testing as it applies to: Large multi-user enterprise IT applications

More information

Performance Testing Percy Pari Salas

Performance Testing Percy Pari Salas Performance Testing Percy Pari Salas Presented by : Percy Pari Salas Agenda What is performance testing? Types of performance testing What does performance testing measure? Where does performance testing

More information

Bringing Value to the Organization with Performance Testing

Bringing Value to the Organization with Performance Testing Bringing Value to the Organization with Performance Testing Michael Lawler NueVista Group 1 Today s Agenda Explore the benefits of a properly performed performance test Understand the basic elements of

More information

Performance Testing: Roles, Activities, and QA Inclusion

Performance Testing: Roles, Activities, and QA Inclusion : Roles, Activities, and QA Inclusion Michael Lawler NueVista Group 1 Today s Agenda Outline the components of a performance test and considerations Discuss various roles, tasks, and activities Review

More information

SOFTWARE PERFORMANCE TESTING SERVICE

SOFTWARE PERFORMANCE TESTING SERVICE SOFTWARE PERFORMANCE TESTING SERVICE Service Definition GTS s performance testing services allows customers to reduce the risk of poor application performance. This is done by performance testing applications

More information

10 Best Practices for Application Performance Testing

10 Best Practices for Application Performance Testing Business white paper 10 Best Practices for Application Performance Testing Leveraging Agile Performance Testing for Web and Mobile Applications 10 Best Practices for Application Performance Testing Table

More information

Load Testing and Monitoring Web Applications in a Windows Environment

Load Testing and Monitoring Web Applications in a Windows Environment OpenDemand Systems, Inc. Load Testing and Monitoring Web Applications in a Windows Environment Introduction An often overlooked step in the development and deployment of Web applications on the Windows

More information

Rapid Bottleneck Identification

Rapid Bottleneck Identification Rapid Bottleneck Identification TM A Better Way to Load Test WHITEPAPER You re getting ready to launch or upgrade a critical Web application. Quality is crucial, but time is short. How can you make the

More information

White paper: Unlocking the potential of load testing to maximise ROI and reduce risk.

White paper: Unlocking the potential of load testing to maximise ROI and reduce risk. White paper: Unlocking the potential of load testing to maximise ROI and reduce risk. Executive Summary Load testing can be used in a range of business scenarios to deliver numerous benefits. At its core,

More information

Rapid Bottleneck Identification A Better Way to do Load Testing. An Oracle White Paper June 2009

Rapid Bottleneck Identification A Better Way to do Load Testing. An Oracle White Paper June 2009 Rapid Bottleneck Identification A Better Way to do Load Testing An Oracle White Paper June 2009 Rapid Bottleneck Identification A Better Way to do Load Testing. RBI combines a comprehensive understanding

More information

Performance Workload Design

Performance Workload Design Performance Workload Design The goal of this paper is to show the basic principles involved in designing a workload for performance and scalability testing. We will understand how to achieve these principles

More information

INTEGRATION SOFTWARE: BUILD OR BUY?

INTEGRATION SOFTWARE: BUILD OR BUY? INTEGRATION SOFTWARE: BUILD OR BUY? DAVID CHAPPELL MARCH 2010 SPONSORED BY MICROSOFT CORPORATION No modern organization uses only one application. A single software solution just isn t enough to address

More information

Throughput Capacity Planning and Application Saturation

Throughput Capacity Planning and Application Saturation Throughput Capacity Planning and Application Saturation Alfred J. Barchi ajb@ajbinc.net http://www.ajbinc.net/ Introduction Applications have a tendency to be used more heavily by users over time, as the

More information

(Refer Slide Time: 01:52)

(Refer Slide Time: 01:52) Software Engineering Prof. N. L. Sarda Computer Science & Engineering Indian Institute of Technology, Bombay Lecture - 2 Introduction to Software Engineering Challenges, Process Models etc (Part 2) This

More information

Documentation for data centre migrations

Documentation for data centre migrations Documentation for data centre migrations Data centre migrations are part of the normal life cycle of a typical enterprise. As organisations expand, many reach a point where maintaining multiple, distributed

More information

Resource Monitoring During Performance Testing. Experience Report by Johann du Plessis. Introduction. Planning for Monitoring

Resource Monitoring During Performance Testing. Experience Report by Johann du Plessis. Introduction. Planning for Monitoring Resource Monitoring During Performance Testing Experience Report by Johann du Plessis Introduction During a recent review of performance testing projects I completed over the past 8 years, one of the goals

More information

Test Run Analysis Interpretation (AI) Made Easy with OpenLoad

Test Run Analysis Interpretation (AI) Made Easy with OpenLoad Test Run Analysis Interpretation (AI) Made Easy with OpenLoad OpenDemand Systems, Inc. Abstract / Executive Summary As Web applications and services become more complex, it becomes increasingly difficult

More information

An Oracle White Paper February 2010. Rapid Bottleneck Identification - A Better Way to do Load Testing

An Oracle White Paper February 2010. Rapid Bottleneck Identification - A Better Way to do Load Testing An Oracle White Paper February 2010 Rapid Bottleneck Identification - A Better Way to do Load Testing Introduction You re ready to launch a critical Web application. Ensuring good application performance

More information

Perform-Tools. Powering your performance

Perform-Tools. Powering your performance Perform-Tools Powering your performance Perform-Tools With Perform-Tools, optimizing Microsoft Dynamics products on a SQL Server platform never was this easy. They are a fully tested and supported set

More information

ISTQB Certified Tester. Foundation Level. Sample Exam 1

ISTQB Certified Tester. Foundation Level. Sample Exam 1 ISTQB Certified Tester Foundation Level Version 2015 American Copyright Notice This document may be copied in its entirety, or extracts made, if the source is acknowledged. #1 When test cases are designed

More information

Performance Testing of a Large Wealth Management Product

Performance Testing of a Large Wealth Management Product Performance Testing of a Large Wealth Management Product Meherphani Nori & Global Head Quality Assurance Krishna Kankipati & Vice President Mohan Pujari & Product Specialist Broadridge Financial Solutions

More information

Case Study - I. Industry: Social Networking Website Technology : J2EE AJAX, Spring, MySQL, Weblogic, Windows Server 2008.

Case Study - I. Industry: Social Networking Website Technology : J2EE AJAX, Spring, MySQL, Weblogic, Windows Server 2008. Case Study - I Industry: Social Networking Website Technology : J2EE AJAX, Spring, MySQL, Weblogic, Windows Server 2008 Challenges The scalability of the database servers to execute batch processes under

More information

Guideline for stresstest Page 1 of 6. Stress test

Guideline for stresstest Page 1 of 6. Stress test Guideline for stresstest Page 1 of 6 Stress test Objective: Show unacceptable problems with high parallel load. Crash, wrong processing, slow processing. Test Procedure: Run test cases with maximum number

More information

Bernie Velivis President, Performax Inc

Bernie Velivis President, Performax Inc Performax provides software load testing and performance engineering services to help our clients build, market, and deploy highly scalable applications. Bernie Velivis President, Performax Inc Load ing

More information

WHAT WE NEED TO START THE PERFORMANCE TESTING?

WHAT WE NEED TO START THE PERFORMANCE TESTING? ABSTRACT Crystal clear requirements before starting an activity are always helpful in achieving the desired goals. Achieving desired results are quite difficult when there is vague or incomplete information

More information

Cloud Computing Characteristics Are Key

Cloud Computing Characteristics Are Key White Paper by Christopher Olive Chief Architect www.gpworldwide.com 2011 What is cloud computing? Cloud computing remains the buzzword winner of the current technology paradigm. The National Institute

More information

Achieving business benefits through automated software testing. By Dr. Mike Bartley, Founder and CEO, TVS (mike@testandverification.

Achieving business benefits through automated software testing. By Dr. Mike Bartley, Founder and CEO, TVS (mike@testandverification. Achieving business benefits through automated software testing By Dr. Mike Bartley, Founder and CEO, TVS (mike@testandverification.com) 1 Introduction During my experience of test automation I have seen

More information

SolovatSoft. Load and Performance Test Plan Sample. Title: [include project s release name] Version: Date: SolovatSoft Page 1 of 13

SolovatSoft. Load and Performance Test Plan Sample. Title: [include project s release name] Version: Date: SolovatSoft Page 1 of 13 SolovatSoft Load and Performance Test Plan Sample Title: [include project s release name] Version: Date: SolovatSoft Page 1 of 13 Approval signatures Project Manager Development QA Product Development

More information

Performance Test Process

Performance Test Process A white Success The performance testing helped the client identify and resolve performance bottlenecks which otherwise crippled the business. The ability to support 500 concurrent users was a performance

More information

Essential. Guide TESTING YOUR DISASTER RECOVERY ABILITY. The. February 2006. By Paul Robichaux

Essential. Guide TESTING YOUR DISASTER RECOVERY ABILITY. The. February 2006. By Paul Robichaux Essential The February 2006 to Guide TESTING YOUR DISASTER RECOVERY ABILITY By Paul Robichaux It doesn t matter how good your disaster recovery procedures and processes are if you don t test them. This

More information

Network Monitoring with Xian Network Manager

Network Monitoring with Xian Network Manager Network Monitoring with Xian Network Manager Did you ever got caught by surprise because of a network problem and had downtime as a result? What about monitoring your network? Network downtime or network

More information

Development, Acquisition, Implementation, and Maintenance of Application Systems

Development, Acquisition, Implementation, and Maintenance of Application Systems Development, Acquisition, Implementation, and Maintenance of Application Systems Part of a series of notes to help Centers review their own Center internal management processes from the point of view of

More information

Performance Testing. on Production System

Performance Testing. on Production System Performance Testing on Production System Abstract Performance testing is conducted to check whether the target application will be able to meet the real users expectations in the production environment

More information

Why You Need Email Archiving

Why You Need Email Archiving Why You Need Email Archiving www.exclaimer.com Table of Contents Introduction...2 The IT Administrator...3 The Email User...5 The Team Leader...6 The Senior Manager/Business Owner...7 Conclusion...8-1

More information

A Guide To Evaluating a Bug Tracking System

A Guide To Evaluating a Bug Tracking System A Guide To Evaluating a Bug Tracking System White Paper By Stephen Blair, MetaQuest Software Published: October, 2004 Abstract Evaluating a bug tracking system requires that you understand how specific

More information

Table of Contents INTRODUCTION... 3. Prerequisites... 3 Audience... 3 Report Metrics... 3

Table of Contents INTRODUCTION... 3. Prerequisites... 3 Audience... 3 Report Metrics... 3 Table of Contents INTRODUCTION... 3 Prerequisites... 3 Audience... 3 Report Metrics... 3 IS MY TEST CONFIGURATION (DURATION / ITERATIONS SETTING ) APPROPRIATE?... 4 Request / Response Status Summary...

More information

CLOUD MIGRATION STRATEGIES

CLOUD MIGRATION STRATEGIES CLOUD MIGRATION STRATEGIES Faculty Contributor: Dr. Rahul De Student Contributors: Mayur Agrawal, Sudheender S Abstract This article identifies the common challenges that typical IT managers face while

More information

Windows Admins... & Long-term capacity planning... do the two go together?

Windows Admins... & Long-term capacity planning... do the two go together? Windows Admins... & Long-term capacity planning... do the two go together? Carl Stanfield EMC Australia Capacity management requires a few minutes of your day to crunch some numbers. The results are priceless

More information

Data Warehouse and Business Intelligence Testing: Challenges, Best Practices & the Solution

Data Warehouse and Business Intelligence Testing: Challenges, Best Practices & the Solution Warehouse and Business Intelligence : Challenges, Best Practices & the Solution Prepared by datagaps http://www.datagaps.com http://www.youtube.com/datagaps http://www.twitter.com/datagaps Contact contact@datagaps.com

More information

Copyright www.agileload.com 1

Copyright www.agileload.com 1 Copyright www.agileload.com 1 INTRODUCTION Performance testing is a complex activity where dozens of factors contribute to its success and effective usage of all those factors is necessary to get the accurate

More information

How to Plan a Successful Load Testing Programme for today s websites

How to Plan a Successful Load Testing Programme for today s websites How to Plan a Successful Load Testing Programme for today s websites This guide introduces best practise for load testing to overcome the complexities of today s rich, dynamic websites. It includes 10

More information

Strategies for a Successful E2E Systems Integration Test. Fiona Charles Let s Test May 9, 2012

Strategies for a Successful E2E Systems Integration Test. Fiona Charles Let s Test May 9, 2012 Strategies for a Successful E2E Systems Integration Test Fiona Charles Let s Test May 9, 2012 This session Describes key project management strategies I have used to manage large- scale Systems Integration

More information

Performance Testing Process A Whitepaper

Performance Testing Process A Whitepaper Process A Whitepaper Copyright 2006. Technologies Pvt. Ltd. All Rights Reserved. is a registered trademark of, Inc. All other trademarks are owned by the respective owners. Proprietary Table of Contents

More information

White Paper Performance Testing Methodology

White Paper Performance Testing Methodology White Paper Performance Testing Methodology by Johann du Plessis Introduction One of the main concerns with a performance testing project is how much value the testing adds. Is performance testing worth

More information

There are a number of factors that increase the risk of performance problems in complex computer and software systems, such as e-commerce systems.

There are a number of factors that increase the risk of performance problems in complex computer and software systems, such as e-commerce systems. ASSURING PERFORMANCE IN E-COMMERCE SYSTEMS Dr. John Murphy Abstract Performance Assurance is a methodology that, when applied during the design and development cycle, will greatly increase the chances

More information

Application and Web Load Testing. Datasheet. Plan Create Load Analyse Respond

Application and Web Load Testing. Datasheet. Plan Create Load Analyse Respond Application and Web Load Testing Datasheet Plan Create Load Analyse Respond Product Overview JAR:load is an innovative web load testing solution delivered from the Cloud* for optimising the performance

More information

CRM: The Golden Rules CRM. The Golden Rules. Part of the Waterdale Group of Companies

CRM: The Golden Rules CRM. The Golden Rules. Part of the Waterdale Group of Companies CRM The Golden Rules 2 1 CRM isn t just technology, it s a way of life 2 Who is the customer? Nobody has all the answers when it comes to the perfect customer strategy. But there are some basic principles

More information

Performance Test Summary Report Skills Development Management System. December 2014

Performance Test Summary Report Skills Development Management System. December 2014 Performance Test Summary Report Skills Development Management System December 2014 Performance Test report submitted to National Skill Development Corporation Version Date Name Summary of Changes 1.0 22/12/2014

More information

The Complete Guide to CUSTOM FIELD SERVICE APPLICATIONS

The Complete Guide to CUSTOM FIELD SERVICE APPLICATIONS The Complete Guide to CUSTOM FIELD SERVICE APPLICATIONS Copyright 2014 Published by Art & Logic All rights reserved. Except as permitted under U.S. Copyright Act of 1976, no part of this publication may

More information

Capacity Planning. Capacity Planning Process

Capacity Planning. Capacity Planning Process Process, page 1 Getting Started, page 2 Collected Data Categorization, page 3 Capacity Utilization, page 6 Process Figure 1: Process You should make changes to an existing Unified ICM/Unified CCE deployment

More information

WHITE PAPER. From Downgraded to Dead: Monitoring Application Performance. By Nick Cavalancia

WHITE PAPER. From Downgraded to Dead: Monitoring Application Performance. By Nick Cavalancia From Downgraded to Dead: Monitoring Application Performance By Nick Cavalancia TABLE OF CONTENTS Introduction 1 Why Monitor? 2 What to Monitor? 4 How to Monitor? 6 Keep Applications Performing 8 Your Microsoft

More information

CREATING BUSINESS VALUE THROUGH INTEGRATION

CREATING BUSINESS VALUE THROUGH INTEGRATION CREATING BUSINESS VALUE THROUGH INTEGRATION WHAT BIZTALK SERVER AND SQL SERVER PROVIDE DAVID CHAPPELL DECEMBER 2009 SPONSORED BY MICROSOFT CORPORATION CONTENTS Why Integration Matters... 3 Application

More information

Inside Track Research Note. In association with. Storage Quality of Service Management. The automation imperative

Inside Track Research Note. In association with. Storage Quality of Service Management. The automation imperative Research Note In association with Storage Quality of Service Management The automation imperative May 2015 In a nutshell About this The insights presented in this document are derived from independent

More information

Why Test ITSM Applications for Performance? Webinar

Why Test ITSM Applications for Performance? Webinar Why Test ITSM Applications for Performance? Webinar Agenda What is performance testing? Why test ITSM for performance Testing? What are the ITSM modules that need performance testing? What are the use

More information

ACME Intranet Performance Testing

ACME Intranet Performance Testing ACME Intranet Performance Testing Purpose of this document Prior to the official launch of the ACME Intranet, a set of performance tests were carried out to ensure that the platform could cope with estimated

More information

Deduplication and Beyond: Optimizing Performance for Backup and Recovery

Deduplication and Beyond: Optimizing Performance for Backup and Recovery Beyond: Optimizing Gartner clients using deduplication for backups typically report seven times to 25 times the reductions (7:1 to 25:1) in the size of their data, and sometimes higher than 100:1 for file

More information

The 2014 Bottleneck Report on Enterprise Mobile

The 2014 Bottleneck Report on Enterprise Mobile The 2014 Bottleneck Report on Enterprise Mobile What s the big bottleneck for enterprise mobile app development this year, and how do you get past it? 1 / 32 The 2014 Bottleneck Report on Enterprise Mobile

More information

Response Time Analysis

Response Time Analysis Response Time Analysis A Pragmatic Approach for Tuning and Optimizing SQL Server Performance By Dean Richards Confio Software 4772 Walnut Street, Suite 100 Boulder, CO 80301 866.CONFIO.1 www.confio.com

More information

Performance Testing. What is performance testing? Why is performance testing necessary? Performance Testing Methodology EPM Performance Testing

Performance Testing. What is performance testing? Why is performance testing necessary? Performance Testing Methodology EPM Performance Testing Performance Testing What is performance testing? Why is performance testing necessary? Performance Testing Methodology EPM Performance Testing What is Performance Testing l The primary goal of Performance

More information

How to Select and Implement an ERP System

How to Select and Implement an ERP System How to Select and Implement an ERP System Prepared by 180 Systems Written by Michael Burns 180 Systems WHAT IS ERP?... 3 ANALYSIS... 4 VENDOR SELECTION... 6 VENDOR DEMONSTRATIONS... 8 REFERENCE CALLS...

More information

Performance and Load Testing. >>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<<

Performance and Load Testing. >>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<<<<<<<< Performance and Load Testing Part 1 Performance & Load Testing Basics Performance & Load Testing Basics Introduction to Performance Testing Difference between Performance, Load and Stress Testing Why Performance

More information

Business Success Blueprints Choosing and using a CRM

Business Success Blueprints Choosing and using a CRM Business Success Blueprints Choosing and using a CRM YOUR BLUEPRINT TO SUPER SUCCESS The definitive Entrepreneurs Circle Guide to managing your customer data. Choosing and Using a CRM Still trying to

More information

Windows Server 2008 R2 Hyper-V Live Migration

Windows Server 2008 R2 Hyper-V Live Migration Windows Server 2008 R2 Hyper-V Live Migration Table of Contents Overview of Windows Server 2008 R2 Hyper-V Features... 3 Dynamic VM storage... 3 Enhanced Processor Support... 3 Enhanced Networking Support...

More information

White Paper. Managed IT Services as a Business Solution

White Paper. Managed IT Services as a Business Solution White Paper Managed IT Services as a Business Solution 1 TABLE OF CONTENTS 2 Introduction... 2 3 The Need for Expert IT Management... 3 4 Managed Services Explained... 4 5 Managed Services: Key Benefits...

More information

Application. Performance Testing

Application. Performance Testing Application Performance Testing www.mohandespishegan.com شرکت مهندش پیشگان آزمون افسار یاش Performance Testing March 2015 1 TOC Software performance engineering Performance testing terminology Performance

More information

Sample Exam Foundation Level Syllabus. Mobile Tester

Sample Exam Foundation Level Syllabus. Mobile Tester Sample Exam Foundation Level Syllabus Mobile Tester September 2015 American Software Testing Qualifications Board Sample Exam Foundation Level Syllabus Mobile Tester MOB-1.2.1 (K2) Explain the expectations

More information

Performance Testing of Java Enterprise Systems

Performance Testing of Java Enterprise Systems Performance Testing of Java Enterprise Systems Katerina Antonova, Plamen Koychev Musala Soft Why Performance Testing? Recent studies by leading USA consultancy companies showed that over 80% of large corporations

More information

27 th March 2015 Istanbul, Turkey. Performance Testing Best Practice

27 th March 2015 Istanbul, Turkey. Performance Testing Best Practice 27 th March 2015 Istanbul, Turkey Performance Testing Best Practice Your Host.. Ian Molyneaux Leads the Intechnica performance team More years in IT than I care to remember Author of The Art of Application

More information

Analyzing IBM i Performance Metrics

Analyzing IBM i Performance Metrics WHITE PAPER Analyzing IBM i Performance Metrics The IBM i operating system is very good at supplying system administrators with built-in tools for security, database management, auditing, and journaling.

More information

Monitoring applications in multitier environment. Uroš Majcen uros@quest-slo.com. A New View on Application Management. www.quest.

Monitoring applications in multitier environment. Uroš Majcen uros@quest-slo.com. A New View on Application Management. www.quest. A New View on Application Management www.quest.com/newview Monitoring applications in multitier environment Uroš Majcen uros@quest-slo.com 2008 Quest Software, Inc. ALL RIGHTS RESERVED. Management Challenges

More information

v AN EXECUTIVE GUIDE TO SAAS & THE CLOUD

v AN EXECUTIVE GUIDE TO SAAS & THE CLOUD SaaS (or Software as a Service to give it s full title) has been around for a several years and it s the term used to describe the concept of running business software and data in the cloud. In this document

More information

Best Practices for Web Application Load Testing

Best Practices for Web Application Load Testing Best Practices for Web Application Load Testing This paper presents load testing best practices based on 20 years of work with customers and partners. They will help you make a quick start on the road

More information

Smarter Balanced Assessment Consortium. Recommendation

Smarter Balanced Assessment Consortium. Recommendation Smarter Balanced Assessment Consortium Recommendation Smarter Balanced Quality Assurance Approach Recommendation for the Smarter Balanced Assessment Consortium 20 July 2012 Summary When this document was

More information

OpenLoad - Rapid Performance Optimization Tools & Techniques for CF Developers

OpenLoad - Rapid Performance Optimization Tools & Techniques for CF Developers OpenDemand Systems, Inc. OpenLoad - Rapid Performance Optimization Tools & Techniques for CF Developers Speed Application Development & Improve Performance November 11, 2003 True or False? Exposing common

More information

a talk for ApacheCon Europe 2008 by Jeremy Quinn Break My Site practical stress testing and tuning

a talk for ApacheCon Europe 2008 by Jeremy Quinn Break My Site practical stress testing and tuning a talk for ApacheCon Europe 2008 by Jeremy Quinn Break My Site practical stress testing and tuning photo credit: Môsieur J This is designed as a beginner s talk. I am the beginner. 1 I will present two

More information

SharePoint 2013 Best Practices

SharePoint 2013 Best Practices SharePoint 2013 Best Practices SharePoint 2013 Best Practices When you work as a consultant or as a SharePoint administrator, there are many things that you need to set up to get the best SharePoint performance.

More information

Mobile Application Performance Testing

Mobile Application Performance Testing StarBase Insight Mobile Application Performance Testing Introduction Purpose This discussion paper aims to educate the reader about mobile applications and demystify their performance testing needs by

More information

ERP Implementation - The Traps

ERP Implementation - The Traps The PROJECT PERFECT White Paper Collection ERP Implementation - The Traps Neville Turbit Overview ERP implementations are littered with tales of lost millions and withdrawals after implementation. Many

More information

Accelerate Testing Cycles With Collaborative Performance Testing

Accelerate Testing Cycles With Collaborative Performance Testing Accelerate Testing Cycles With Collaborative Performance Testing Sachin Dhamdhere 2005 Empirix, Inc. Agenda Introduction Tools Don t Collaborate Typical vs. Collaborative Test Execution Some Collaborative

More information

CDC UNIFIED PROCESS JOB AID

CDC UNIFIED PROCESS JOB AID CDC UNIFIED PROCESS JOB AID Independent Verification & Validation Activities Document Purpose This Job Aid is a brief document listing the items to be noted, checked, remembered, and delivered when completing

More information

Agile Power Tools. Author: Damon Poole, Chief Technology Officer

Agile Power Tools. Author: Damon Poole, Chief Technology Officer Agile Power Tools Best Practices of Agile Tool Users Author: Damon Poole, Chief Technology Officer Best Practices of Agile Tool Users You ve decided to transition to Agile development. Everybody has been

More information

Data Driven Success. Comparing Log Analytics Tools: Flowerfire s Sawmill vs. Google Analytics (GA)

Data Driven Success. Comparing Log Analytics Tools: Flowerfire s Sawmill vs. Google Analytics (GA) Data Driven Success Comparing Log Analytics Tools: Flowerfire s Sawmill vs. Google Analytics (GA) In business, data is everything. Regardless of the products or services you sell or the systems you support,

More information

Navigating The Cloud: A Primer For Understanding Cloud Computing. White Paper: 2012

Navigating The Cloud: A Primer For Understanding Cloud Computing. White Paper: 2012 Navigating The Cloud: A Primer For Understanding Cloud Computing White Paper: 2012 Cloud Computing: Modern Solutions for Today s Businesses What Do You See In The Cloud? For most people, looking up at

More information

TRACE PERFORMANCE TESTING APPROACH. Overview. Approach. Flow. Attributes

TRACE PERFORMANCE TESTING APPROACH. Overview. Approach. Flow. Attributes TRACE PERFORMANCE TESTING APPROACH Overview Approach Flow Attributes INTRODUCTION Software Testing Testing is not just finding out the defects. Testing is not just seeing the requirements are satisfied.

More information

Integrating a web application with Siebel CRM system

Integrating a web application with Siebel CRM system Integrating a web application with Siebel CRM system Mika Salminen, Antti Seppälä Helsinki University of Technology, course Business Process Integration: Special Course in Information Systems Integration,

More information

SearchWinIT.com SearchExchange.com SearchSQLServer.com

SearchWinIT.com SearchExchange.com SearchSQLServer.com TechTarget Windows Media SearchWinIT.com SearchExchange.com SearchSQLServer.com SearchEnterpriseDesktop.com SearchWindowsServer.com SearchDomino.com LabMice.net E-Guide Step-by-Step Guide: How to consolidate

More information

Identify and control performance and capacity risks. Introduction... 2

Identify and control performance and capacity risks. Introduction... 2 Application performance testing in VMware environments Identify and control performance and capacity risks Table of contents Introduction... 2 Performance and capacity planning techniques... 2 Rough sizing

More information

A White Paper. Best Practices Guide To Improving The End-User Experience: How to Maximize SLA Performance with Synthetic Web Transaction Monitoring

A White Paper. Best Practices Guide To Improving The End-User Experience: How to Maximize SLA Performance with Synthetic Web Transaction Monitoring A White Paper Best Practices Guide To Improving The End-User Experience: How to Maximize SLA Performance with Synthetic Web Transaction Monitoring Introduction Synthetic transactions, or transaction tests

More information

Cloud Building Blocks: Paving the Road

Cloud Building Blocks: Paving the Road Cloud Building Blocks: Paving the Road By understanding what provides a strong foundation for the cloud, CIOs can help make their deployments smooth and cost-effective. Companies are turning to cloud computing

More information

5 Group Policy Management Capabilities You re Missing

5 Group Policy Management Capabilities You re Missing 5 Group Policy Management Capabilities You re Missing Don Jones 1. 8 0 0. 8 1 3. 6 4 1 5 w w w. s c r i p t l o g i c. c o m / s m b I T 2011 ScriptLogic Corporation ALL RIGHTS RESERVED. ScriptLogic, the

More information

Backup architectures in the modern data center. Author: Edmond van As edmond@competa.com Competa IT b.v.

Backup architectures in the modern data center. Author: Edmond van As edmond@competa.com Competa IT b.v. Backup architectures in the modern data center. Author: Edmond van As edmond@competa.com Competa IT b.v. Existing backup methods Most companies see an explosive growth in the amount of data that they have

More information

PC Power Management FAQ

PC Power Management FAQ PC Power Management FAQ What is Verismic Power Manager? Verismic Power Manager is a powerful web- based PC Power Management Software that enables people to benefit from the carbon and energy savings that

More information

EQUELLA Whitepaper. Performance Testing. Carl Hoffmann Senior Technical Consultant

EQUELLA Whitepaper. Performance Testing. Carl Hoffmann Senior Technical Consultant EQUELLA Whitepaper Performance Testing Carl Hoffmann Senior Technical Consultant Contents 1 EQUELLA Performance Testing 3 1.1 Introduction 3 1.2 Overview of performance testing 3 2 Why do performance testing?

More information

First Step for Your Virtual Desktop Initiative: Conduct a Proof of Concept

First Step for Your Virtual Desktop Initiative: Conduct a Proof of Concept First Step for Your Virtual Desktop Initiative: Conduct a Proof of Concept Is your organization looking to break-away from the old ways of deploying desktops? If so, how do you validate that a new approach

More information

Virtual Desktop Infrastructure Optimization with SysTrack Monitoring Tools and Login VSI Testing Tools

Virtual Desktop Infrastructure Optimization with SysTrack Monitoring Tools and Login VSI Testing Tools A Software White Paper December 2013 Virtual Desktop Infrastructure Optimization with SysTrack Monitoring Tools and Login VSI Testing Tools A Joint White Paper from Login VSI and Software 2 Virtual Desktop

More information

Deploying and Optimizing SQL Server for Virtual Machines

Deploying and Optimizing SQL Server for Virtual Machines Deploying and Optimizing SQL Server for Virtual Machines Deploying and Optimizing SQL Server for Virtual Machines Much has been written over the years regarding best practices for deploying Microsoft SQL

More information

HOW TO GET STARTED WITH DATA MODELING

HOW TO GET STARTED WITH DATA MODELING By Laura Brandenburg, CBAP Lesson Objective: After completing this lesson, you will understand the role of data modeling in the business analysis lifecycle and why it is important for business stakeholders

More information

Selling Windows Azure Projects IT INFRASTRUCTURE

Selling Windows Azure Projects IT INFRASTRUCTURE Selling Windows Azure Projects IT INFRASTRUCTURE A GUIDE FOR MICROSOFT SI PARTNERS Sponsored by Microsoft Corporation 1/ Why Should You Sell Infrastructure Projects that Use Windows Azure? 2/ Why Sell

More information