Developing a Load Testing Strategy
|
|
- Elmer Parker
- 8 years ago
- Views:
Transcription
1 Developing a Load Testing Strategy Michele Ruel St.George Bank CMGA 2005 Page 1
2 Overview... 3 What is load testing?... 4 Scalability Test... 4 Sustainability/Soak Test... 4 Comparison Test... 4 Worst Case... 4 Spike Test/Break Point... 4 Back-end Slowdown Test... 4 Performance Regression/Benchmark Test... 4 Why do we need Load Testing?... 5 How do we start Load Testing?... 6 Where does it fit in the organisation?... 6 Who runs the tests?... 6 Who decides if the test is successful?... 6 Who gathers information?... 6 When do you test?... 7 Software... 7 Hardware... 8 Anatomy of a Load Test... 8 Planning... 8 Reporting Testing Archive And how's it going at St.George Bank? Where we are in designing And how did your first big test go? But we do already have some lessons we've learned Conclusion Page 2
3 Overview The purpose of this paper is to describe how St.George Bank is creating a load test strategy to fit into its corporate framework. It will define load testing, suggest when to do load testing, highlight the considerations around load testing, and describe our experience during our first big load test. Page 3
4 What is load testing? Load testing is a method of applying a (usually synthetic) transaction load to an application system to test various performance and stability aspects of code and architecture. Here is a partial list of types of tests: Scalability Test These ramp-up tests determine the extent to which the application scales before reaching a choking point. This test will give us the bottleneck or limiting factor to performance. This could be CPU, memory or other system resources of the server, frontend or backend throughput limitation, or application limitations (e.g. Queue size, connection max, etc.). Sustainability/Soak Test This is a long running-test under low, medium and high load that determines if the application exhibits problems over an extended period of time (12-48 hours). It will tell us if there are any problems like memory leaks or other resource usage problems. Comparison Test Establish the impact/overhead of the new version of the application over the existing version. This test is done only if there is an existing version of the application. However, we have to make sure that we are comparing apples to apples as newer versions could have a different implementation from the old and/or the behavior of the application could differ. Worst Case This tests behaviour of a worst case scenario of the application under load. All of the parameters are at their maximum values. All other tests should be done with a vanilla scenario. This test could bring to light problems with the application that may have not been caught. This is basically boundary condition testing. Spike Test/Break Point This test helps to view the impact of a spiking load on the application behaviour/performance. It determines if a sudden increase in load causes any functional/performance problems. Back-end Slowdown Test Determine the impact of a back-end slowdown on the application performance. The back end server is artificially limited. These can be short ramp-up tests or long running tests. This test could bring out code defects that would not show up under normal conditions. Performance Regression/Benchmark Test This is particularly important for critical scenarios of the application that should be run as a performance regression mix to see the impact on overall performance and capacity. This test will tell us the overhead of the new release over the old one. Also this test could be run when doing optimization or tuning. Page 4
5 Why do we need Load Testing? Load testing done well will greatly improve the end product of your applications. If it is incorporated into the development framework, users will see a marked improvement in the systems that they work on every day. Those errors and system disasters that are always feared as an application moves to productions will no longer show up, because production loads and hardware will have been provend. Often organisations come to load testing after a very painful implementation occurred or has witnessed some other organisation s very painful implementation. Incorporating load testing is a lot of work to undertake to be sure of your code and environment, but consider the time it takes to constantly fix and explain to your customers why their application is so slow or unstable. The next section has a list of times when this type of automated testing would be userful. Page 5
6 How do we start Load Testing? Where does it fit in the organisation? Here we are talking about management of load testing activities. This could be the most important decision that you make. It will greatly influence the success of the testing. Think about the structure of your organisation and the politics that are involved. Most types of testing take place within the development area of the IT hierarchy, but most studies of this placement point to problems with conflicting goals. The project s success is measured by the on time, under budget rule. This goal usually limits the amount of testing allowed and skews the results of any testing done to positive. When the application then goes to production, all the load- and stress-related problems are solved there, with attendant bad experience for the users. The alternative is to place this function in an area that is separated from the development management, for instance, in production operations, the testing organisation, or capacity planning. If this is done, testing can be more objective and, if major performance or stabiliy problems are encountered, production implementation delay can be debated rather than allowed to continue even to meet budget or dates. Who runs the tests? Development Development could be a good choice because they have the best access to application architecture and can easily create the scripts that run the load tests. This way, load testing procedures can be developed as the application is developed and used for unit and system testing. The testing organisation This would seem to be the perfect area for load testing management, unless they are report to development. They will understand the importance of different types of tests and will have a methodology to structure and track their testing. Who decides if the test is successful? This is probably a committee decision. There should be a group composed of interested parties. project management, production support, capacity planning, and the testing organisation. This will be discussed later under exit criteria. Who gathers information? Once again, you must look to your organisation chart for the answer to this question. The type of data that is needed is business usage metrics, and then they must be translated to system load. Places in the typical IT organisation where people are used to dealing with these questions are capacity planning and performance analysis. Your production support organisation may also have the business contacts and the experience to translate these numbers to system load. Page 6
7 When do you test? During development We have developed a Risk Matrix that helps us to estimate the time needed for load testing and the types of load tests to run. If we know a particular bit of the system is prone to cause performance problems, or we are doing something like running an unpredictable query on our mainframe, we will plan on using these synthetic transactions to drive development tests. Before implementation This is the traditional time to do load testing. Our plan is to do several iterations and tests in a preliminary phase to shake out the scripts and determine code readiness for the final test. The next step would be to do a final load test at the end of customer acceptance testing. This test will have a go-no go decision on implementation; the passing (exit) criteria are set up at the beginning of load test planning; a group of concerned people from different areas will make this decision, so that development s bias for implementing and fixing in production will not be a factor. Before peak periods After the development testing requirements are finished, you can then move on to planning load tests to run before each important systems peak times, for instance, before the Christmas peak, to be sure that it will work smoothly during these times. At specific intervals For some systems that are fairly stable in terms of not much development going on, it might be useful to just schedule a yearly test to make sure that they re not about to break under increased load or different usage. This is also useful in that the planning of the test will ensure at least yearly contact with the business to see if there are any changes in usage metrics planned. Software There are a number of products that are available to assist you in running a load test; it is outside the charter of this paper to discuss specific software. Suffice to say that these products are of a level of sophistication now that they can not only control the injection of load into the system, but also collect system metrics during the run and collate the response times of each of the transactions. The software goes through a learning phase in which screen movements and choices are captured. This is then translated into a script that should be modified to vary input and transaction choices. If the software you choose doesn t have built-in metric measurement, then you must consider how you are going to monitor the test systems for load; perhaps your production organisation has tools that you can use in test. When you re choosing software, you must consider what types of systems you will be testing: web-based, mainframe-based, or other platforms. You will probably have to buy a certain number of licences for virtual users, so make sure you ll have enough to apply the peak load to the systems. Software varies widely in cost, so that will also be a big factor in your decision. Page 7
8 Hardware Hardware, ah hardware this is pivot on which all load testing turns. If your test system hardware is inadequate, or unstable, you will not be able to trust your test numbers. There are two main test hardware configurations to consider: A system exactly like the production system, duplicating network and system hardware and software, using production-size databases A test system that can easily be extrapolated to the production size, for instance, a complete system at half or a quarter the size of production. Keep in mind that the further away from production configuration the system gets, the harder to draw conclusions it will be. For this you might consider capacity planning modelling software to compare the test to the production performance. Be creative about where you get hardware from; perhaps you have older hardware that you can borrow to set up the system; if you re implementing a new system on new hardware, ask to bring in the production hardware earlier and test on it. Just keep in mind that this hardware will have to be readied for production, so build that into your schedule. Anatomy of a Load Test Planning Here are the minimal topics that you must think about before you run a load test: Each one is discussed in the following sections. Business metrics and translation to application load Software Hardware Test strategy: o What to test o What types of tests to run Entry criteria (what is needed before you start the formal testing) Exit criteria (how to know the load testing is over) Reporting Test archive The hard part: business metrics Usage Modelling is a the estimation of an application s production load, in terms of user volumes and transaction rates. These numbers are required to ensure load test scenarios apply a realistic work mix, running at realistic transaction rates. In addition, Exit Criteria are specified at loads determined by Usage Modelling this ensures that the response times we measure as a result can be compared against those specified in our Exit Criteria. Page 8
9 Capacity Planning is the logical organisation to take ownership of this phase and responsibility for its delivery to both load testing and other appropriate customer. They are the people who will probably have the data from the production systems, and they will also probably have the contacts in the business organisation. Who do you get the metrics from? Well, the business, of course? No? The customers should have these numbers. They should have been part of the justification of the project, shouldn t they? The answer here is get the best figures you can from the best people available. Start with the project sponsors. Find out how they justified the project. If this is a replacement application, then someone probably has these statistics somewhere. Maybe capacity planning has them, maybe production support has them, and maybe there are some key customers who know them. In the end, though, you may have to just make an educated guess. Remember to document the process and the outcomes. Give each metric a confidence value, so you can refine how you estimate in the future, and keep notes on where you got them. How do you verify them? Huh? Verify them? I had enough trouble getting them together. This has to be part of this fact-finding process; it s you chance to be a detective. Apply reasonability tests to the final figures. If it s an internal application, and the number of users is more than the number of employees, maybe there s something wrong! Really, dig a little further; ask another person or different questions; think about the relationship of the transaction numbers and evaluate if they seem reasonable. And, most important of all, have your customers, development, and capacity planning sign off on them. What are the minimal metrics needed? Seasonal peak date(s) and usage This is usually a percentage over average use, and maybe there are several peaks, so make sure you have all of them. There may be peaks with different usage criteria. Daily peak time(s) and usage Number of users Batch job schedule and usage System interfaces This is important from the user standpoint and also from the application designer standpoint. It will give you early warning of the extent of your test environment and the scope of your test. Transaction numbers for all the important transactions Response time expectation. It may be necessary to break this down by transaction type, like logon, and short medium, and long transactions. (Since customers only experience response times as comparative, it might be useful to have the technical people come up with standard response times by type of application, and then those numbers can be the basis for that discussion.) If you have a current system, this is easier. Page 9
10 Software Check to see you have the correct types of user platforms for this project and enough virtual users for your peaks. If you don t have enough users, plan on purchasing more or devise a strategy to use the ones you have in some creative way. Hardware Plan the hardware you need; start early to be sure that it s available when you need it and be adamant about the time you need to exclusively use it. Make sure that you have the technical people available to set it up and test it before you plan to do the formal test. Test Planning What to test Each project is different. The place to start would be assessing the risks in the project. Make sure you test the really risky pieces of the application, which would be new architecture, new programs, or any new or complicated system interfaces. After that, look at the high volume or resource-intensive transactions, and then everything else. Make sure to prioritise your tests. You probably won t have time for all of them. What types of tests to do Once again, this should be a product of your risk assessment. If you re concerned about memory leaks in a windows web server environment, for instance, you would do soak tests. If this is a customer-facing web application, you ll want to do extensive break and peak load testing Entry Criteria Entry Criteria are the conditions needed to start the formal testing. There is a standard set that you will probably use is: Stable code Stable hardware Data availability and quality Completion of metric collection This is an area where you have to be firm. You will probably be pushed to start without stable code and hardware, and you may ease into what other people think is formal load testing by helping with some functional testing. Make a point of reminding people that you are not in formal load testing if you haven t had these criteria met. It would be a good idea to establish this with each test team early in the project; get signoff on these criteria from all interested parties business, production, development, and capacity planning. Exit Criteria Formal load testing passes or fails depending on whether or not Exit (or Success) Criteria are satisfied. It is therefore crucial to seek agreement before commencing load testing. Page 10
11 Typical Exit Criteria specifications include: Acceptable response times slowest acceptable response times (expressed as 90 th percentile figures) for a range of transactions at peak load Acceptable server resource limits typically CPU limits at peak load. Exit Criteria Specification requires: Completion of the Metrics exercise a clear picture of Peak Load is required Stakeholder involvement with participation from Business, Project, Production support and load testing. Refinement with successive project experience. Schedule Load testing must be planned into the project schedule, so it is necessary to have a tool that will allow the project planners to gauge how long load testing will take. We have developed a risk matrix tool that will allow the planners to estimate how long testing will take. It takes into account various elements of risk that should be considered in planning a project schedule. This is still in its formative stages, and we expect there will be much more refinement as we do more load tests. Page 11
12 Reporting Reporting should be done daily to assess progress. Reports should go to all of the load test committee. At end of the load test schedule, a full report should be sent to all of the concerned project participants detailing the tests done and their outcomes. If the exit committee decides that the project has failed testing, then detailed reasons for the failure must be given as well as the tests which have to be repeated to pass load testing. Testing Archive Consideration should be given as to what data should be saved during the tests and beyond. If this information is saved, it can be used for regression testing and release load testing. Scripts While it will probably be necessary to regenerate the scripts for each new round of load testing, notes should be kept on adjustments and procedures used to accomplish various script items, such as how unique new records were generated, and how problems were solved Reports. Reports and data from major tests should be saved so they can be compared with the next round of test. And how's it going at St.George Bank? Where we are in designing Our processes keep changing, so we're aiming at a moving target. We re also trying to keep up with organisation changes, so we remain flexible. We have created a risk matrix tool for the estimators and are working with our risk planners to incorporate all of our concerns into one tool. And how did your first big test go? We are still immersed in a large CRM implementation. Our load testing core workgroup consisted of : One experienced load test planner One or two experienced load test scripter(s) One capacity planner One or two vendor software experts Production support people for our platforms, the network, and web operations. One problem we had with personnel was that because this project was so long, contract people kept changing and personal plans took people away from the team at crucial times. We have been testing for three months, first on the production platform, and then moving to a staging platform. This also turned out to be quite a problem Page 12
13 because we moved from the production platform to start the pilot. The stagining platform had not been used before and took forever to configure because all of our experts were off fixing pilot problems. But we do already have some lessons we've learned Business metrics: do them early, do them often and VERIFY them Try to insure that LT is separate from development Be tough about your entry and exit criteria Make sure technical experts are available to fix the hardware and application Check with your technical experts to see what they use as monitoring tools. Conclusion Load Testing can be of great value to an organisation in terms of guaranteed implementation success, stability and performance of an application. As expected in this type of outcome, this guarantee comes only from rigorous adherence to certain requirements: entry and exit criteria, and well-designed and executed tests. Page 13
Application Performance Testing Basics
Application Performance Testing Basics ABSTRACT Todays the web is playing a critical role in all the business domains such as entertainment, finance, healthcare etc. It is much important to ensure hassle-free
More informationPerformance Testing for Managers. Presented by Stuart Moncrieff at SIGiST Melbourne on June 15 th, 2011
Performance Testing for Managers Presented by Stuart Moncrieff at SIGiST Melbourne on June 15 th, 2011 What will be covered? Performance Testing as it applies to: Large multi-user enterprise IT applications
More informationPerformance Testing Percy Pari Salas
Performance Testing Percy Pari Salas Presented by : Percy Pari Salas Agenda What is performance testing? Types of performance testing What does performance testing measure? Where does performance testing
More informationRecommendations for Performance Benchmarking
Recommendations for Performance Benchmarking Shikhar Puri Abstract Performance benchmarking of applications is increasingly becoming essential before deployment. This paper covers recommendations and best
More informationBringing Value to the Organization with Performance Testing
Bringing Value to the Organization with Performance Testing Michael Lawler NueVista Group 1 Today s Agenda Explore the benefits of a properly performed performance test Understand the basic elements of
More informationSOFTWARE PERFORMANCE TESTING SERVICE
SOFTWARE PERFORMANCE TESTING SERVICE Service Definition GTS s performance testing services allows customers to reduce the risk of poor application performance. This is done by performance testing applications
More informationHow To Test For Performance
: Roles, Activities, and QA Inclusion Michael Lawler NueVista Group 1 Today s Agenda Outline the components of a performance test and considerations Discuss various roles, tasks, and activities Review
More informationINTEGRATION SOFTWARE: BUILD OR BUY?
INTEGRATION SOFTWARE: BUILD OR BUY? DAVID CHAPPELL MARCH 2010 SPONSORED BY MICROSOFT CORPORATION No modern organization uses only one application. A single software solution just isn t enough to address
More information10 Best Practices for Application Performance Testing
Business white paper 10 Best Practices for Application Performance Testing Leveraging Agile Performance Testing for Web and Mobile Applications 10 Best Practices for Application Performance Testing Table
More informationResource Monitoring During Performance Testing. Experience Report by Johann du Plessis. Introduction. Planning for Monitoring
Resource Monitoring During Performance Testing Experience Report by Johann du Plessis Introduction During a recent review of performance testing projects I completed over the past 8 years, one of the goals
More informationLoad Testing and Monitoring Web Applications in a Windows Environment
OpenDemand Systems, Inc. Load Testing and Monitoring Web Applications in a Windows Environment Introduction An often overlooked step in the development and deployment of Web applications on the Windows
More informationRapid Bottleneck Identification
Rapid Bottleneck Identification TM A Better Way to Load Test WHITEPAPER You re getting ready to launch or upgrade a critical Web application. Quality is crucial, but time is short. How can you make the
More informationRapid Bottleneck Identification A Better Way to do Load Testing. An Oracle White Paper June 2009
Rapid Bottleneck Identification A Better Way to do Load Testing An Oracle White Paper June 2009 Rapid Bottleneck Identification A Better Way to do Load Testing. RBI combines a comprehensive understanding
More informationWhite paper: Unlocking the potential of load testing to maximise ROI and reduce risk.
White paper: Unlocking the potential of load testing to maximise ROI and reduce risk. Executive Summary Load testing can be used in a range of business scenarios to deliver numerous benefits. At its core,
More informationPerformance Workload Design
Performance Workload Design The goal of this paper is to show the basic principles involved in designing a workload for performance and scalability testing. We will understand how to achieve these principles
More informationTest Run Analysis Interpretation (AI) Made Easy with OpenLoad
Test Run Analysis Interpretation (AI) Made Easy with OpenLoad OpenDemand Systems, Inc. Abstract / Executive Summary As Web applications and services become more complex, it becomes increasingly difficult
More informationBernie Velivis President, Performax Inc
Performax provides software load testing and performance engineering services to help our clients build, market, and deploy highly scalable applications. Bernie Velivis President, Performax Inc Load ing
More informationDocumentation for data centre migrations
Documentation for data centre migrations Data centre migrations are part of the normal life cycle of a typical enterprise. As organisations expand, many reach a point where maintaining multiple, distributed
More informationWindows Admins... & Long-term capacity planning... do the two go together?
Windows Admins... & Long-term capacity planning... do the two go together? Carl Stanfield EMC Australia Capacity management requires a few minutes of your day to crunch some numbers. The results are priceless
More informationAn Oracle White Paper February 2010. Rapid Bottleneck Identification - A Better Way to do Load Testing
An Oracle White Paper February 2010 Rapid Bottleneck Identification - A Better Way to do Load Testing Introduction You re ready to launch a critical Web application. Ensuring good application performance
More informationData Warehouse and Business Intelligence Testing: Challenges, Best Practices & the Solution
Warehouse and Business Intelligence : Challenges, Best Practices & the Solution Prepared by datagaps http://www.datagaps.com http://www.youtube.com/datagaps http://www.twitter.com/datagaps Contact contact@datagaps.com
More informationGuideline for stresstest Page 1 of 6. Stress test
Guideline for stresstest Page 1 of 6 Stress test Objective: Show unacceptable problems with high parallel load. Crash, wrong processing, slow processing. Test Procedure: Run test cases with maximum number
More informationPerform-Tools. Powering your performance
Perform-Tools Powering your performance Perform-Tools With Perform-Tools, optimizing Microsoft Dynamics products on a SQL Server platform never was this easy. They are a fully tested and supported set
More informationCRM: The Golden Rules CRM. The Golden Rules. Part of the Waterdale Group of Companies
CRM The Golden Rules 2 1 CRM isn t just technology, it s a way of life 2 Who is the customer? Nobody has all the answers when it comes to the perfect customer strategy. But there are some basic principles
More information(Refer Slide Time: 01:52)
Software Engineering Prof. N. L. Sarda Computer Science & Engineering Indian Institute of Technology, Bombay Lecture - 2 Introduction to Software Engineering Challenges, Process Models etc (Part 2) This
More informationThroughput Capacity Planning and Application Saturation
Throughput Capacity Planning and Application Saturation Alfred J. Barchi ajb@ajbinc.net http://www.ajbinc.net/ Introduction Applications have a tendency to be used more heavily by users over time, as the
More informationWHAT WE NEED TO START THE PERFORMANCE TESTING?
ABSTRACT Crystal clear requirements before starting an activity are always helpful in achieving the desired goals. Achieving desired results are quite difficult when there is vague or incomplete information
More informationPerformance Testing. What is performance testing? Why is performance testing necessary? Performance Testing Methodology EPM Performance Testing
Performance Testing What is performance testing? Why is performance testing necessary? Performance Testing Methodology EPM Performance Testing What is Performance Testing l The primary goal of Performance
More informationSolovatSoft. Load and Performance Test Plan Sample. Title: [include project s release name] Version: Date: SolovatSoft Page 1 of 13
SolovatSoft Load and Performance Test Plan Sample Title: [include project s release name] Version: Date: SolovatSoft Page 1 of 13 Approval signatures Project Manager Development QA Product Development
More informationAchieving business benefits through automated software testing. By Dr. Mike Bartley, Founder and CEO, TVS (mike@testandverification.
Achieving business benefits through automated software testing By Dr. Mike Bartley, Founder and CEO, TVS (mike@testandverification.com) 1 Introduction During my experience of test automation I have seen
More informationInside Track Research Note. In association with. Storage Quality of Service Management. The automation imperative
Research Note In association with Storage Quality of Service Management The automation imperative May 2015 In a nutshell About this The insights presented in this document are derived from independent
More informationEssential. Guide TESTING YOUR DISASTER RECOVERY ABILITY. The. February 2006. By Paul Robichaux
Essential The February 2006 to Guide TESTING YOUR DISASTER RECOVERY ABILITY By Paul Robichaux It doesn t matter how good your disaster recovery procedures and processes are if you don t test them. This
More informationDevelopment, Acquisition, Implementation, and Maintenance of Application Systems
Development, Acquisition, Implementation, and Maintenance of Application Systems Part of a series of notes to help Centers review their own Center internal management processes from the point of view of
More informationWHITEPAPER: Harnessing intelligence from your customers with an integrated CRM SYSTEM
Harnessing intelligence from your customers with an integrated CRM SYSTEM Hooking the big fish in times of economic uncertainty Once upon a time, businesses used to be able to rely on long term relationships
More informationPerformance Testing of a Large Wealth Management Product
Performance Testing of a Large Wealth Management Product Meherphani Nori & Global Head Quality Assurance Krishna Kankipati & Vice President Mohan Pujari & Product Specialist Broadridge Financial Solutions
More informationCase Study - I. Industry: Social Networking Website Technology : J2EE AJAX, Spring, MySQL, Weblogic, Windows Server 2008.
Case Study - I Industry: Social Networking Website Technology : J2EE AJAX, Spring, MySQL, Weblogic, Windows Server 2008 Challenges The scalability of the database servers to execute batch processes under
More informationPerformance Testing. on Production System
Performance Testing on Production System Abstract Performance testing is conducted to check whether the target application will be able to meet the real users expectations in the production environment
More informationISTQB Certified Tester. Foundation Level. Sample Exam 1
ISTQB Certified Tester Foundation Level Version 2015 American Copyright Notice This document may be copied in its entirety, or extracts made, if the source is acknowledged. #1 When test cases are designed
More informationWhy You Need Email Archiving
Why You Need Email Archiving www.exclaimer.com Table of Contents Introduction...2 The IT Administrator...3 The Email User...5 The Team Leader...6 The Senior Manager/Business Owner...7 Conclusion...8-1
More informationWhite Paper Performance Testing Methodology
White Paper Performance Testing Methodology by Johann du Plessis Introduction One of the main concerns with a performance testing project is how much value the testing adds. Is performance testing worth
More informationACME Intranet Performance Testing
ACME Intranet Performance Testing Purpose of this document Prior to the official launch of the ACME Intranet, a set of performance tests were carried out to ensure that the platform could cope with estimated
More informationCloud Computing Characteristics Are Key
White Paper by Christopher Olive Chief Architect www.gpworldwide.com 2011 What is cloud computing? Cloud computing remains the buzzword winner of the current technology paradigm. The National Institute
More informationHow to Plan a Successful Load Testing Programme for today s websites
How to Plan a Successful Load Testing Programme for today s websites This guide introduces best practise for load testing to overcome the complexities of today s rich, dynamic websites. It includes 10
More informationApplication. Performance Testing
Application Performance Testing www.mohandespishegan.com شرکت مهندش پیشگان آزمون افسار یاش Performance Testing March 2015 1 TOC Software performance engineering Performance testing terminology Performance
More informationResponse Time Analysis
Response Time Analysis A Pragmatic Approach for Tuning and Optimizing SQL Server Performance By Dean Richards Confio Software 4772 Walnut Street, Suite 100 Boulder, CO 80301 866.CONFIO.1 www.confio.com
More informationHow To Test On The Dsms Application
Performance Test Summary Report Skills Development Management System December 2014 Performance Test report submitted to National Skill Development Corporation Version Date Name Summary of Changes 1.0 22/12/2014
More informationA Guide To Evaluating a Bug Tracking System
A Guide To Evaluating a Bug Tracking System White Paper By Stephen Blair, MetaQuest Software Published: October, 2004 Abstract Evaluating a bug tracking system requires that you understand how specific
More informationHow to Select and Implement an ERP System
How to Select and Implement an ERP System Prepared by 180 Systems Written by Michael Burns 180 Systems WHAT IS ERP?... 3 ANALYSIS... 4 VENDOR SELECTION... 6 VENDOR DEMONSTRATIONS... 8 REFERENCE CALLS...
More informationSharePoint 2013 Best Practices
SharePoint 2013 Best Practices SharePoint 2013 Best Practices When you work as a consultant or as a SharePoint administrator, there are many things that you need to set up to get the best SharePoint performance.
More information730 Yale Avenue Swarthmore, PA 19081 www.raabassociatesinc.com info@raabassociatesinc.com
Lead Scoring: Five Steps to Getting Started 730 Yale Avenue Swarthmore, PA 19081 www.raabassociatesinc.com info@raabassociatesinc.com Introduction Lead scoring applies mathematical formulas to rank potential
More informationA Comparison of Oracle Performance on Physical and VMware Servers
A Comparison of Oracle Performance on Physical and VMware Servers By Confio Software Confio Software 4772 Walnut Street, Suite 100 Boulder, CO 80301 303-938-8282 www.confio.com Comparison of Physical and
More informationTEN TIPS FOR A SUCCESSFUL INFOR IMPLEMENTATION
TEN TIPS FOR A SUCCESSFUL INFOR IMPLEMENTATION Copyright 2015 Panorama Consulting Solutions. All Rights Reserved. 720.515.1377 Panorama- Consulting.com Successfully implementing an Infor ERP system involves
More informationPerformance Test Process
A white Success The performance testing helped the client identify and resolve performance bottlenecks which otherwise crippled the business. The ability to support 500 concurrent users was a performance
More informationNavigating The Cloud: A Primer For Understanding Cloud Computing. White Paper: 2012
Navigating The Cloud: A Primer For Understanding Cloud Computing White Paper: 2012 Cloud Computing: Modern Solutions for Today s Businesses What Do You See In The Cloud? For most people, looking up at
More informationA Talk ForApacheCon Europe 2008
a talk for ApacheCon Europe 2008 by Jeremy Quinn Break My Site practical stress testing and tuning photo credit: Môsieur J This is designed as a beginner s talk. I am the beginner. 1 I will present two
More informationDeduplication and Beyond: Optimizing Performance for Backup and Recovery
Beyond: Optimizing Gartner clients using deduplication for backups typically report seven times to 25 times the reductions (7:1 to 25:1) in the size of their data, and sometimes higher than 100:1 for file
More informationPC Power Management FAQ
PC Power Management FAQ What is Verismic Power Manager? Verismic Power Manager is a powerful web- based PC Power Management Software that enables people to benefit from the carbon and energy savings that
More informationWHAT IS SOFTWARE PERFORMANCE ENGINEERING? By Michael Foster www.cmgaus.org
WHAT IS SOFTWARE PERFORMANCE ENGINEERING? By Michael Foster www.cmgaus.org DEFINITION Software Engineering is: A systematic and quantitative approach for the cost effective development of software systems
More informationIntegrating a web application with Siebel CRM system
Integrating a web application with Siebel CRM system Mika Salminen, Antti Seppälä Helsinki University of Technology, course Business Process Integration: Special Course in Information Systems Integration,
More informationIdentify and control performance and capacity risks. Introduction... 2
Application performance testing in VMware environments Identify and control performance and capacity risks Table of contents Introduction... 2 Performance and capacity planning techniques... 2 Rough sizing
More informationThe 2014 Bottleneck Report on Enterprise Mobile
The 2014 Bottleneck Report on Enterprise Mobile What s the big bottleneck for enterprise mobile app development this year, and how do you get past it? 1 / 32 The 2014 Bottleneck Report on Enterprise Mobile
More informationCLOUD MIGRATION STRATEGIES
CLOUD MIGRATION STRATEGIES Faculty Contributor: Dr. Rahul De Student Contributors: Mayur Agrawal, Sudheender S Abstract This article identifies the common challenges that typical IT managers face while
More informationBusiness Success Blueprints Choosing and using a CRM
Business Success Blueprints Choosing and using a CRM YOUR BLUEPRINT TO SUPER SUCCESS The definitive Entrepreneurs Circle Guide to managing your customer data. Choosing and Using a CRM Still trying to
More informationTable of Contents INTRODUCTION... 3. Prerequisites... 3 Audience... 3 Report Metrics... 3
Table of Contents INTRODUCTION... 3 Prerequisites... 3 Audience... 3 Report Metrics... 3 IS MY TEST CONFIGURATION (DURATION / ITERATIONS SETTING ) APPROPRIATE?... 4 Request / Response Status Summary...
More informationAnalyzing IBM i Performance Metrics
WHITE PAPER Analyzing IBM i Performance Metrics The IBM i operating system is very good at supplying system administrators with built-in tools for security, database management, auditing, and journaling.
More informationCopyright www.agileload.com 1
Copyright www.agileload.com 1 INTRODUCTION Performance testing is a complex activity where dozens of factors contribute to its success and effective usage of all those factors is necessary to get the accurate
More informationApplication and Web Load Testing. Datasheet. Plan Create Load Analyse Respond
Application and Web Load Testing Datasheet Plan Create Load Analyse Respond Product Overview JAR:load is an innovative web load testing solution delivered from the Cloud* for optimising the performance
More informationPerformance Testing Process A Whitepaper
Process A Whitepaper Copyright 2006. Technologies Pvt. Ltd. All Rights Reserved. is a registered trademark of, Inc. All other trademarks are owned by the respective owners. Proprietary Table of Contents
More informationR a p i d I m p l e m e n tat i o n ARIES ARIES METHODOLOGY OVERVIEW
Architecture for R a p i d I m p l e m e n tat i o n ARIES of Enterprise Systems ARIES METHODOLOGY OVERVIEW ARIES is a methodology for rapidly implementing complex large-scale enterprise software systems,
More informationWhy Test ITSM Applications for Performance? Webinar
Why Test ITSM Applications for Performance? Webinar Agenda What is performance testing? Why test ITSM for performance Testing? What are the ITSM modules that need performance testing? What are the use
More informationPerformance Testing of Java Enterprise Systems
Performance Testing of Java Enterprise Systems Katerina Antonova, Plamen Koychev Musala Soft Why Performance Testing? Recent studies by leading USA consultancy companies showed that over 80% of large corporations
More informationTESTING AND OPTIMIZING WEB APPLICATION S PERFORMANCE AQA CASE STUDY
TESTING AND OPTIMIZING WEB APPLICATION S PERFORMANCE AQA CASE STUDY 2 Intro to Load Testing Copyright 2009 TEST4LOAD Software Load Test Experts What is Load Testing? Load testing generally refers to the
More informationThere are a number of factors that increase the risk of performance problems in complex computer and software systems, such as e-commerce systems.
ASSURING PERFORMANCE IN E-COMMERCE SYSTEMS Dr. John Murphy Abstract Performance Assurance is a methodology that, when applied during the design and development cycle, will greatly increase the chances
More information27 th March 2015 Istanbul, Turkey. Performance Testing Best Practice
27 th March 2015 Istanbul, Turkey Performance Testing Best Practice Your Host.. Ian Molyneaux Leads the Intechnica performance team More years in IT than I care to remember Author of The Art of Application
More informationCDC UNIFIED PROCESS JOB AID
CDC UNIFIED PROCESS JOB AID Independent Verification & Validation Activities Document Purpose This Job Aid is a brief document listing the items to be noted, checked, remembered, and delivered when completing
More informationAutomation can dramatically increase product quality, leading to lower field service, product support and
QA Automation for Testing Medical Device Software Benefits, Myths and Requirements Automation can dramatically increase product quality, leading to lower field service, product support and liability cost.
More informationOpenLoad - Rapid Performance Optimization Tools & Techniques for CF Developers
OpenDemand Systems, Inc. OpenLoad - Rapid Performance Optimization Tools & Techniques for CF Developers Speed Application Development & Improve Performance November 11, 2003 True or False? Exposing common
More informationWhite Paper. Managed IT Services as a Business Solution
White Paper Managed IT Services as a Business Solution 1 TABLE OF CONTENTS 2 Introduction... 2 3 The Need for Expert IT Management... 3 4 Managed Services Explained... 4 5 Managed Services: Key Benefits...
More informationERP Implementation - The Traps
The PROJECT PERFECT White Paper Collection ERP Implementation - The Traps Neville Turbit Overview ERP implementations are littered with tales of lost millions and withdrawals after implementation. Many
More informationSmarter Balanced Assessment Consortium. Recommendation
Smarter Balanced Assessment Consortium Recommendation Smarter Balanced Quality Assurance Approach Recommendation for the Smarter Balanced Assessment Consortium 20 July 2012 Summary When this document was
More informationNetwork Monitoring with Xian Network Manager
Network Monitoring with Xian Network Manager Did you ever got caught by surprise because of a network problem and had downtime as a result? What about monitoring your network? Network downtime or network
More informationMonitoring Application Performance by Design
From Downgraded to Dead: Monitoring Application Performance By Nick Cavalancia TABLE OF CONTENTS Introduction 1 Why Monitor? 2 What to Monitor? 4 How to Monitor? 6 Keep Applications Performing 8 Your Microsoft
More informationCapacity Planning. Capacity Planning Process
Process, page 1 Getting Started, page 2 Collected Data Categorization, page 3 Capacity Utilization, page 6 Process Figure 1: Process You should make changes to an existing Unified ICM/Unified CCE deployment
More informationWhat can the. SaaS Whitepaper. Cloud do for You?
What can the SaaS Whitepaper Cloud do for You? Content Introduction 1 A Quick Look at Benefits 2 Evaluating the Costs 3 Evaluating the Security 4 Evaluating the People Behind the Cloud 5 1 Introduction
More informationAccelerate Testing Cycles With Collaborative Performance Testing
Accelerate Testing Cycles With Collaborative Performance Testing Sachin Dhamdhere 2005 Empirix, Inc. Agenda Introduction Tools Don t Collaborate Typical vs. Collaborative Test Execution Some Collaborative
More informationPerformance Testing. Slow data transfer rate may be inherent in hardware but can also result from software-related problems, such as:
Performance Testing Definition: Performance Testing Performance testing is the process of determining the speed or effectiveness of a computer, network, software program or device. This process can involve
More informationHow To Understand The Benefits Of Cloud Computing
SaaS (or Software as a Service to give it s full title) has been around for a several years and it s the term used to describe the concept of running business software and data in the cloud. In this document
More informationMobile Application Performance Testing
StarBase Insight Mobile Application Performance Testing Introduction Purpose This discussion paper aims to educate the reader about mobile applications and demystify their performance testing needs by
More informationVirtual Desktop Infrastructure Optimization with SysTrack Monitoring Tools and Login VSI Testing Tools
A Software White Paper December 2013 Virtual Desktop Infrastructure Optimization with SysTrack Monitoring Tools and Login VSI Testing Tools A Joint White Paper from Login VSI and Software 2 Virtual Desktop
More informationBest Practices for Web Application Load Testing
Best Practices for Web Application Load Testing This paper presents load testing best practices based on 20 years of work with customers and partners. They will help you make a quick start on the road
More informationHow To Test For Elulla
EQUELLA Whitepaper Performance Testing Carl Hoffmann Senior Technical Consultant Contents 1 EQUELLA Performance Testing 3 1.1 Introduction 3 1.2 Overview of performance testing 3 2 Why do performance testing?
More informationA Comparison of Oracle Performance on Physical and VMware Servers
A Comparison of Oracle Performance on Physical and VMware Servers By Confio Software Confio Software 4772 Walnut Street, Suite 100 Boulder, CO 80301 www.confio.com Introduction Of all the tier one applications
More informationSelling Windows Azure Projects IT INFRASTRUCTURE
Selling Windows Azure Projects IT INFRASTRUCTURE A GUIDE FOR MICROSOFT SI PARTNERS Sponsored by Microsoft Corporation 1/ Why Should You Sell Infrastructure Projects that Use Windows Azure? 2/ Why Sell
More informationSearchWinIT.com SearchExchange.com SearchSQLServer.com
TechTarget Windows Media SearchWinIT.com SearchExchange.com SearchSQLServer.com SearchEnterpriseDesktop.com SearchWindowsServer.com SearchDomino.com LabMice.net E-Guide Step-by-Step Guide: How to consolidate
More informationHow To Test A Web Server
Performance and Load Testing Part 1 Performance & Load Testing Basics Performance & Load Testing Basics Introduction to Performance Testing Difference between Performance, Load and Stress Testing Why Performance
More informationIncreasing Business Efficiency and Agility for ATGbased. Systems. the business challenge: upgrading the development pipeline
Increasing Business Efficiency and Agility for ATGbased ecommerce Systems This case study follows a Tier 1 retailer migrating to an ATG-based ecommerce platform and upgrading its software development process
More informationCase Study: Load Testing and Tuning to Improve SharePoint Website Performance
Case Study: Load Testing and Tuning to Improve SharePoint Website Performance Abstract: Initial load tests revealed that the capacity of a customized Microsoft Office SharePoint Server (MOSS) website cluster
More informationCREATING BUSINESS VALUE THROUGH INTEGRATION
CREATING BUSINESS VALUE THROUGH INTEGRATION WHAT BIZTALK SERVER AND SQL SERVER PROVIDE DAVID CHAPPELL DECEMBER 2009 SPONSORED BY MICROSOFT CORPORATION CONTENTS Why Integration Matters... 3 Application
More informationStrategies for a Successful E2E Systems Integration Test. Fiona Charles Let s Test May 9, 2012
Strategies for a Successful E2E Systems Integration Test Fiona Charles Let s Test May 9, 2012 This session Describes key project management strategies I have used to manage large- scale Systems Integration
More informationUnit 9: License Management
Unit 9: License Management 9-1 Introduction This unit provides instructions for the tasks involved in creating and maintaining license entitlement information and licenses for products. Objectives Objectives
More information