White paper: Unlocking the potential of load testing to maximise ROI and reduce risk.

Transcription

1 White paper: Unlocking the potential of load testing to maximise ROI and reduce risk. Executive Summary Load testing can be used in a range of business scenarios to deliver numerous benefits. At its core, load testing enables organisations to improve system performance, increasing throughput and creating happier, more productive end-users. It also plays a crucial role in risk management and mitigation. Load testing is a valuable tool to maximise ROI, improve user experience and productivity, and quantify the impact of system changes. There are a range of scenarios where load testing can add significant business value including: Maximum ROI, minimum risk Load testing is a fast, easy and cost effective way of generating business gains that is often overlooked by decision makers. In this white paper, Myriad IT details the uses of load testing and how they deliver business benefit. We also provide an outline of specific scenarios where load testing can add significant value. Ensuring maximum ROI is achieved from existing investment; Risk management and mitigation; Quantifying the benefits/risks of new technology investments; Identification of bottlenecks; Managing upgrades; and Benchmarking against peers and competitors. Topics covered include: Business benefits of load testing Load testing as a service Load testing has a vast range of applications to technology and can be used to test reliability and performance across a range of platforms and applications including hardware, software, networks, web applications, virtualised environments and databases. Load testing can be provided as a service which eliminates the need to purchase expensive software or have specialised in-house expertise in order to benefit. How fast is your ERP? The value of benchmarking When to use load testing 3 load testing case studies Page 1

2 BUSINESS BENEFITS OF LOAD TESTING Maximising ROI on your technology Efficient allocation of memory and processing power means that IT resources are utilised optimally to generate maximum business benefit. This is applicable regardless of the technology in question and extends from hardware to networks, servers, databases, and applications. Virtualised environments make this dynamic allocation of system resources easy and allow environments to be quickly fine-tuned based upon load testing findings. Risk management & mitigation The stability and performance of production and operations systems is critical to ensuring business risks are kept at a minimum. Load testing can assist with the identification, management and mitigation of risks associated with performance degradation and stability problems. Any issues can be identified and addressed prior to them becoming problematic or causing critical system failure. Myriad IT s load testing service can be used to test reliability & performance across a range of platforms & applications including: Hardware Software Networks Web applications Virtualised environments Databases Load testing results can also be used to develop a management strategy for monitoring potential pressure points. Quantifying benefits/risks of new investment Load testing is an effective means of quantifying the effects of any system changes enabling managers to demonstrate to stakeholders that proposed changes will deliver measurable results. This includes implementation of new hardware, architecture changes, virtualisation or an application upgrade. Page 2

3 For example: To understand how much faster an ERP system is after implementing a new database server, a load testing exercise can quantify the costs to the business. It can also evaluate whether a new database will address performance problems Identify bottlenecks Load testing enables us to understand where it is necessary to augment hardware, change software or hardware configuration so throughput can be improved. ERP changes constantly and new data, new processes and new modules will affect system performance. Performance tuning getting the most out of your system Undertaking a load testing exercise enables Myriad IT to identify where gains are to be found in a system. Of equal importance, we are also able to identify where gains cannot be realised. Upgrade Load testing is particularly useful when it comes to upgrade time. Not only can it be used to quantify performance benefits of an upgrade, it can also be used prior to go-live to stress test architecture in a range of different scenarios to ensure that systems will not crash. LOAD TESTING AS A SERVICE How does it work? Load testing software can be expensive and requires specialist skill to develop testing scenarios, interpret results and make recommendations. Myriad IT s load testing service is a very costeffective approach to undertaking testing. We utilise our Oracle Application Testing Suite licence to create and execute the tests for up to 500 concurrent users. Myriad IT s testing service is a turnkey solution to your load testing needs. It eliminates the need to purchase load testing software & clients benefit from our expertise in providing a frame of reference to interpret results. How easy is it? The load testing process is very simple. Myriad IT typically logs in remotely and installs load testing software on an allocated virtual machine. We create the testing scenarios and our testing consultants configure all system monitoring required. Myriad IT assists in the identification of key business processes and then records these into load test scenarios. How long does it take? The length of time a load testing assignment takes is dependent upon the complexity of the assignment. For example testing in preparation for a major ERP upgrade with around 250 users and scenarios can be completed in less than 10 days. This includes the preparation and delivery of detailed reports and an executive summary. Myriad IT is easily able to provide a fixed scope at a fixed price. Page 3

4 HOW FAST IS YOUR ERP? Standardised performance testing Ensuring your ERP is running optimally can be crucial in maintain your competitive edge however, many organisations find it challenging to understand how their ERP compares to their peers and competitors. Undertaking load testing on a stand-alone basis will yield information that can be used to allocate memory and processing power efficiently however it does not provide an indication of whether the performance profile is within industry standards. Your system can be faster Load testing provides insight into a number of key metrics including hits per second and pages per second. These metrics can then be used to measure the impact of any changes as set out in the table below: Table 1: Key considerations for system speed How does your ERP performance profile compare to other organisations? How do you know what is normal? How do you know what is fast? How much bandwidth does a busy ERP user use? How do you know where improvements can be made? All of these questions are difficult to answer without a frame of reference. Understand peak usage A 200 concurrent user load test can generate 400GB of HTML traffic in less than 2 hours. In the same test HTML servers talk at a rate of 1MB/second but you application server can be up to 10MB. Consider impact of test results on network architecture? Understand impact of your changes The same 200 user test can produc over 100,000 unique ERP pages and generate over half a million hits. Load testing helps you to understand the impact of every change that is made on CPU, disk utilisation, network, hits/second and pages/second. What's the best server for JD Edwards? Ever wanted to know Whether Oracle Weblogic server is faster than IBM's Websphere for JD Edwards? Ever wanted to know how much traffic a single compressed IIS reverse proxy could save you? Optimal sizing With Load Testing as a Service, Myriad IT is able to identify the ideal number of JD Edwards users per Java Virtual Machine and also what size your JVM should be. Fast & easy kernel testing Kernel testing is a breeze with load testing as a service. Understanding exactly which kernels are being used and how many messages they are processing helps us to fine tune them leading to a faster system Page 4

5 CASE STUDY: ERP IMPLEMENTATION GO-LIVE The scenario Client A was preparing for go-live on a large JD Edwards implementation. Having changed database, web and more they needed to know that the system would survive 400 users logging in on day one. Myriad IT was asked to help. Dynamic testing & monitoring solution Over a period of 10 days, Myriad IT simulated 20 of Client A s top processes using Oracle Application Testing Suite (OATS). The OATS scripts were programmed dynamically to read dynamic data from tables & spreadsheets and were then run in a range of concurrencies to simulate a day in the life processing. The workload was a mixture of batch and interactive processing. Myriad IT co-ordinated the tests, and checked the system before and after each test to ensure transactions hit the database. Additionally, Myriad IT configured monitoring tools to understand exactly what was happening on all of the servers. Improved stability & performance Myriad IT gathered system data and performance metrics from OATS to deliver a report to Client A. During the course of testing, numerous bottlenecks were identified and quickly resolved. Myriad It made over ten (10) system changes (hardware allocations & configuration changes) to improve stability and performance of the ERP. Performance gains & peace of mind Using the load test results, Myriad IT was able to double the throughput of JD Edwards in comparison to the initial configuration values i.e. performance was improved twofold. Knowing in advance that the system could handle the load meant that the go-live was a huge success for Client A. The client is now looking at more testing services from Myriad IT, including functional test automation. Plugins for Enterprise Manager 12c were used to extract the real-time performance data. Page 5

6 THE VALUE OF BENCHMARKING Myriad IT has undertaken load testing for numerous clients, all with different architectures and goals. We assist with the development of scenarios, interpret results and provide recommendations about how to best leverage resources and manage processes. Myriad IT has developed standardised benchmark scripts so that we can determine how a particular installation compares with other organisations and provide feedback on relative performance for batch and interactive processing. Without the ability to compare and interpret results, metrics are of little value. By creating a benchmark, it becomes fast and easy to understand the impact of any changes to a system. Once the scripts have been run at a particular organisation, we then have a benchmark for that organisation which can be used for future tests to quantify performance. When is benchmarking useful? Benchmarks are a powerful tool to quantify changes with statistics and data: New tools releases & upgrades New software New hardware Disaster recovery testing Test the cloud (cloud migration, cloud utilisation & cloud optimisation) Changes to network architecture. Benchmarks also enable Myriad IT to identify if a database is working harder, how much CPU is being consumed as well as make recommendations on kernel turning or web server tuning. Page 6

7 WHEN SHOULD I USE LOAD TESTING? Have you ever wanted to know the actual performance improvements that have been realised as a result of your innovation? Load testing has a multitude of applications for enhancing performance managing risk as set out below. Table 2: Key applications for load testing Go-live New hardware Web compression New JD Edwards tools release New database release or patch The system will support the anticipated user load Win the users over from day one Justification of hardware investment Fine tuning of hardware allocations Ensure performance levels are maintained, web traffic is reduced and machine CPU utilisation is appropriate Ensure interactive and batch processes perform as well, or better using the new tools release. Stress test the tools release before users. Fine tune all aspects of your database to leverage the most critical piece of the performance puzzle. CASE STUDY: MICROSOFT DYNAMICS CRM GO-LIVE Testing scenario Oracle Application Testing Suite (OATS) is not just limited to Oracle products. Myriad IT client, Client B was preparing to go live with a new member management system based on Microsoft Dynamics CRM. It was critical that on day one the system could be accessed by any number of Client B s members. Consistent performance Australia-wide Myriad IT generated the use case scenarios and logged into Client B s system via the internet. We then used OATS to simulate users from different parts of Australia in order to generate comparison information on network latency based upon geographical location. In just 4 days, Myriad IT was able to simulate 250 concurrent users logging into Client B s cloud based CRM system. Users from Perth, Sydney and Melbourne were tested to ensure that the performance was consistent across Australia. As this was a cloud based solutions, CPU utilisation was not important to Client B, the primary concern was with end user performance metrics. Platform change SSL/SSO implementations Ensure that the new platform is stable and will perform. Quantify the improvements to management Ensure that any implemntation does not excessively affect CPU consumption on the web tier. Safe and secure SSL implementation. Stress free go-live The system performance was ratified and Client B was able to enjoy a relatively stress free go-live, thanks to load testing as a service. Changes to database configuration Test changes with a full production load to ensure they will not cause performance or stability issues. Page 7

8 Myriad IT s load testing results are based on real business processes. Our reports can be customised to meet specific needs but typically provide: Summary issues that will need resolution Summary of performance recommendations Summary of recommended configuration changes Error analysis & resolution Network traffic analysis Hardware utilisation Web, database & application memory consumption Identification of weakest Pathway for ongoing management CASE STUDY: IMPLEMENTATION OF AN ALTERNATE AUTHENTICATION PACKAGE Testing Scenario Myriad IT s client, Client C, wanted to implement Myriad IT s Active Directory Authentication product, which allows long username and password authentication in JD Edwards. Client C also needed to ensure that the product could handle the anticipated user load and complex load balancing. Straightforward testing in just 2 days The tests were very simple to create and implement, as Myriad IT was not testing transactions in the database but the ability to sign in. As part of the exercise, Myriad IT also added some compression of the web traffic, so data compression was also tested. Successful log-ins & enhanced performance Client C was provided with all of the results for the successful log-ins and also some valuable information on the savings that compression would realise in terms of CPU utilisation on the web server. Client C was exceptionally happy with the results. By utilising Myriad IT s Load Testing as a Service for two (2) days, they eliminated the need to learn OATS or buy the software. As a result of testing, Client C was confident that all users could log in successfully. The client also gained a full understanding of the impact of web compression in their environment. Page 8

9 About Myriad IT Headquartered in Melbourne, Australia, Myriad IT provides intelligent, reliable technology solutions and outstanding technical support to clients, Australia-wide & overseas. Our reputation has been built by helping clients achieve business goals through the development, implementation and integration of technology within an organisation. Maximising your competitive edge Technology is critical in today s business world yet it is often difficult to find the right mix of skills and expertise to manage it in-house. Myriad IT s team of specialists which includes functional, technical and development consultants, engineers, architects and managers who help companies optimise their technology, ensuring it runs smoothly and delivers maximum competitive advantage to our clients. Comprehensive technology services Myriad IT offers an end-to-end service and advises clients on the best and most cost-effective solutions to meet their business needs. Our services include: Oracle consulting ERP systems Microsoft Dynamics CRM solutions Infrastructure services Testing services Virtualisation & cloud computing Managed & hosted services JD Edwards consulting Support Enterprise architecture CONTACT US: Myriad IT Services Pty Ltd Level 9, 455 Bourke St Melbourne VIC 3000 AUSTRALIA Tel (61) Document Disclaimer This document is issued in commercial confidence only for the purpose for which it is supplied. Myriad IT cannot be held responsible for any typography or diagrammatic errors. The Myriad IT logo is a trademark of Myriad IT Services Pty Ltd. Oracle, JD Edwards, and PeopleSoft are registered trademarks of Oracle and/or its affiliates. Other trademarks and trade names that may have been used in this document are not the property of Myriad IT and Myriad IT disclaims proprietary interest in marks and names of others. Page 9