#96600 Using Oracle Application Testing Suite (OATS): A JD Edwards EnterpriseOne Load and Functional Testing Case Study. David Scheeff Bob Lynch

Transcription

1 #96600 Using Oracle Application Testing Suite (OATS): A JD Edwards EnterpriseOne Load and Functional Testing Case Study David Scheeff Bob Lynch Krisztina Scheeff Jim Mackowiak

2 ABOUT US KDS Technologies We are an ERP consulting and implementation company focused on Oracle s JD Edwards application suite. We provide technical solutions to companies running Oracle's JD Edwards EnterpriseOne software. We also offer a full range of software testing solutions. Edwards Lifesciences Edwards Lifesciences is the global leader in the science of heart valves and hemodynamic monitoring. Supplying products and technologies to clinicians in nearly 100 countries around the world. Global company, 7000 employees, 500 concurrent JDE users in 6 regions.

3 GOALS OF OUR PRESENTATION Introduce OATS (Oracle Application Testing Suite) Features & Functions How to get started (beginners welcome!) Script creation demonstration Case study (JDE load test / Japan network test) Requirements and scenarios Load test results Tips and lessons learned; Advanced Topics; Q&A

4 TESTING CONCEPTS AND OATS Functional test - does the application meets end user business requirements? OpenScript is Oracle s scripting platform to create automated test scripts Load / stress test - can the system handle the number of users expected? Stress test. When / how big of a load causes the system to fail In OATS load tests are run by OLT (Oracle Load Testing) Test management - this is where you keep all your stuff. Integrated modules for total test management. OTM (Oracle Test Manager) Test cases, scripts, requirements, bugs, and reports

5 OATS ARCHITECTURE OATS Windows Server OpenScript WEB LOGIC WINDOWS SERVICES OATS Agent Service OATS Application Service OATS Helper Service Repository C:\OracleATS\OFT OATS Administrator Oracle Load Testing (OLT) Oracle Test Manager (OTM) DATABASE Oracle XE

6 OPENSCRIPT Create Scripts A script is a sequence of actions and tests performed on a website or application. Eclipse based scripting tool for OATS For JD Edwards create a Web script Automated functional and load scripts Run scripts from OLT and Test Manager OATS doesn t know about JD Edwards...yet!

7 OPENSCRIPT Tree View shows the script navigations and data as nodes in a tree view. Step groups Recorded actions (clicks, typing, delays) Modifications Text match Object test

8 OPENSCRIPT Java Code View shows the script navigations and data as Java programing code. Java Code view corresponds to the Tree View. Modify OATS generated code Add custom functions

9 OPENSCRIPT Initialize script actions executed once at beginning of the script. ie. Login & Setup Run script actions to perform one or more times during playback. Finish script actions executed once at the end of the script. ie. Close & Signout

10 VIDEO Recording a script in Open Script

11 LOAD TEST ORACLE LOAD TESTING (OLT) OLT provides an easy and accurate way to test scalability of your applications. It can emulate thousands of users accessing your site simultaneously. Reusable scripts uses the scripts you created in OpenScript Distributed agents Scenario manager and autopilot Real time graphs and reports Post run analysis Server side monitoring with ServerStats

12 LOAD TESTING OLT SETUP Setup - VUs - System - Iteration - Think Time - Other Settings

13 LOAD TESTING OLT SETUP Autopilot - Start & Stop - Ramp-Up - Server Stats

14 LOAD TESTING OLT REPORTS Reports - Run Graphs - VU Grid - Saved Scenario - Graphs - Reports (text)

15 Load Test Video

16 RECORDED A SCRIPT. NOW WHAT? Test Manager Integrated modules for total test process management requirements, test cases, bugs related to it Run functional test cases from Test Manager, automatically save results Web based interface Manual and automated tests User defined roles and fields Reports on results etc by user, test cases failed, passed, time stamp, area etc.

17

18

19 CASE STUDY: OATS AT EDWARDS LIFESCIENCES Oracle / JDE Load Test - load test prior to July 25th, 2011 go-live New infrastructure: hardware refresh & software upgrades Moving from Citrix (JDE 8.10) to Web (JDE 9.0 on tools release) VmWare: SQL Server, EnterpriseOne & WebLogic on Windows 2008 R2 5 different scripts Japan Network Test - Japan user base migrating from World to EnterpriseOne Network impact: user traffic moved from local systems to corporate system Project goal: use load test to verify network infrastructure Plan: re-run load test after infrastructure upgrade to compare with baseline 13 different scripts

20 JDE Environment at Edwards Lifesciences

21 ORACLE JDE LOAD TEST 450 Users on one machine and two agents 30 min ramp up, run for 1.25 hours Verify that new hardware can support the business New software upgrade 5 scripts

22 ORACLE JDE LOAD TEST - RESULTS Users vs Time

23 ORACLE JDE LOAD TEST - RESULTS KBs per sec, Users vs Time

24 ORACLE JDE LOAD TEST - RESULTS

25 ORACLE JDE LOAD TEST - CONCLUSION Infrastructure problem identified before go-live Intermittent response time issue in JDE Found problem only occurred when Data Warehouse was in use Problem: shared SAN (per vendors recommendation) Resolution: added SAN controller and disks to separate data Final result: System can handle 450 users OATS paid for itself by helping to identify this issue

26 JAPAN NETWORK LOAD TEST GOAL: load system to test bandwidth The test scenario designed for this run was to ramp up a concurrent user load to max out at 150 users from 4 different Locations in Japan OATS agents in 4 machines located in Tokyo, Sapporo, Osaka, & Fukoka 25 users per location with an additional 50 users in the Tokyo data center Different activities while emulating users on the system Iteration delay to allow for user think time 13 scripts recorded and played back in Kanji

27 JAPAN NETWORK LOAD TEST - SETUP

28 JAPAN NETWORK LOAD TEST - SETUP Recorded vs Random Ramp up 30 min Run for 45 minutes One script failed Data preparation

29 JAPAN NETWORK LOAD TEST - RESULTS Users vs Time (per script)

30 JAPAN NETWORK LOAD TEST - RESULTS Users vs Time (per script and total users)

31 JAPAN NETWORK LOAD TEST - RESULTS Number of Users / HitsPerSec/ KB RecSent Traffic from Tokyo

32 JAPAN NETWORK LOAD TEST - RESULTS Performance vs Time

33 JAPAN NETWORK LOAD TEST - RESULTS

34 JAPAN NETWORK LOAD TEST - RESULTS

35 JAPAN NETWORK LOAD TEST - CONCLUSION OATS generated load of 150 fast paced users Ramp up 30 minutes, followed by 45 minutes run Only 100 normal users are expected concurrently from Japan Network handled the load without issue Upgraded network should be even better

36 TIPS & LESSONS LEARNED New Users: keep it simple when getting started; re-record! Subject matter experts *must* participate when creating custom scripts Planning takes time - scripts and data Load and Functional scripts must be recorded separately Learning curve for more advanced scripts Restart OATS services

37 ADVANCED TOPICS Parameterization - alter scripts to use data from a DataBank Correlation - replace dynamic data with variables (ie: session ID) ServerStats - record server performance information in OLT OLT collects: scripts, # users, time to complete each script ServerStats adds server data: CPU, Memory, JVM Heap usage, etc Command Line Interface - run OpenScript from a command line Create Windows batch files to run scripts; batch files can be scheduled

38 ADVANCED TOPICS Error Handling Error types: Fail, Warn, Ignore, ReportAndContinue Default is to Fail for any error Different types of errors can be independently configured Concurrency two users might select the same order at the same time Business Object Reservation ensures mutually exclusive access How do you handle that type of situation? - Script Synchronization - Detect Reserved By

39 NEXT STEPS Download OATS (30 day trial) Getting Started Guide OTN: QA/Testing Forum Coming soon: Oracle Functional sample scripts

40 QUESTIONS? us Download presentation at Find us at the 80 s party tomorrow!