#96600 Using Oracle Application Testing Suite (OATS): A JD Edwards EnterpriseOne Load and Functional Testing Case Study David Scheeff Bob Lynch Krisztina Scheeff Jim Mackowiak
ABOUT US KDS Technologies We are an ERP consulting and implementation company focused on Oracle s JD Edwards application suite. We provide technical solutions to companies running Oracle's JD Edwards EnterpriseOne software. We also offer a full range of software testing solutions. Edwards Lifesciences Edwards Lifesciences is the global leader in the science of heart valves and hemodynamic monitoring. Supplying products and technologies to clinicians in nearly 100 countries around the world. Global company, 7000 employees, 500 concurrent JDE users in 6 regions.
GOALS OF OUR PRESENTATION Introduce OATS (Oracle Application Testing Suite) Features & Functions How to get started (beginners welcome!) Script creation demonstration Case study (JDE load test / Japan network test) Requirements and scenarios Load test results Tips and lessons learned; Advanced Topics; Q&A
TESTING CONCEPTS AND OATS Functional test - does the application meets end user business requirements? OpenScript is Oracle s scripting platform to create automated test scripts Load / stress test - can the system handle the number of users expected? Stress test. When / how big of a load causes the system to fail In OATS load tests are run by OLT (Oracle Load Testing) Test management - this is where you keep all your stuff. Integrated modules for total test management. OTM (Oracle Test Manager) Test cases, scripts, requirements, bugs, and reports
OATS ARCHITECTURE OATS Windows Server OpenScript WEB LOGIC WINDOWS SERVICES OATS Agent Service OATS Application Service OATS Helper Service Repository C:\OracleATS\OFT OATS Administrator Oracle Load Testing (OLT) Oracle Test Manager (OTM) DATABASE Oracle XE
OPENSCRIPT Create Scripts A script is a sequence of actions and tests performed on a website or application. Eclipse based scripting tool for OATS For JD Edwards create a Web script Automated functional and load scripts Run scripts from OLT and Test Manager OATS doesn t know about JD Edwards...yet!
OPENSCRIPT Tree View shows the script navigations and data as nodes in a tree view. Step groups Recorded actions (clicks, typing, delays) Modifications Text match Object test
OPENSCRIPT Java Code View shows the script navigations and data as Java programing code. Java Code view corresponds to the Tree View. Modify OATS generated code Add custom functions
OPENSCRIPT Initialize script actions executed once at beginning of the script. ie. Login & Setup Run script actions to perform one or more times during playback. Finish script actions executed once at the end of the script. ie. Close & Signout
VIDEO Recording a script in Open Script
LOAD TEST ORACLE LOAD TESTING (OLT) OLT provides an easy and accurate way to test scalability of your applications. It can emulate thousands of users accessing your site simultaneously. Reusable scripts uses the scripts you created in OpenScript Distributed agents Scenario manager and autopilot Real time graphs and reports Post run analysis Server side monitoring with ServerStats
LOAD TESTING OLT SETUP Setup - VUs - System - Iteration - Think Time - Other Settings
LOAD TESTING OLT SETUP Autopilot - Start & Stop - Ramp-Up - Server Stats
LOAD TESTING OLT REPORTS Reports - Run Graphs - VU Grid - Saved Scenario - Graphs - Reports (text)
Load Test Video
RECORDED A SCRIPT. NOW WHAT? Test Manager Integrated modules for total test process management requirements, test cases, bugs related to it Run functional test cases from Test Manager, automatically save results Web based interface Manual and automated tests User defined roles and fields Reports on results etc by user, test cases failed, passed, time stamp, area etc.
CASE STUDY: OATS AT EDWARDS LIFESCIENCES Oracle / JDE Load Test - load test prior to July 25th, 2011 go-live New infrastructure: hardware refresh & software upgrades Moving from Citrix (JDE 8.10) to Web (JDE 9.0 on 8.98.4.3 tools release) VmWare: SQL Server, EnterpriseOne & WebLogic on Windows 2008 R2 5 different scripts Japan Network Test - Japan user base migrating from World to EnterpriseOne Network impact: user traffic moved from local systems to corporate system Project goal: use load test to verify network infrastructure Plan: re-run load test after infrastructure upgrade to compare with baseline 13 different scripts
JDE Environment at Edwards Lifesciences
ORACLE JDE LOAD TEST 450 Users on one machine and two agents 30 min ramp up, run for 1.25 hours Verify that new hardware can support the business New software upgrade 5 scripts
ORACLE JDE LOAD TEST - RESULTS Users vs Time
ORACLE JDE LOAD TEST - RESULTS KBs per sec, Users vs Time
ORACLE JDE LOAD TEST - RESULTS
ORACLE JDE LOAD TEST - CONCLUSION Infrastructure problem identified before go-live Intermittent response time issue in JDE Found problem only occurred when Data Warehouse was in use Problem: shared SAN (per vendors recommendation) Resolution: added SAN controller and disks to separate data Final result: System can handle 450 users OATS paid for itself by helping to identify this issue
JAPAN NETWORK LOAD TEST GOAL: load system to test bandwidth The test scenario designed for this run was to ramp up a concurrent user load to max out at 150 users from 4 different Locations in Japan OATS agents in 4 machines located in Tokyo, Sapporo, Osaka, & Fukoka 25 users per location with an additional 50 users in the Tokyo data center Different activities while emulating users on the system Iteration delay to allow for user think time 13 scripts recorded and played back in Kanji
JAPAN NETWORK LOAD TEST - SETUP
JAPAN NETWORK LOAD TEST - SETUP Recorded vs Random Ramp up 30 min Run for 45 minutes One script failed Data preparation
JAPAN NETWORK LOAD TEST - RESULTS Users vs Time (per script)
JAPAN NETWORK LOAD TEST - RESULTS Users vs Time (per script and total users)
JAPAN NETWORK LOAD TEST - RESULTS Number of Users / HitsPerSec/ KB RecSent Traffic from Tokyo
JAPAN NETWORK LOAD TEST - RESULTS Performance vs Time
JAPAN NETWORK LOAD TEST - RESULTS
JAPAN NETWORK LOAD TEST - RESULTS
JAPAN NETWORK LOAD TEST - CONCLUSION OATS generated load of 150 fast paced users Ramp up 30 minutes, followed by 45 minutes run Only 100 normal users are expected concurrently from Japan Network handled the load without issue Upgraded network should be even better
TIPS & LESSONS LEARNED New Users: keep it simple when getting started; re-record! Subject matter experts *must* participate when creating custom scripts Planning takes time - scripts and data Load and Functional scripts must be recorded separately Learning curve for more advanced scripts Restart OATS services
ADVANCED TOPICS Parameterization - alter scripts to use data from a DataBank Correlation - replace dynamic data with variables (ie: session ID) ServerStats - record server performance information in OLT OLT collects: scripts, # users, time to complete each script ServerStats adds server data: CPU, Memory, JVM Heap usage, etc Command Line Interface - run OpenScript from a command line Create Windows batch files to run scripts; batch files can be scheduled
ADVANCED TOPICS Error Handling Error types: Fail, Warn, Ignore, ReportAndContinue Default is to Fail for any error Different types of errors can be independently configured Concurrency two users might select the same order at the same time Business Object Reservation ensures mutually exclusive access How do you handle that type of situation? - Script Synchronization - Detect Reserved By
NEXT STEPS Download OATS (30 day trial) Getting Started Guide OTN: QA/Testing Forum Coming soon: Oracle Functional sample scripts
QUESTIONS? Email us info@kdstechnologies.com Download presentation at www.kdstechnologies.com Find us at the 80 s party tomorrow!