How To Test For A Test On A Test Server

Transcription

1 Real Application Testing Dave Foster Master Principal Sales Consultant

2 The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle.

3 Why Consider Oracle 11g for Upgrades? Oracle Database 9i Release 2 Premium Support ended Jul 30, yr free Extended Support offer ends Jul 30, 2008 Today August 2012 August 2015 R2 July 2010 July 2013 Jul 2011 January 2009 January 2012 Premier Support Extended Support Sustaining Support R2 July 2007 Jul 2008 July 2010 t

4 Oracle Enterprise Manager Top-Down Application Management Application Operations Management Real User Experience Monitoring Service Level Management Packaged Application Management Oracle E-Business Suite Change Management Configuration Mgmt for Applications SOA Application Management Diagnostic and Tuning Provisioning, Configuration Management Change Tracking and Synchronization Application Testing Suite Application Quality Management Load Testing and Accelerators for Packaged Apps & Web Services Functional Testing and Accelerators for Packaged Apps & Web Services Test Management Real Application Testing Data Masking

5 Real Application Testing: Tools of the Trade What is it? What Purpose? How it works? SQL Performance Analyzer Predicts SQL performance deviations before end-users can be impacted Assess impact of change on SQL response time Executes each SQL, stored in SQL Tuning Set, in isolation using production context and then compares before and after execution plans and run-time statistics Database Replay Replays real database workload on test system Assess impact of change on workload throughput Captures workloads and replays it with production characteristics including concurrency, synchronization & dependencies When to use? Unit testing of SQL with the goal to identify the set of SQL statements with improved/regressed performance Comprehensive testing of all sub-systems of the database server using real production workload SQL Performance Analyzer and Database Replay are complementary

6 Real Application Testing Load and SQL impact testing solution providing highest quality testing for the database tier Value Rapid technology adoption Higher testing quality Business Benefit Lower cost Lower risk Features Database Replay SQL Performance Analyzer (SPA) Test Change Remediate Deploy Business Agility through Superior Testing

7 Testing Today Production 1,000s of Real Online Users Application Database PRODUCTION

8 Testing Today Test 1-2 testers trying to be 1,000s of users Application Database Database PRODUCTION TEST

9 Database Replay Real workload for 1,000s of online users captured Application Database Capture Workload PRODUCTION

10 Database Replay Real workload for 1,000s of online users replayed Application Database Capture Replay Database Workload PRODUCTION TEST

11 Database Replay Test your system changes at production levels Application Application Database , , 11g Capture Workload Replay Database 11g ONLY PRODUCTION TEST

12 Real Application Testing with Database Replay Replay production workload in test environment Identify, analyze and fix potential instabilities before making changes to production Capture Workload in Production Capture full production workload with real load, timing & concurrency characteristics Move the captured workload to test system Replay Workload in Test Make the desired changes in test system Replay workload with full production characteristics Honor commit ordering Analyze & Report Errors Data divergence Performance divergence Analysis & Reporting

13 Why DB Replay? From: Artificial workloads Partial workflows Months of development Manual intensive To: Production workloads Complete workflows Days of development Automated 150 Days High risk Low risk 10 Days

14 Replay Options Synchronized Replay (Default) Workload is replayed in full synchronized mode Same concurrency and timing as production workload Transaction commit order is honored Ensures minimal data divergence Synchronization controls Workload can be replayed in unsynchronized mode Useful for load/stress testing High data divergence Parameters for controlling synchronization Commit order synchronization: SYNCHRONIZATION Think time synchronization: THINK_TIME_SCALE Connect (logon) time synchronization: CONNECT_TIME_SCALE Request rate preservation: THINK_TIME_AUTO_CORRECT

15 Analysis & Reporting Error Divergence: For each call error divergence is reported New: Error encountered during replay not seen during capture Not Found: Error encountered during capture not seen during replay Mutated: Different error produced in replay than during capture Data Divergence Replay: Number of rows returned by each call are compared and divergences reported User: Application level validation scripts Performance Reporting Capture and Replay Report: Provides high-level performance information ADDM Report: Provides in-depth performance analysis AWR, ASH Report: Facilitates comparative or skew analysis

16 Clients Database Replay Workflow Production ( , 10gR2, 11g) Test (11g) Replay Driver Mid-Tier Storage Storage Capture Process Replay Analysis & Reporting

17 Step 1: Workload Capture All external client requests captured in binary files Production System System background and internal activity excluded Client Client Client Minimal overhead Avoids function call when possible Buffered I/O Middle Tier File System Independent of client protocol File 1 Can capture on , 10gR2, 11g and replay on 11g File 2 Capture load for interesting time period, e.g., peak workload, month-end processing, etc. Storage File n

18 Step 2: Process Workload Files Setup test system Application data should be same as production system as of capture start time Use RMAN, Snapshot Standby, imp/exp, Data Pump, etc. to create test system Make change: upgrade db and/or OS, change storage, migrate platforms, etc. Processing transforms captured data into replayable format Once processed, workload can be replayed many times For RAC copy all capture files to single location for processing or use shared file system File 1 File 2 File n Capture Files Test System File 1 File 2 File n Metadata Replay Files

19 Step 3: Replay Workload Replays workload preserving timing, concurrency and dependencies of the capture system Test System Replay Clients Replay Client is a special program that consumes processed workload and sends requests to the replay system Clients interpret captured calls into sequence of OCI calls and submit to database For high concurrency workloads, it may be necessary to start multiple clients File 1 File 2 File n Metadata Replay Files

20 Supported Changes with Real Application Testing Changes Unsupported Application Changes Supported Database Upgrades, Patches Schema, Parameters RAC nodes, Interconnect OS Platforms, OS Upgrades CPU, Memory Storage Etc. Database Recording of External Client Requests

21 Real Application Testing with SQL Performance Analyzer (SPA) Test impact of change on SQL query performance Capture SQL workload in production including statistics & bind variables Re-execute SQL queries in test environment Tune regressed SQL and seed SQL plans for production Client Client Client Production Test Middle Tier Re-execute SQL Queries Oracle DB Capture SQL Use SQL Tuning Advisor to tune regression 9i, 10g, 11g 10gR2, 11g Storage

22 Why SQL Performance Analyzer (SPA)? Businesses want systems that are performant and meet SLA s SQL performance regressions are #1 cause of poor system performance Solution for proactively detecting all SQL regressions resulting from changes not available DBA s use ineffective and time-consuming manual scripts to identify problems SPA identifies all changes in SQL performance before impacting users

23 SQL Performance Analyzer: Overview Production Test Client Middle Tier Re-execute SQL Oracle DB Capture SQL Transport SQL Storage Make Changes / Tuning Regressions If adequate spare cycles available, optionally execute SQL here * No middle & application tier setup required

24 SQL Performance Analyzer Restrictions SQL Performance Analyzer does not support the following features in the current release Shared server (Oracle MTS) 9i Parallel Query (SQL Trace Capture) Remote Transactions

25 SQL Performance Analyzer for Earlier Releases Help customers smoothly transition to Oracle Database 10g and 11g SQL Test Execution feature of SPA backported to 10gR2 For more details: Metalink Note: Real Application Testing for Earlier Releases Metalink Note: Testing Performance Impact of an Oracle 9i to Oracle Database 10g Release 2 Upgrade with SQL Performance Analyzer Upgrade From Upgrade To Release / Patches needed Comments 10gR2 10gR2 or 11g one-off patch or gR2 + one-off patch Uses STS to capture SQL stats 9i 10gR1 10gR2 or 11g one-off patch or gR2 + one-off patch Uses SQL*Trace to capture SQL stats

26 Use Real Application Testing to Manage Change with Confidence and LOWER change risk LOWER testing time without compromising quality LOWER unplanned outages IMPROVE application reliability IMPROVE system performance IMPROVE end-user experience IMPROVE quality of service ADOPT new technology faster