Oracle Database 11g: Real Application Testing

Transcription

1

2 <Insert Picture Here> Oracle Database 11g: Real Application Testing Katharina Römer Principal Sales Consultant

3 Oracle Database 11g: Real Application Testing SQL Performance Analyzer Database Replay <Insert Picture Here>

4 Why Real Application Testing? Businesses want to adopt new technology Extensive testing and validation is expensive Despite extensive testing success rate low Many issues go undetected System availability and performance are negatively impacted Cause of low success rate Current tools provide inadequate testing Simulate unrealistic synthetic workload Provide partial workflow coverage Real-Application Testing makes real-world testing possible

5 Realistic Testing: SQL Performance Analyzer <Insert Picture Here>

6 SQL Performance Analyzer Identify the benefits of using SQL Performance Analyzer Describe the SQL Performance Analyzer workflow phases Use SQL Performance Analyzer to ascertain performance changes following a database change

7 Realistic Testing: Real Application Testing 1. USE SQL PERFORMANCE ANALYZER FOR SQL RESPONSE TIME AND UNIT TESTING DONE? Yes No 2. USE DATABASE REPLAY FOR LOAD AND CONCURRENCY TESTING DONE? Yes No DEPLOY CHANGES AND ONGOING TUNING

8 SQL Performance Analyzer: Overview Targeted users: DBAs, QAs, application developers Helps predict the impact of system changes on SQL workload response time Builds different versions of SQL workload performance (that is, SQL execution plans and execution statistics) Executes SQL serially (concurrency not honored) Analyzes performance differences Offers fine-grained performance analysis on individual SQL Is integrated with SQL Tuning Advisor and SQL Plan Management to tune regressions

9 SQL Performance Analyzer: Use Cases Database upgrades Implementation of tuning recommendations Schema changes Statistics gathering Database parameter changes OS and hardware changes

10 Using SQL Performance Analyzer 1. Capture SQL workload on production 2. Transport the SQL workload to a test system 3. Build before-change performance data 4. Make changes 5. Build after-change performance data 6. Compare results from steps 3 and 5 7. Tune regressed SQL

11 Step 1: Capture SQL Workload Cursor cache Incremental capture Database Instance SQL tuning set (STS) is used to store SQL workload. It includes: SQL Text Bind variables Execution plans Execution statistics Incremental capture is used to populate STS from the cursor cache over a period of time STS s filtering and ranking capabilities filter out undesirable SQL Production database

12 Step 2: Transport to a Test System Cursor cache Database instance Database instance Production database Test database Copy the SQL tuning set to the staging table ( pack ) Transport staging table to the test system (data pump, DB link, etc) Copy the SQL tuning set from the staging table ( unpack )

13 Step 3: Build Before-Change Performance Data Before change, SQL performance version is the SQL workload performance baseline SQL performance = Execution plans + execution statistics Test execute SQL in STS: Produce execution plans & statistics Execute one SQL statement at a time (no concurrency) Every SQL statement is executed at least twice Avoid DDL/DML effects Explain plan SQL in STS generates only SQL plans Before changes Test database Test execute Database instance

14 Step 4: Implement Planned Change & Step 5: Build After-Change Performance Data Manually implement the planned change: Database upgrade Implementation of tuning recommendations Schema changes Statistics gathering Database parameter changes OS and hardware changes Reexecute SQL after change: Test execute SQL in STS to generate SQL execution plans and statistics After changes Test database Database instance After changes implemented

15 Step 6: Compare and Analyze Performance & Step 7: Tune Regressed SQL Rely on user-specified metric to compare SQL performance: ELAPSED_TIME, BUFFER_GETS, Calculate the impact of change on individual SQLs and SQL workload: Overall impact on workload Net SQL impact on workload Use the SQL execution frequency to define a weight of importance Detect improvements, regressions, and unchanged performance Detect changes in execution plans Recommend running SQL Tuning Advisor to tune regressed SQLs The analysis results can be used to seed SQL Plan Management baselines SQL Tuning Advisor Compare analysis Improvement Test database Regression Database instance

16 Remote Test Execution 11g SQL Performance Analyzer System No application schema or data is required in the SQL Performance Analyzer database Remote databases can be different versions SPA database DB link Prod database You can maintain a single repository for many tests DB link Test database

17 Accessing SQL Performance Analyzer Use Enterprise Manager Use the DBMS_SQLPA package

18 SQL Performance Analyzer: Enterprise Manager Access SQL Performance Analyzer on the Software and Support tab Select one of the five types of workflows: Upgrade from 9i or 10.1 Upgrade from 10.2 or 11g Parameter Change Exadata Simulation Guided Workflow Tune regressing statements by invoking SQL Tuning Advisor Prevent regressions by using SQL plan baselines

19 SQL Performance Analyzer: Enterprise Manager

20 SQL Performance Analyzer: PL/SQL Example Create the tuning task: exec :tname:= dbms_sqlpa.create_analysis_task( - sqlset_name => 'MYSTS', task_name => 'MYSPA'); Execute the task to build the before-change performance data: exec dbms_sqlpa.execute_analysis_task(task_name => :tname, - execution_type => 'TEST EXECUTE', execution_name => 'before'); Produce the before-change report: SELECT dbms_sqlpa.report_analysis_task(task_name => :tname, type=>'text', section=>'summary') FROM dual;

21 SQL Performance Analyzer: PL/SQL Example After making your changes, perform the following: Create the after-change performance data: EXEC dbms_sqlpa.execute_analysis_task(task_name => :tname, - execution_type => 'TEST EXECUTE', execution_name => 'after'); Generate the after-change report: SELECT dbms_sqlpa.report_analysis_task(task_name => :tname, type=>'text', section=>'summary') FROM dual; Compare the task executions: EXEC dbms_sqlpa.execute_analysis_task(task_name => :tname, execution_type => 'COMPARE PERFORMANCE'); Generate the analysis report: SELECT dbms_sqlpa.report_analysis_task(task_name => :tname, type=>'text', section=>'summary') FROM dual;

22 Tuning Regressed SQL Statements To tune the regressed SQL statements reported by SQL Performance Analyzer, create a SQL tuning task for the SQL Performance Analyzer execution by using the DBMS_SQLTUNE.CREATE_TUNING_TASK function: BEGIN DBMS_SQLTUNE.CREATE_TUNING_TASK( spa_task_name => 'MYSPA', spa_compare_exec => 'MYCOMPEXEC'); END; /

23 Testing Database Upgrades: Oracle9i Database & Oracle Database 10g Release 1 SQL Performance Analyzer supports testing database upgrades of Oracle9i and Oracle Database 10g Release 1, to Oracle Database 10g Release 2 and later releases Execute the SQL tuning set on the upgraded database remotely over a database link The production system that you are upgrading from should be running Oracle9i or Oracle Database 10g Release 1 The test system containing the Oracle Database version that you are upgrading to, should be running Oracle Database 10g Release 2 ( ) or later Set up a separate system for SQL Performance Analyzer with Oracle Database 11g Release 1 ( ) or a later release

24 Testing Database Upgrades: Oracle9i Database & Oracle Database 10g Release 1 To use SQL Performance Analyzer in a database upgrade from Oracle9i or Oracle Database 10g Release 1 to a newer release, perform the following steps: 1. Enable the SQL trace facility on the production system Tip: Enable SQL trace only for a subset of the sessions, and only for as long as required, to capture all the important SQL statements at least once 2. On the production system, create a mapping table 3. Move the SQL trace files and the mapping table from the production system to the SQL Performance Analyzer system 4. On the SQL Performance Analyzer system, construct a SQL tuning set by using the SQL trace files

25 Testing Database Upgrades: Oracle9i Database & Oracle Database 10g Release 1 5. On the SQL Performance Analyzer system: Use SQL Performance Analyzer to create a SQL Performance Analyzer task and convert the contents in the SQL tuning set into a preupgrade SQL trial that will be used as a baseline for comparison Remotely test execute the SQL statements on the test system over a database link to build a postupgrade SQL trial 6. Compare SQL performance and fix regressed SQL statements

26 Testing Database Upgrades: Oracle Database 10g Release 2 & Later Releases SQL Performance Analyzer supports testing database upgrades of Oracle Database 10g Release 2 or a later release to any later release Capture a SQL tuning set on the production system and execute it twice remotely over a database link on a test system The production system that you are upgrading from should be running Oracle Database 10g Release 2 or a later release Initially, the test system should be running the same Oracle Database release as the production system Set up a separate system for running SQL Performance Analyzer: Oracle Database 11g Release 1 ( ) or a later release

27 Testing Database Upgrades: Oracle Database 10g Release 2 & Later Releases Perform the following steps to use SQL Performance Analyzer in a database upgrade from Oracle Database 10g Release 2 and later releases to a newer release: 1. On the production system, capture the SQL workload that you intend to analyze and store it in a SQL tuning set 2. Set up the test system such that it matches the production environment as closely as possible 3. Transport the SQL tuning set to the SQL Performance Analyzer system

28 Testing Database Upgrades: Oracle Database 10g Release 2 & Later Releases 4. On the SQL Performance Analyzer system, create a SQL Performance Analyzer task using the SQL tuning set as its input source. Remotely test execute the SQL statements on the test system to build a preupgrade SQL trial that will be used as a baseline for comparison 5. Upgrade the test system 6. Remotely test execute the SQL statements a second time on the upgraded test system over a database link to build a postupgrade SQL trial 7. Compare SQL performance and fix regressed SQL statements

29 SQL Performance Analyzer: Data Dictionary Views Modified views in Oracle Database 11g: DBA{USER}_ADVISOR_TASKS: Displays details about the analysis task DBA{USER}_ADVISOR_FINDINGS: Displays analysis findings New views in Oracle Database 11g: DBA{USER}_ADVISOR_EXECUTIONS: Lists metadata information for task execution DBA{USER}_ADVISOR_SQLPLANS: Displays the list of SQL execution plans DBA{USER}_ADVISOR_SQLSTATS: Displays the list of SQL compilation and execution statistics

30 SQL Performance Analyzer: Summary Identify the benefits of using SQL Performance Analyzer Describe the SQL Performance Analyzer workflow phases Use SQL Performance Analyzer to determine performance changes following a database change

31 SQL Performance Analyzer: Tutorial <Insert Picture Here>

32 SQL Performance Analyzer: Guided Workflow

33 SQL Performance Analyzer Task Create

34 Create SQL Trial in Initial Environment

35 Create SQL Trial in Initial Environment

36 Create SQL Trial in Changed Environment

37 Create SQL Trial in Changed Environment

38 Guided Workflow: Compare Step 2 & Step 3

39 Guided Workflow: Compare Step 2 & Step 3

40 Guided Workflow: View Trial Comparison Report

41 SQL Performance Analyzer: Comparison Report

42 Realistic Testing: Database Replay <Insert Picture Here>

43 Database Replay Identify the benefits of using Database Replay List the steps involved in using Database Replay Use Enterprise Manager to record and replay workloads

44 Why Use Database Replay? System changes (hardware, software upgrades ) are a fact of life Changes can negatively affect system availability and performance Customers want to identify the full impact of changes before going live Extensive testing and validation can be expensive in terms of time and money Despite expensive testing, success rates are low: The inability to properly test with real-world production workloads results in many issues going undetected Database Replay makes it possible to do realistic testing

45 Using Database Replay Re-create actual production database workload in the test environment Capture the workload in production: Capture full production workload with real load & concurrency Move the captured workload to the test system Replay the workload in the test environment: Make the desired changes in the test system Replay the workload with production load & concurrency Honor commit ordering Analyze and report: Errors Data divergence Performance divergence Identify and analyze potential instabilities before making changes to the production system

46 Process capture files The Big Picture Prechange production system Clients/app servers Capture directory Shadow capture file Postchange test system Replay system Shadow capture file Production system Shadow capture file Shadow capture file Test system with changes Production database Database backup Database restore Can use Snapshot Standby as test system

47 System Architecture: Capture Capture directory Shadow Shadow Shadow Shadow Recording infrastructure Database stack Shadow capture file Shadow capture file Shadow capture file Background Background Shadow capture file Database backup Production database

48 Preprocess capture files System Architecture: Workload Preprocessing Capture directory Shadow capture file Shadow capture file Database stack Shadow capture file Background Background Shadow capture file Database backup Production database Preprocess capture

49 Preprocess capture files System Architecture: Replay Replay system Replay client Replay client Capture directory Shadow Shadow Shadow Shadow Shadow capture file Shadow capture file Database stack Shadow capture file Background Background Shadow capture file Database backup Test system with changes Test database

50 Changes Supported by Database Replay Changes not supported Clients/app servers Supported changes: Database upgrades, patches Schema, parameters RAC nodes, interconnect OS platforms, OS upgrades CPU, memory Storage Production database Production system

51 Capture Considerations Ensure that there is adequate disk space for captured workload (binary files) Database restart: Is the only way to guarantee authentic replay Start the database in restricted mode Starting the capture process changes the database into normal (unrestricted) mode May not be necessary depending on the workload Following are ways to restore the database for replay: Physical restore (scn/time provided) Logical restore of application data Flashback/snapshot standby Filters can be specified to capture subset of workload SYSDBA or SYSOPER privileges and appropriate OS privileges are required

52 Capture Considerations Overhead: Performance overhead for TPC-C benchmark: 4.5% Memory overhead: 64 KB per session Disk space

53 Replay Considerations Preprocessing the captured workload: Is a one-time action Must use the same database software version as the replay system Should be conducted on a system other than the production server (recommendation) Restore database, and then perform the change: Upgrade Schema changes OS change Hardware change Instance addition

54 Replay Considerations Manage external interactions Remap connection strings to be used for the workload: One-to-one: For simple instance-to-instance remapping Many-to-one: Use of load balancer (for example, single node to RAC) Modify the DB links and directory objects that point to the production systems Set up one or more replay clients Each replay client is multithreaded and can drive multiple workload sessions

55 Replay Options Synchronized replay: Commit-based synchronization Minimal data divergence Unsynchronized replay: Useful for load/stress testing Original commit ordering not honored Possibility for high data divergence Login time options Percentage (default is 100%)

56 Replay Options Think time options: THINK_TIME_SCALE adjusts the think time to modify captured request rate: 0%: No think time (highest possible request rate) <100%: Higher request rate 100%: Exact think time >100%: Lower request rate THINK_TIME_AUTO_CORRECT automatically shortens the think time if calls execute slower in replay than during capture: The default setting is TRUE Workload scale-up during replay (SCALE_UP_MULTIPLIER): Replay captured session concurrently for specified number of times Only one session in each set of identical replay sessions will execute both queries and updates

57 Workload Replay Filters Use workload filters to specify which database calls to include in or exclude from the workload Following are the types of workload replay filters: Inclusion filter: Specifies database calls that will be replayed Exclusion filter: Specifies database calls that will not be replayed Create and use workload replay filters as follows: 1. Use DBMS_WORKLOAD_REPLAY.ADD_FILTER to define the workload replay filters 2. Use DBMS_WORKLOAD_REPLAY.CREATE_FILTER_SET to add the workload replay filters to a replay filter set 3. Use DBMS_WORKLOAD_REPLAY.USE_FILTER_SET to filter the replay

58 Replay Analysis Data divergence: Number of rows compared for each call (queries, DML) Error divergence: New errors Mutated errors Errors that have disappeared Performance: Capture and Replay report ADDM report ASH report for skew analysis AWR report

59 Database Replay Workflow: Enterprise Manager 1. Capture the workload on a database by using the Capture Workload wizard 2. Use the Capture Workload wizard to also optionally export the AWR data relating to the captured workload 3. Restore the replay database on a test system 4. Make changes to the test system as required 5. Copy the workload to the test system 6. Preprocess the captured workload by using the Preprocess Captured Workload wizard 7. Configure the test system for the replay 8. Replay the workload on the restored database by using the Replay Workload wizard

60 Accessing Database Replay

61 Packages and Procedures DBMS_WORKLOAD_CAPTURE START_CAPTURE FINISH_CAPTURE ADD_FILTER DELETE_FILTER DELETE_CAPTURE_INFO GET_CAPTURE_INFO EXPORT_AWR IMPORT_AWR REPORT DBMS_WORKLOAD_REPLAY PROCESS_CAPTURE INITIALIZE_REPLAY PREPARE_REPLAY START_REPLAY CANCEL_REPLAY DELETE_REPLAY_INFO REMAP_CONNECTION EXPORT_AWR IMPORT_AWR GET_REPLAY_INFO REPORT COMPARE_PERIOD_REPORT

62 Database Replay: PL/SQL Example exec DBMS_WORKLOAD_CAPTURE.ADD_FILTER(fname => 'sessfilt',- fattribute => USER,- fvalue => 'JFV'); exec DBMS_WORKLOAD_CAPTURE.START_CAPTURE(name => 'june_peak',- dir => 'jun07'); Execute your workload exec DBMS_WORKLOAD_CAPTURE.FINISH_CAPTURE(); exec DBMS_WORKLOAD_REPLAY.PROCESS_CAPTURE(capture_dir => 'jun07'); exec DBMS_WORKLOAD_REPLAY.INITIALIZE_REPLAY(replay_name => 'j_r',- replay_dir => 'jun07'); exec DBMS_WORKLOAD_REPLAY.REMAP_CONNECTION(connection_id => 101,- replay_connection => 'edlin44:3434/bjava21');

63 Database Replay: PL/SQL Example exec DBMS_WORKLOAD_REPLAY.PREPARE_REPLAY(synchronization => TRUE,- think_time_scale=> 2); wrc userid=system password=oracle replaydir=/dbreplay exec DBMS_WORKLOAD_REPLAY.START_REPLAY (); DECLARE cap_id NUMBER; rep_id NUMBER; rep_rpt CLOB; BEGIN cap_id := DBMS_WORKLOAD_REPLAY.GET_REPLAY_INFO(dir => 'jun07'); /* Get the latest replay for that capture */ SELECT max(id) INTO rep_id FROM dba_workload_replays WHERE capture_id = cap_id; rep_rpt := DBMS_WORKLOAD_REPLAY.REPORT(replay_id => rep_id, format => DBMS_WORKLOAD_REPLAY.TYPE_TEXT); END;

64 Data Dictionary Views: Database Replay DBA_WORKLOAD_CAPTURES: Lists all the workload captures performed in the database DBA_WORKLOAD_FILTERS: Lists all the workload filters defined in the database DBA_WORKLOAD_REPLAYS: Lists all the workload replays that have been performed in the database DBA_WORKLOAD_REPLAY_DIVERGENCE: Is used to monitor workload divergence DBA_WORKLOAD_CONNECTION_MAP: Is used to review all connection strings that are used by the workload replays V$WORKLOAD_REPLAY_THREAD: Monitors the status of the external replay clients

65 Calibrating Replay Clients $ wrc system/oracle@oc11 mode=calibrate replaydir=/dbreplay Workload Replay Client: Release Production on Tue Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved. Report for Workload in: /dbreplay Recommendation: Consider using at least 1 clients divided among 1 CPU(s). Workload Characteristics: - max concurrency: 4 sessions - total number of sessions: 11 Assumptions: - 1 client process per 50 concurrent sessions - 4 client process per CPU - think time scale = connect time scale = synchronization = TRUE $

66 Database Replay: Summary Identify the benefits of using Database Replay List the steps involved in using Database Replay Use Enterprise Manager to record and replay workloads

67 Database Replay: Tutorial <Insert Picture Here>