Risk-Free Administration for Expert Database Administrators

Transcription

1

2 Risk-Free Administration for Expert Database Administrators Kurt Engeleiter, Oracle Jason Lentz, Cerner Bart Gaddis, Cerner

3 Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle s products remains at the sole discretion of Oracle. 3

4 Program Agenda DBA Challenges Risk Free Administration Tools New Performance Management Tools 4

5 Program Agenda DBA Challenges Risk Free Administration Tools New Performance Management Tools 5

6 Top Challenges Database Management 48% Unpredictable application (SQL) performance 78% Downtime resulting from unverified changes? Key Takeaway: Improve Quality of Service 40% Accurate capacity planning with 20-40% data store growth Source: From Database Clouds Copyright to Big 2015, Data: Oracle 2013 and/or IOUG its affiliates. Survey All rights On reserved. Database Oracle Confidential Manageability Internal 6

7 DBA Actions That Affect Performance Adding or dropping indexes, partitioning tables, adding materialized views Gathering optimizer statistics Applying patches one offs, PSUs, CPUs Modifying init.ora parameters Implementing new features such as Compression, Database In-Memory Version or patchset upgrades, e.g. 11g -> 12c >

8 Environment Changes That Affect Performance Data growth User population growth Infrastructure changes Server upgrades Changes in size or type of disk storage Migration to new platforms including migration to Cloud Database Consolidation 8

9 Program Agenda DBA Challenges Risk Free Administration Tools New Performance Management Tools 9

10 Risk Free Performance Management Tools Proactively identifies impact of routine DBA tasks on database performance SPA Quick Check Lower Risk Automated Cut Costs Database Replay Load and throughput testing solution Designed for detecting and remediating throughout problems SQL response time (unit) testing solution Designed for identifying and tuning regressed SQLs SPA Concurrent Replay Consolidation and capacity planning solution 10

11 Program Agenda DBA Challenges Risk Free Administration Tools SPA Quick Check SQL Performance Analyzer Database Replay Concurrent Database Replay New Performance Management Tools 11

12 SPA Quick Check Overview Helps users predict the impact of specific system changes on SQL workload Input is SQL Tuning Set (STS) and a database configuration change Automatically executes two trials pre and post change - and generates report Integrated with STS, SQL Plan Baselines, & SQL Tuning Advisor to form an endto-end solution SQL Plans + Stats Pre-change Trial Compare SQL Performance SQL Workload (STS) SQL Plans + Stats Post-change Trial Analysis Report 12

13 SPA Quick Check Optimized Controlled Optimized for use on prod systems Optimal Trial or Explain Plan mode Disable multi-executions, full DML execute disabled Per SQL time limits Testing scoped to private session Associate with Resource Consumer Group Optimal Trial Mode, no DML execute SPA Quick Check Optimized Controlled/S coped Change-Aware Per SQL Time Limits, Limits testing scope to private session Context-aware change testing Change- Aware Context-aware change testing workflows, such as, Optimizer gather statistics Init.ora parameter changes Pre-selected STS and default SPA settings Production Database DBA 13

14 SPA Quick Check Report

15 Risk Free Changes with SPA Quick Check Statistics management Gather statistics in pending mode Use SPA Quick Check to determine whether to publish Init.ora parameter changes Current parameter value is used for pre-change Select new value SPA Quick Check validates new parameter setting SQL Tuning Validation Validates SQL Profiles 15

16 SPA Quick Check Demo 16

17 Program Agenda DBA Challenges Risk Free Administration Tools SPA Quick Check SQL Performance Analyzer Database Replay Concurrent Database Replay New Performance Management Tools 17

18 SQL Performance Analyzer (SPA) Overview Helps users predict the impact of any system change on SQL workload User selects SQL Tuning Set User executes different SQL trials (experiments) of SQL statements performance by test execution or explain plan Analysis shows per SQL execution differences Integrated with STS, SQL Plan Baselines, & SQL Tuning Advisor to form an end-to-end solution 18

19 Risk Free Changes with SPA Access structure changes: Indexes Materialized views Parallelism Patching Upgrades Implementing new features such as Compression or Database In-Memory Validating new hardware servers or storage 19

20 Program Agenda DBA Challenges Risk Free Administration Tools SPA Quick Check SQL Performance Analyzer Database Replay Concurrent Database Replay New Performance Management Tools 20

21 Database Replay Database load and performance testing with real production workloads Production workload characteristics such as timing, transaction dependency, think time, etc., fully maintained Test and measure transaction throughput improvements Identify application scalability and concurrency problems on test system before production deployment Usage scenarios: platform migration (e.g., Exadata), upgrades, patching, new database features or options, etc. Production Clients Test Replay Driver Storage Storage Capture Process Replay Analysis & Reporting 21

22 Risk Free Changes with Database Replay Patching Upgrades Implementing new features Validating new hardware servers or storage Stress testing for capacity planning 22

23 Program Agenda DBA Challenges Risk Free Administration Tools SPA Quick Check SQL Performance Analyzer Database Replay Concurrent Database Replay New Performance Management Tools 23

24 Capacity Planning: Motivation User populations and data are continually growing How long can my environment handle the workload Organizations are pursuing database consolidation to deliver cost savings while increasing business agility How can we guarantee service levels between different workloads How can we test if one errant application is not affecting the others Consolidating workloads and databases has significant challenges Applications have different workload profiles CPU Memory Storage Network Will my Multitenant database handle peak workloads and co-exist? Is there enough headroom? 24

25 Concurrent DB Replay For Risk Free Consolidation and Capacity Planning Capture workload on different databases that needs to co-exists concurrently Works for schema consolidation and Pluggable Databases Use for schema and CDB consolidation Identify and remediate inter-application scalability and concurrency problems Perform capacity planning through scale up, subsetting, scheduling (time-shifting) of multiple workloads 25

26 Capacity Planning Using Database Replay and Concurrent Database Replay Comprehensive scale-up support and what-if scenarios testing Scale-up techniques superior to traditional methods Zero-scripting approach extended for scale-up Scales data and user population Realistic data and bindsets Flexible, supports custom workload creation Capacity Planning Strategies Scale-up by scheduling concurrent replays Scale-up with multiple PDBs Scale-up by workload folding Copyright 2015, Oracle and/or its affiliates. All rights reserved.

27 Capacity Planning Strategies: Peak Workload Testing Time-Shifting Application Workloads CRM Align Workload Peaks Replay Captured Workload ERP DW Peak Workload Testing Through Time-Shifting Exercise worst case scenario where workload peaks line up Evaluate and experience fallout from the safety of a test system 27

28 Capacity Planning Strategies: Multiple PDBs Sales Workload Test Database Replay Concurrent Workload Mapping PDB Clones Scale-up with Multiple Duplicate PDBs Perform scale-up testing to identify possible host bottlenecks when deploying multiple instances of an application Deploying a multitenant application or adding a new geographical area to an existing application or line of business 28

29 Capacity Planning Strategies: Workload Folding Sales Workload Workload Subsets Subset 1 Replay Concurrent Workload Subset Subset 2 Scale-up Through Workload Folding Perform scale-up testing by subsetting an existing workload capture into 2 & then perform concurrent replay of workload subsets Suitable for workloads where transactions in subsets are mostly independent 29

30 Concurrent Replay Capacity Planning Strategies Summary Scale-up Strategy Concurrent Database Replay Multiple identical PDBs Workload Folding Workload Suitability Enables consolidation validation Can be used for all workloads Simultaneous workload and data scale-up Multitenant what-if scenarios N * Workload scale-up Suitable for relatively stateless workloads Copyright 2015, Oracle and/or its affiliates. All rights reserved.

31 Program Agenda DBA Challenges Risk Free Administration Tools SPA Quick Check SQL Performance Analyzer Database Replay Concurrent Database Replay New Performance Management Tools 31

32 Database Performance Diagnostics Tools Automatic Workload Repository AWR Reports about performance and workload data from AWR Active Session History ASH Gathers fine-grain data about every active database session every second Automatic Database Diagnostics Monitor - ADDM Data Analysis and Problem Identification Findings and Advise on how best to resolve bottlenecks Real-time SQL and Database Operations Monitoring Provides in-depth diagnostics about SQL execution at row source level Database Performance Hub Provides Unified Monitoring Solution! 32

33 Database Performance Hub Unified Performance Monitoring Single view of DB performance ADDM, ASH analytics, Real-Time SQL Monitoring, SQL Tuning Switch between ASH analytics, workload view, ADDM findings and SQL monitoring seamlessly Supports both real-time & historical mode Historical view of SQL Monitoring reports Dedicated tab for RAC 33

34 Performance Hub Report New interactive report for analyzing AWR data Performance Hub report generated from OR calling dbms_perf.report_perfhub(.) function Single view of DB performance ADDM, SQL Tuning, Real-Time SQL Monitoring, ASH Analytics Switch between ASH analytics, workload view, ADDM findings and SQL monitoring seamlessly Supports both real-time & historical mode Historical view of SQL Monitoring reports 34

35 Perfhub Demo 35

36 New AWR Active-HTML Report New AWR report type active-html introduced in Oracle Database Provides best of HTML and Performance Hub Reports HTML report contains embedded Performance Hub Report as the last section ADDM task finds and recommendations are also presented Exadata-aware Highly recommended to use AWR active-html reports instead of HTML reports Combines power for EM navigation and drill down for offline analysis Can be saved and mailed like other Active Reports and does not need EM connectivity for viewing 36

37 AWR Report Vs Performance Hub Report 37

38 Customer Use Case Oracle Real Application Testing Jason Lentz Cerner Abilities Center Bart Gaddis Cerner Database Architecture October 26, 2015

39 Health care is too important to stay the same.tm

40 Cerner today BRNDEXP Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 40

41 Abilities Center Cerner s Wind Tunnel IP Development Millennium Platform Hardware Technologies Layered Software Technologies Java Services, RCP, Thin Client Country Specific Solutions Controlled Testing Environment Test Measure Analyze Correct Ability-Proven Affordability Reduce TCO High Availability 99.99% Uptime Perform-ability 2 Seconds is Too Slow Scalability Regional National Global CAMM Platform ibus Platform Dedicated Testing Resources Quality Isolated Network Secured Environment Experienced Associates IP Engineering Teams 30+ Millennium Domains Real Client Databases Automated Testing Tools SAN Storage AIX Citrix HPUX Linux Reliability 1 Crash is Too Many MQ Oracle VMWare WebSphere Test Design Lights On Network Test Partners / Upgrade Center CernerWorks BRNDEXP Leverage clinical and domain expertise Automate client representative workflows Test with role and venue data complexity Collects 7 billion client timers per month Built-in instrumentation allows lab/client comparison Enables focused enhancements in workflow coverage Release + Tech Production Validation Controlled / Monitored Implementation Experienced IP Resources Available 2014 Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 41

42 Why RAT? Capture and Replay client-like production workloads Volume Data diversity Minimal effort to utilize Workload Scale-up and Folding Test hardware at 10x current volume Stress storage and compute nodes Integrate/augment with existing workflow automation Consume replay activity data in real-time Measure impact of replay load on automated workflows BRNDEXP Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 42

43 Test Methodology Controlled Approach Measure one change Same SGA Same replay parameters Repeatable Replays Same number of user calls Low divergence Consistent duration Actionable Data Confidence in metrics Process/parameter changes Purchase decisions BRNDEXP Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 43

44 Workload Capture and Replay Procedure Two hour capture during peak volume on live client production system Database duplicate and recovery to capture SCN (Full copy) Database upgrade to SQL regression tuning Preprocess and tune replay clients Repeatable Workload Replays Flashback to guaranteed restore point Cache warming - Consolidated replay of entire workload (read only) Read-write workload replay Standardized data and report collection (Replay, AWR, OS) BRNDEXP Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 44

45 Test Methodology Controlled Approach Measure one change Same SGA Same replay parameters Repeatable Replays Same number of user calls Low divergence Consistent duration Actionable Data Confidence in metrics Process/parameter changes Purchase decisions BRNDEXP Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 45

46 Database Replay Analysis SQL Perspective SQL Tuning Sets, primary tool Capture SQL during baseline replay and subsequent replays SPA (SQL Performance Analyzer) has built in reporting using SQL Tuning Sets Find SQL which has regressed or improved Find SQL with plan changes Relate the change to the impact on workload STS comparison - Example Report BRNDEXP Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 46

47 SQL Tuning Sets Use Cases Optimizer Related Changes DBMS_STATS sample size, impact on execution plan changes DBMS_STATS histogram methodology (frequency histograms with autosample) Optimizer patches, impact on execution plan changes Optimizer parameters, impact on execution plan changes Discovery of bug related to ORA-7445 on hard parse due to corrupt histogram. Structural Changes Validation of Cluster Tables, impact on SQL execution statistics Investigation/evaluation of plan changes associated to Cluster Tables Key Views DBA_SQLSET_STATEMENTS, DBA_SQLSET_PLANS, DBA_SQLSET_BINDS BRNDEXP Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 47

48 Test Methodology Controlled Approach Measure one change Same SGA Same replay parameters Repeatable Replays Same number of user calls Low divergence Consistent duration Actionable Data Confidence in metrics Process/parameter changes Purchase decisions BRNDEXP Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 48

49 Major Replay Use Cases OS tuning (OS version, parameter change) Vertical scaling (Server hardware upgrade) Horizontal scaling (Add RAC instances) Notable Results IP Fragmentation Thresholds (IPFRAG) RESULTS: Increasing all thresholds decreased packet reassembles and failures and improved global cache metrics while remaining within hardware capabilities. DETERMINATION: Safe and effective to roll out to clients. BRNDEXP Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 49

50 Major Replay Use Cases OS tuning (OS version, parameter change) Vertical scaling (Server hardware upgrade) Horizontal scaling (Add RAC instances) Notable Results Migrate to new hosts with more CPU and RAM RESULTS: More efficient reads and gets increased user/application activity while decreasing waits (concurrency, cluster, commit). DETERMINATION: Improved performance and scaling point. Benefits outweigh cost. BRNDEXP Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 50

51 Major Replay Use Cases OS tuning (OS version, parameter change) Vertical scaling (Server hardware upgrade) Horizontal scaling (Add RAC instances) Notable Results Add RAC instances, up to 6 wide RESULTS: Resource utilization gradually decreased with every instance added to cluster. DETERMINATION: Adding additional hardware is an effective solution for increasing capacity. BRNDEXP Cerner Corporation. All rights reserved. This document contains Cerner confidential and/or proprietary information belonging to Cerner Corporation and/or its related affiliates which may not be reproduced or transmitted in any form or by any means without the express written consent of Cerner. 51

52 Contact Jason Lentz Bart Gaddis

53 53

54 54

55