Adaptive Query Optimization

Transcription

1 Oracle OpenWorld, Christian Antognini BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA 1

2 @ChrisAntognini Senior principal consultant, trainer and partner at Trivadis in Zurich (CH) Focus: get the most out of Oracle Database Logical and physical database design Query optimizer Application performance management Author of Troubleshooting Oracle Performance (Apress, 2008/2014) OakTable Network, Oracle ACE Director 2

3 Introduction Over the years Oracle has extremely improved the capabilities of the query optimizer Most of the improvements fall into one of the following areas Enhance the quality of the inputs (e.g. objects statistics) Make the gathering and management of object statistics easier and more efficient Use new data structures Implement or enhance query transformations Improve plan stability Cope with poor estimations that leads to poor execution plans 12c is no exception, every one of these areas were improved 3

4 Features Adaptive plans (Enterprise Edition only) Join methods Parallel distribution methods Star transformation Adaptive statistics SQL plan directives Automatic reoptimization - Statistics feedback (evolution of cardinality feedback) - Performance feedback Dynamic statistics (evolution of dynamic sampling) 4

5 OPTIMIZER_ADAPTIVE_FEATURES Enables or disables adaptive query optimization features Adaptive plans SQL plan directives Automatic reoptimization - It isn t the case in (bug ) The default value is TRUE OPTIMIZER_DYNAMIC_SAMPLING controls dynamic statistics 5

6 OPTIMIZER_ADAPTIVE_REPORTING_ONLY Enables or disables reporting mode for adaptive query optimization features Useful to assess how an execution plan would change Use DBMS_XPLAN to get detail information about the analysis Might fail with an ORA-1001 (bug ) SELECT * FROM table(dbms_xplan.display_cursor(format=>'report')) The default value is FALSE 6

7 Adaptive Plans Challenge Object statistics don t always provide sufficient information to find an optimal execution plan To get additional insights, the query optimizer can take advantage of features like dynamic sampling and cardinality feedback Don t solve all issues, though 7

8 Adaptive Plans Concept As of 12c, the query optimizer can postpone some decisions until the execution phase The idea is to leverage information collected while executing part of an execution plan to determine how another part should be carried out The query optimizer uses adaptive plans in three situations: To switch the join method from a nested loops join to a hash join and vice versa To switch the PX distribution method from hash to broadcast/round-robin To disable the access to a dimension for execution plans using the star transformation 8

9 Adaptive Plans Join Method Switch The query optimizer adds subplans to execution plans One represents the nested loops join, the other the hash join One of the alternatives is the default plan One of the subplans is chosen during the first execution The choice is based on the number of rows actually processed The query optimizer computes an inflection point A new row source operation is used to partially buffer and count the rows STATISTICS COLLECTOR Limitation: if a too large buffer is required, no adaptive plan is used The execution plan that is actually executed is called the final plan 9

10 Adaptive Plans Join Method Switch Example adaptive_plan.sql SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = Id Operation Name SELECT STATEMENT 1 HASH JOIN 2 NESTED LOOPS 3 NESTED LOOPS 4 STATISTICS COLLECTOR 5 TABLE ACCESS FULL T1 6 INDEX UNIQUE SCAN T2_PK 7 TABLE ACCESS BY INDEX ROWID T2 8 TABLE ACCESS FULL T

11 Adaptive Plans Join Method Switch Example adaptive_plan.sql SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = Id Operation Name SELECT STATEMENT 1 HASH JOIN 2 NESTED LOOPS 3 NESTED LOOPS 4 STATISTICS COLLECTOR 5 TABLE ACCESS FULL T1 6 INDEX UNIQUE SCAN T2_PK 7 TABLE ACCESS BY INDEX ROWID T2 8 TABLE ACCESS FULL T

12 Adaptive Plans Join Method Switch Example adaptive_plan.sql SELECT * FROM t1, t2 WHERE t1.id = t2.id AND t1.n = Id Operation Name SELECT STATEMENT 1 HASH JOIN 2 NESTED LOOPS 3 NESTED LOOPS 4 STATISTICS COLLECTOR 5 TABLE ACCESS FULL T1 6 INDEX UNIQUE SCAN T2_PK 7 TABLE ACCESS BY INDEX ROWID T2 8 TABLE ACCESS FULL T

13 Adaptive Plans Join Method Switch Inflection Point For both join methods, the cost associated to different cardinalities is estimated The cardinality of the outer table varies The cardinality of the inner table remains fixed The query optimizer uses a binary search The search takes place between a minimum and maximum cardinality 13

14 Cost Adaptive Plans Join Method Switch Inflection Point Example 20,000 2, Nested Loops Join Hash Join 20 Cardinality (In Order of Execution) 14

15 Adaptive Plans Parallel Distribution Method New distribution method HYBRID HASH It helps avoiding some data skewing problems The actual distribution method is chosen at execution time STATISTICS COLLECTOR The decision takes place during each execution Number of rows < 2 * DOP Left input: BROADCAST / Right input: ROUND-ROBIN Number of rows 2 * DOP Left input: HASH / Right input: HASH 15

16 Adaptive Plans Parallel Distribution Method Example hybrid_hash.sql Operation Name TQ IN-OUT PQ Distrib SELECT STATEMENT SORT AGGREGATE PX COORDINATOR PX SEND QC (RANDOM) :TQ10002 Q1,02 P->S QC (RAND) SORT AGGREGATE Q1,02 PCWP HASH JOIN Q1,02 PCWP PX RECEIVE Q1,02 PCWP PX SEND HYBRID HASH :TQ10000 Q1,00 P->P HYBRID HASH STATISTICS COLLECTOR PX BLOCK ITERATOR Q1,00 PCWC Q1,00 PCWC TABLE ACCESS FULL T1 Q1,00 PCWP PX RECEIVE Q1,02 PCWP PX SEND HYBRID HASH :TQ10001 Q1,01 P->P HYBRID HASH PX BLOCK ITERATOR Q1,01 PCWC TABLE ACCESS FULL T2 Q1,01 PCWP

17 Adaptive Plans Star Transformation With the star transformation, the data of each dimension that has a restriction applied to it might be joined to the corresponding bitmap index of the fact If the number of rowids returned by such a join is underestimated, applying the filter can be detrimental to the performance With an adaptive plan the access to some dimensions can be disabled Decision takes place during the first execution only 17

18 Adaptive Plans Star Transformation Example Operation Name VIEW VW_ST_5497B905 NESTED LOOPS BITMAP CONVERSION TO ROWIDS BITMAP AND BITMAP MERGE BITMAP KEY ITERATION TABLE ACCESS FULL COLORS BITMAP INDEX RANGE SCAN CAR_COLOR_IDX STATISTICS COLLECTOR BITMAP MERGE BITMAP KEY ITERATION TABLE ACCESS FULL MODELS BITMAP INDEX RANGE SCAN CAR_MODEL_IDX TABLE ACCESS BY USER ROWID CARS

19 Adaptive Plans V$SQL.IS_RESOLVED_ADAPTIVE_PLAN NULL means that the execution plan associated to the cursor isn t adaptive N means that the final execution plan hasn t been determined Y means that the final execution plan was determined Also set if reporting mode is enabled Set for join method switches and star transformation only 19

20 Adaptive Statistics SQL Plan Directives SQL plan directives are automatically created when misestimates occur They are temporarily stored in the SGA and flushed to disk every 15 min by a background process They aren t tied to a specific SQL statement, but to specific tables/columns Several of them can be used for a single SQL statement They instruct the database engine to automatically create extended statistics Only column groups are considered The limit of 20 extensions applies to them as well If creating extended statistics isn t supported/possible/sensible, they instruct the query optimizer to use dynamic sampling 20

21 Adaptive Statistics Management of SQL Plan Directives The database engine automatically maintains SQL plan directives They are automatically created They are automatically purged if not used (by default after 53 weeks) They can be manually managed through DBMS_SPD Flush to disk SQL plan directives temporarily stored in the SGA Change retention period Immediate purge Alter attributes (ENABLED, AUTO_DROP) Pack/Unpack Requirement: ADMINISTER SQL MANAGEMENT OBJECT privilege 21

22 Adaptive Statistics SQL Plan Directives in the Data Dictionary DBA_SQL_PLAN_DIRECTIVES DBA_SQL_PLAN_DIR_OBJECTS SQL> SELECT type, reason, count(*) 2 FROM dba_sql_plan_directives 3 GROUP BY type, reason; TYPE REASON COUNT(*) DYNAMIC_SAMPLING SINGLE TABLE CARDINALITY MISESTIMATE 81 DYNAMIC_SAMPLING JOIN CARDINALITY MISESTIMATE 180 DYNAMIC_SAMPLING GROUP BY CARDINALITY MISESTIMATE 6 22

23 Adaptive Statistics Statistics Feedback statistics_feedback1.sql statistics_feedback2.sql Evolution of cardinality feedback Used for single-table cardinalities as well as join cardinalities Information about misestimates might be persisted through SQL plan directives For misestimates due to table functions no information is stored V$SQL.IS_REOPTIMIZABLE Y means that the next execution will trigger a reoptimization N means that no reoptimization is necessary R means that reoptimization information is available, but reporting mode was enabled 23

24 Adaptive Statistics Performance Feedback Enabled when PARALLEL_DEGREE_POLICY = ADAPTIVE Assessment of the auto DOP after the first execution If auto DOP is suboptimal, the next execution will trigger a reoptimization 24

25 Adaptive Statistics Dynamic Statistics dynamic_statistics.sql Evolution of dynamic sampling Used for single-table cardinalities as well as join and group-by cardinalities The query optimizer decides when and how to use dynamic statistics The data collected through dynamic statistics is shared through the result cache The sampling can t exceed a given amount of time The maximum is determined based on past executions (cursor cache, AWR) To use the new features OPTIMIZER_DYNAMIC_SAMPLING has to be set to 11 25

26 Summary Adaptive query optimization introduced a number of very interesting features The query optimizer is getting more and more dynamic 26

27 Questions and answers... Christian Antognini Senior Principal Consultant BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA Trivadis