Explaining EXPLAIN: DB2 10 Edition

Transcription

1 #db2zos Willie Favero Dynamic Warehouse on z/os Swat Team DB2 SME IBM Silicon Valley Laboratory

2 Notices This information was developed for products and services offered in the U.S.A. Note to U.S. Government Users Restricted Rights Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-ibm product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-ibm Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Information concerning non-ibm products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs conforming to IBM's application programming interfaces. Slide 2 of 51

3 Trademarks This presentation contains trade-marked IBM products and technologies. Refer to the following Web site: Slide 3 of 51

4 What is EXPLAIN? EXPLAIN captures detailed information about what could impact SQL performance -or - It gathers access path selection information -or - - A peek at what the optimizer might do! - Slide 4 of 51

5 EXPLAIN was introduced in DB2 Version 1.2 on March 7, 1986 Slide 5 of 51

6 How is it Executed? Can be dynamically prepared Run as a static SQL statement BIND/REBIND command option Automatically at automatic REBIND Special register Slide 6 of 51

7 Access Path Selection DB2 Catalog Tables Valid SQL DB2 Internal Cost Formulas I/O Cost TS Scan vs. Index Scan Index Scan (MATCHCOLS, CLUSTERRATION) Workfile Scans DSNDB07 DB2 Optimizer CPU Cost Traversing Rows and s Applying Predicates(RDS vs. DM) Cost of sorting = "Least Cost Access Path" Slide 7 of 51

8 Statistics RUNSTATS Inline stats (REORG, LOAD, etc ) And don t forget about Stats Advisor Slide 8 of 51

9 SQL EXPLAIN EXPLAIN PLAN SET QUERY=nnn sql_statement EXPLAIN STMTCACHE ALL EXPLAIN STMTCACHE STMTID value EXPLAIN STMTCACHE STMTTOKEN value EXPLAIN PACKAGE COLLECTION cccccc PACKAGE pppppp Only works with DB2 9 packages and above. Available in Conversion Mode (CM) Slide 9 of 51

10 Plan (Package) Management DSNZPARM keyword PLANMGMT Need to specify BASIC or EXTENDED BASIC Access path information in repository One access path retained EXTENDED Access path information in repository Two access paths are retained; previous and original EXPLAIN PACKAGE keyword COPY specifies CURRENT, PREVIOUS, or ORIGINAL should be used HINT_USED contains which one 0 (current copy of package) 1 (previous copy of package) 2 (the original copy of the package) Slide 10 of 51

11 QUERYNO The column that brings it all together Of the 18 (DB2 9 has 16 or V8 has 15) EXPLAIN tables, all but 1 has a QUERNO column Populated by EXPLAIN SQL statement EXPLAIN ALL SET QUERYNO=nnn FOR... <<<<< optional but should be used Slide 11 of 51

12 QUERYNO Populated by When EXPLAIN (YES) or EXPLAIN (ONLY) is specified on the BIND/REBIND commands QUERYNO is SQL statement number in pgm unless Including QUERYNO keyword on SQL statement SELECT... FROM... WHERE... WITH... QUERYNO nnn statement limit (is that really a problem) Could simplify using OPTHINTS Also used as QUERYNO in SYSIBM.SYSSTMT and SYSIBM.SYSPACKSTMT Slide 12 of 51

13 EXPLAIN(ONLY) Populate EXPLAIN tables without creating or replacing package Available in Conversion Mode (CM) Same cost as bind Do not need the authority to run SQL in package to EXPLAIN package Populate EXPLAIN tables even though EXPLAIN(NO) was originally specified Test access path before replacing Goes well with APREUSE(ERROR) or APCOMPARE(ERROR) APREUSE(ERROR) Reuse previous access path. If any statement cannot be reused, bind ends. APCOMPARE(ERROR) compare old and new access for each statement. Terminate if different. Slide 13 of 51

14 EXPLAIN Tip DSNZPARM ABEXP keyword on the DSN6SPRM macro Should EXPLAIN processing take place at automatic rebind? YES Default; perform EXPLAIN processing at auto rebind You want to specify YES NO - no EXPLAIN processing at auto rebind If package bound with EXPLAIN(NO), no EXPLAIN processing takes place at rebind regardless of this ZPARM s setting. If the package was bound with EXPLAIN(YES) and ABEXP is YES EXPLAIN processing still happens during auto rebind even if the PLAN_TABLE doesn't exist, there's just no output produced. Slide 14 of 51

15 EXPLAIN Special Register SET CURRENT EXPLAIN MODE = NO - turns of capturing EXPLAIN information. This is the default YES - turns feature on causing EXPLAIN information to be collected and plan tables to be updated. EXPLAIN - this option is the same of YES. However, the dynamic SQL statements are not executed host_variable must be CHAR or VARCHAR and can only contain one of the above in uppercase and left justified Valid for eligible SQL only SELECT, INSERT, and the searched form of UPDATE and DELETE. For this one, you ll have to wait for new function mode (NFM) to use Slide 15 of 51

16 EXPLAIN Privilege EXPLAIN is considered a system privilege that allows you to grant select privileges without granting execute on the SQL EXPLAIN specifying any of these keywords ALL/PLAN, STMTCACHE ALL, STMTCACHE STMTID, STMTCACHE STMTTOKEN, MONITORED STMTS, Issue the SQL statements DESCRIBE TABLE and PREPARE Specify BIND options EXPLAIN(ONLY) and SQLERROR(CHECK), Explain dynamic SQL statements executing under special register CURRENT EXPLAIN MODE Slide 16 of 51

17 EXPLAIN Privilege Syntax: GRANT EXPLAIN TO. Can grant on authorization ID, role, or to PUBLIC Can also specify WITH GRANT OPTION Once again, for this one, you ll have to wait for new function mode (NFM) to use Slide 17 of 51

18 Populates the PLAN_TABLE? EXPLAIN s will go here: PLAN_TABLE Describes access path of SQL statement Help better design SQL statements Can be used for optimization hints No wait That doesn t seem correct? BTW, Remember when everyone was trying to get to the hidden EXPLAIN tables? Slide 18 of 51

19 Populate the PLAN_TABLE? Explain s output actually goes three places: PLAN_TABLE Describes access path of SQL statement Help better design SQL statements Can give optimization hints Added in DB2 Version 6 DSN_STATEMNT_TABLE Provides cost estimates No, Still not correct Cost in service units and in milliseconds For dynamic and static SQL statements DSN_FUNCTION_TABLE How DB2 resolves functions One row for each function in an SQL statement Slide 19 of 51

20 Populate the PLAN_TABLE? And then there were 15 : PLAN_TABLE, DSN_STATEMNT_TABLE, DSN_FUNCTION_TABLE Tables added in DB2 Version 8 DSN_STATEMENT_CACHE_TABLE If you have dynamic statement caching enabled, the statement cache table contains the information about any SQL statements in the statement cache as a result of issuing the SQL statement EXPLAIN STATEMENT CACHE ALL. No, But getting closer You may want to consider having IFCIDs 316,317,and 318 active for additional details. DSN_DETCOST_TABLE The detailed cost table contains information about detailed cost estimation of the mini-plans in a query. Slide 20 of 51

21 Populate the PLAN_TABLE? Tables used by EXPLAIN : DSN_FILTER_TABLE The filter table contains information about how predicates are used processing. during query DSN_PGRANGE_TABLE The page range table, DSN_PGRANGE_TABLE, contains information about qualified partitions for all page range scans in a query. DSN_PGROUP_TABLE No, But getting closer The parallel group table, DSN_PGROUP_TABLE, contains information about the parallel groups in a query. Slide 21 of 51

22 Populate the PLAN_TABLE? Tables used by EXPLAIN : DSN_PREDICAT_TABLE The predicate table, DSN_PREDICAT_TABLE, contains information about all of the predicates in a query. DSN_PTASK_TABLE The parallel tasks table, DSN_PTASK_TABLE, contains information about all of the parallel tasks in a query. DSN_QUERY_TABLE No, But getting closer The query table contains information about a SQL statement, and displays the statement before and after query transformation. Not populated for static SQL statements at BIND or REBIND with EXPLAIN(YES) Slide 22 of 51

23 Populate the PLAN_TABLE? Tables used by EXPLAIN : DSN_SORTKEY_TABLE The sort key table contains information about sort keys for all of the sorts required by a query. DSN_SORT_TABLE The sort table contains information about the sort operations required by a query. DSN_STRUCT_TABLE The structure table contains information about all of the query blocks in a query. No, But getting closer DSN_VIEWREF_TABLE The view reference table contains information about all of the views and materialized query tables that are used to process a query. Slide 23 of 51

24 Populate the PLAN_TABLE? Tables used as input to EXPLAIN : DSN_VIRTUAL_INDEXES The virtual indexes table enables optimization tools to test the effect of creating and dropping indexes on the performance of particular queries Added in DB2 Version 8 DSN_USERQUERY_TABLE No, But getting closer contains the information about optimization hints Added in DB2 10 Slide 24 of 51

25 Populate the PLAN_TABLE? Tables used by EXPLAIN as of DB2 10: DSN_KEYGTDIST_TABLE This table contains non-uniform index expression statistic that are obtained dynamically by the DB2 optimizer. DSN_COLDIST_TABLE contains non-uniform column group statistics that are obtained dynamically by DB2 from non-index leaf pages Maybe now DSN_QUERYINFO_TABLE Used for the IBM DB2 Analytics Accelerator Slide 25 of 51

26 DB2 10 Format EBCDIC EXPLAIN tables deprecated in DB2 10 EXPLAIN tables in format before DB2 Version 8 disallowed Migration job SDSNSAMP(DSNTIJPM) has queries to discover non-compliant tables DDL for EXPLAIN tables SDSNSAMP(DSNTESC) contains DDL to build all EXPLAIN tables Data Studio (no-charge download) Optim Query Tuner (charge) Optim Query Workload Tuner (charge) Slide 26 of 51

27 DSN_STATEMENT_CACHE_TABLE The next few slides discuss how to use the dynamic statement cache Slide 27 of 51

28 Activate Dynamic Statement Cache Activate Dynamic Statement Cache and make sure it s of adequate size To activate the Dynamic Statement Cache: Install panel CACHE DYNAMIC SQL field or DSNZPARM keyword DSN6SPRM macro, keyword CACHEDYN Statement cache must of size to hold the entire workload Install panel EDM STATEMENT CACHE field 5000 to KB allowed, default is The dynamic statement cache space is above the 2GB bar or DSNZPARM DSN6SPRM EDMSTMTC (specified in KB) Slide 28 of 51

29 Start Trace for Cache Non-Data-Sharing -START TRACE command example -STA TRACE(MON) CLASS(30) IFCID(316,317,318) DEST(SMF) DSNW130I - MON TRACE STARTED, ASSIGNED TRACE NUMBER 04 DSN9022I - DSNWVCM1 '-STA TRACE' NORMAL COMPLETION Data-Sharing START TRACE command example -STA TRACE(MON) CLASS(30) IFCID(316,317,318) DEST(SMF) SCOPE(GROUP) DSNW130I - MON TRACE STARTED, ASSIGNED TRACE NUMBER 04 DSN9022I - DSNWVCM1 '-STA TRACE' NORMAL COMPLETION * The DEST is optional on the START TRACE command in the above examples. Note: Do not use a Monitor Class 29 trace. It only starts IFCIDs 316 & 318 when IFCID 317 is also required. Slide 29 of 51

30 Start Trace for Cache IFCID 316 First 60 bytes of SQL statement plus identifying information and statistics (statement Id, authid, usage counters) m.ibm.omegamon.xe_db2.doc/ko2rrd20228.htm IFCID 317 Used in addition to IFCID 316 to obtain the full SQL statement text m.ibm.omegamon.xe_db2.doc/ko2rrd20228.htm IFCID 318 Contains no data and acts only as a switch for IFCID 316 to collect all available information Stopping and Starting IFCID 318 will reset stats and cause a new interval to start Minimal impact on CPU and minimal impact (if any) on SMF. Slide 30 of 51

31 Capture Dynamic Statement Cache Execute the EXPLAIN STMTCACHE ALL Issuing the above SQL statements results in one row being returned for each statement cached in the dynamic statement cache to the DSN_STATEMENT_CACHE_TABLE table. Each row returned contains identifying information about the cached statement along with statistics reflecting that statements execution. Statistics are returned ONLY when the IFCID 316 and 318 traces are active. This SQL statement can be executed using SPUFI, DSNTEP2, DSNTEP4, or any number of programs that allow dynamically prepared SQL statements. Ensure SCHEMA (SQLID if pre DB2 9) is set correct for the EXPLAIN tables you are using for processing. Make sure you have the proper authorization to dump all the SQL in the dynamic statement cache. For example, running the EXPLAIN STMTCACHE ALL with SYSADM authority will avoid this issue. Slide 31 of 51

32 Stop Trace Stop the trace ONLY after verifying DSN_STATEMENT_CACHE_TABLE table appears to be accurate Stop trace -STOP TRACE(MON) CLASS(30) DSNW131I - STOP TRACE SUCCESSFUL FOR TRACE NUMBER(S) 04 DSN9022I - DSNWVCM1 '-STO TRACE' NORMAL COMPLETION Verify trace has been successfully stopped -DISPLAY TRACE(*) Slide 32 of 51

33 PLAN_TABLE, etc... EXPLAIN Output tables: You are responsible for creating all EXPLAIN tables Ownership For Dynamic - SQLID For PLAN - Plan owner For Package - Package owner Can use ALIAS as of Version 8 Data in tables MUST be interpreted Must Understand Column Meaning and Codes EXPLAIN Won't Tell You Everything Run Time Access Enhancements Referential Integrity Access to LOB values Slide 33 of 51

34 The PLAN_TABLE COLUMN COLUMN NAME NAME QUERYNO QUERYNO QBLOCKNO QBLOCKNO APPLNAME APPLNAME PROGNAME PROGNAME PLANNO PLANNO METHOD METHOD CREATOR CREATOR TNAME TNAME TABNO TABNO ACCESSTYPE ACCESSTYPE MATCHCOLS MATCHCOLS ACCESSCREATOR ACCESSCREATOR ACCESSNAME ACCESSNAME INDEXONLY INDEXONLY 25 Column Format (V2.1) Index name and creator Join Method 0, 0, 1, 1, 2, 2, 3, 3, 4 Type of Access Scan or Index R, R, I, I, I1, I1, IN, IN, N, N, M, M, MX, MX, MI, MI, MU, MU, MH, MH, A, A, DI, DI, DU, DU, DX, DX, E, E, H, H, HN, HN, O, O, NR, NR, P, P, R, R, RW, RW, V, V, blank blank Matching Columns from Index Is a table space scan bad? And, is an index access good? It s not all magic!!!! Slide 34 of 51

35 MATCHCOLS X(C1, C2, C3, C4) C1=1 C2=2 ACCESSTYPE='I' and MATCHCOLS=3 C3>3 C4=4 C1=1 C3>2 ACCESSTYPE='I' and MATCHCOLS=1 C4=3 C5=1 ACCESSTYPE='I' and MATCHCOLS=0 Slide 35 of 51

36 The PLAN_TABLE 25 Column Format (V2.1) COLUMN NAME SORTN_UNIQ SORTN_JOIN SORTN_ORDERBY SORTN_GROUPBY SORTC_UNIQ SORTC_JOIN SORTC_ORDERBY SORTC_GROUPBY TSLOCKMODE TIMESTAMP REMARKS IS, IX, S, U, X, SIX, N -or - NS, NIS, NSS, SS Was Sorting Required Y=Yes - or - N=No Slide 36 of 51

37 The PLAN_TABLE COLUMN NAME 28 Column Format (V2.2) PREFETCH COLUMN_FN_EVAL MIXOPSEQ S, L, D, U, or Blank R, S, or Blank 30 Column Format (V2.3) COLUMN NAME VERSION COLLID For Packages Slide 37 of 51

38 The PLAN_TABLE 34 Column Format (V3) COLUMN NAME ACCESS_DEGREE ACCESS_PGROUP_ID JOIN_DEGREE JOIN_PGROUP_ID Parallelism COLUMN NAME 43 Column Format (V4) SORTC_PGROUP_ID SORTN_PGROUP_ID PARALLELISM_MODE MERGE_JOIN_COLUMNS CORRELATION_NAME PAGE_RANGE JOIN_TYPE GROUP_MEMBER IBM_SERVICE_DATA I, C, X Y=Yes - or blank F, L, S, P, blank Slide 38 of 51

39 The PLAN_TABLE COLUMN NAME WHEN_OPTIMIZE QBLOCK_TYPE BIND_TIME 46 Column Format (V5) Blank, B, R See List CREATE DDL in Member DSNTESC in DSNSAMP COLUMN NAME OPTHINT HINT_USED PRIMARY_ACCESSTYPE COLUMN NAME PARENT_QBLOCKNO TABLE_TYPE 49 Column Format (V6) Hint identifier Hint identifier Blank, D, T 51 Column Format (V7) F, Q, T, W, M, B, C, I, R, S SELECT INSERT UPDATE MERGE DELETE SELUPD DELCUR UPDCUR CORSUB TRUNCA NCOSUB TABLEX TRIGGER UNION UNIONA INTERS INTERA EXCEPT EXCEPTA PRUNED Slide 39 of 51

40 The PLAN_TABLE 58 Column Format (V8) COLUMN NAME TABLE_ENCODE TABLE_SCCSID TABLE_MCCSID TABLE_DCCSID ROUTINE_ID E EBCDIC A ASCII U Unicode M multi CCSID sets 59 Column Format (DB2 9) COLUMN NAME CTEREF STMTTOKEN PARENT_PLANNO Slide 40 of 51

41 The PLAN_TABLE 64 Column Format (DB2 10) COLUMN NAME BIND_EXPLAIN_ONLY SECTNOI EXPLAIN_TIME MERGC MERGN Y if EXPLAIN(ONLY) specified on BIND/REBIND Static SQL only Same as SECTNO in SYSSTMT/SYSPACKSTMT Replaces TIMESTAMP and BIND_TIME Y (yes), N (no) Slide 41 of 51

42 Plan_Table Access Version 8 allows the use of an ALIAS to access PLAN_TABLE CREATE ALIAS WILLIE.PLAN_TABLE FOR PRODUCTION.PLAN_TABLE Willie runs query EXPLAIN PLAN FOR SELECT. Slide 42 of 51

43 Ordering PLAN_TABLE Data PLAN_TABLE should be sorted by: EXPLAIN_TIME When EXPLAIN information was captured QUERYNO QUERYNO clause on EXPLAIN QUERYNO clause on a DML statement Line number from source QBLOCKNO Query position in statement 1 is outer most query block PLANNO Execution order of steps MIXOPSEQ Step order for multiple index operations TIMESTAMP & BIND_TIME have been deprecated. Use EXPLAIN_TIME. Slide 43 of 51

44 Which Rows Do You Want? Search for Plans: APPLNAME = the application plan name Search for Packages: PROGNAME = the package name COLLID = the collection id VERSION = the version identifier Only applies to rows produced from some kind of BIND Slide 44 of 51

45 Querying from Plan/Package For Plans: SELECT * FROM YOUR_PLAN_TABLE WHERE APPLNAME = application_plan_name' ORDER BY EXPLAIN_TIME, QUERYNO, QBLOCKNO, PLANNO, MIXOPSEQ; For Packages: SELECT * FROM YOUR_PLAN_TABLE WHERE PROGNAME = package_name' AND COLLID = collection_id' AND VERSION = version_identifier' ORDER BY EXPLAIN_TIME, QUERYNO, QBLOCKNO, PLANNO,MIXOPSEQ; Slide 45 of 51

46 SELECT A.BIND_TIME,',', A.QUERYNO,',', A.QBLOCKNO,',', A.PLANNO,',', METHOD AS MTHD,',', SUBSTR(A.CREATOR,1,14) AS CREATOR,',', SUBSTR(TNAME,1,14) AS TABLENAME,',', BIGINT(C.CARDF) TCARDF, ',', BIGINT(C.NPAGESF) TNPAGESF, ',', C.PCTPAGES TPCTPAGES, ',', TABNO,',', B.PROCMS, ',', B.PROCSU, ',', B.COST_CATEGORY, ',', SUBSTR(B.REASON,1,25) AS REASON,',', ACCESSTYPE AS ACCTYP,',', MATCHCOLS AS MCOLS,',', SUBSTR(ACCESSCREATOR,1,14) AS ACCSSCRTR,',', SUBSTR(ACCESSNAME,1,14) AS ACCSSNM,',', INDEXONLY AS INDXO,',', SORTN_UNIQ AS SN_U,',', SORTN_JOIN AS SN_J,',', SORTN_ORDERBY AS SN_O,',', SORTN_GROUPBY AS SN_G,',', SORTC_UNIQ AS SC_U,',', SORTC_JOIN AS SC_J,',', SORTC_ORDERBY AS SC_O,',', SORTC_GROUPBY AS SC_G,',', TSLOCKMODE,',, PREFETCH AS PFTCH,',', COLUMN_FN_EVAL AS COL_FN_E,',', MIXOPSEQ,',', SUBSTR(A.VERSION,1,14) AS VERSION,',', SUBSTR(A.COLLID,1,14) AS COLLID,',', ACCESS_DEGREE AS ACC_D,',', ACCESS_PGROUP_ID AS ACC_PGID,',', JOIN_DEGREE AS JN_D,',', JOIN_PGROUP_ID AS JN_PGID,',', SORTC_PGROUP_ID AS SC_PGID,',', SORTN_PGROUP_ID AS SN_PGID,',', PARALLELISM_MODE AS P_ISM_MD,',', MERGE_JOIN_COLS AS M_J_COL,',', SUBSTR(CORRELATION_NAME,1,10) AS CORL_NM,',', PAGE_RANGE,',', JOIN_TYPE,',', SUBSTR(A.GROUP_MEMBER,1,8) AS GRP_MBR,',', WHEN_OPTIMIZE,',', QBLOCK_TYPE,',', PRIMARY_ACCESSTYPE AS PRI_ACCES_TYPE,',', PARENT_QBLOCKNO,',', PARENT_PLANNO,',', TABLE_TYPE,',', TABLE_ENCODE,',', CTEREF,',', SUBSTR (STMTTOKEN,1,14) AS STMTTOKEN,',', Slide 46 of 51

47 A.EXPLAIN_TIME,',', MERGC,',',MERGN,',', (CASE SUBSTR(HEX(SUBSTR(IBM_SERVICE_DATA,17,1)),1,1) WHEN 'A' THEN 10 WHEN 'B' THEN 11 WHEN 'C' THEN 12 WHEN 'D' THEN 13 WHEN 'E' THEN 14 WHEN 'F' THEN 15 ELSE INTEGER(SUBSTR(HEX(SUBSTR(IBM_SERVICE_DATA,17,1)),1,1)) END * POWER (16, 3) + CASE SUBSTR(HEX(SUBSTR(IBM_SERVICE_DATA,17,1)),2,1) WHEN 'A' THEN 10 WHEN 'B' THEN 11 WHEN 'C' THEN 12 WHEN 'D' THEN 13 WHEN 'E' THEN 14 WHEN 'F' THEN 15 ELSE INTEGER(SUBSTR(HEX(SUBSTR(IBM_SERVICE_DATA,17,1)),2,1)) END * POWER (16, 2) + CASE SUBSTR(HEX(SUBSTR(IBM_SERVICE_DATA,18,1)),1,1) WHEN 'A' THEN 10 WHEN 'B' THEN 11 WHEN 'C' THEN 12 WHEN 'D' THEN 13 WHEN 'E' THEN 14 WHEN 'F' THEN 15 ELSE INTEGER(SUBSTR(HEX(SUBSTR(IBM_SERVICE_DATA,18,1)),1,1)) END * POWER (16, 1) + CASE SUBSTR(HEX(SUBSTR(IBM_SERVICE_DATA,18,1)),2,1) WHEN 'A' THEN 10 WHEN 'B' THEN 11 WHEN 'C' THEN 12 WHEN 'D' THEN 13 WHEN 'E' THEN 14 WHEN 'F' THEN 15 ELSE INTEGER(SUBSTR(HEX(SUBSTR(IBM_SERVICE_DATA,18,1)),2,1)) END * POWER (16, 0)) AS NUMCP_EXP,',', HEX(SUBSTR(IBM_SERVICE_DATA,23,1)) AS REASON,',', HEX(SUBSTR(IBM_SERVICE_DATA,25,2)) AS CPU,',', HEX(SUBSTR(IBM_SERVICE_DATA,69,4)) AS CPUSPEED,',', HEX(SUBSTR(IBM_SERVICE_DATA,13,4)) AS RIDPOOL,',', HEX(SUBSTR(IBM_SERVICE_DATA,9,4)) AS SORT_POOL_SIZE_HEX, SUBSTR(IBM_SERVICE_DATA,67,7) APAR,',', IBM_SERVICE_DATA FROM WILLIE.PLAN_TABLE A, DSN_STATEMNT_TABLE B, SYSIBM.SYSTABLES C WHERE C.DBNAME = 'BID1TB' AND C.NAME = A.TNAME AND C.CREATOR = A.CREATOR AND C.TYPE = 'T' AND A.QUERYNO = ORDER BY A.EXPLAIN_TIME, A.QUERYNO,A.QBLOCKNO, A.PLANNO, A.MIXOPSEQ ; Slide 47 of 51

48 Using OPTHINT Create and index on PLAN_TABLE Key = QUERYNO, APPLNAME, PROGNAME, VERSION, COLLID, OPTHINT On installation panel DSNTIP4 Set OPTIMIZATION HINTS to YES Update Plan_Table Set OPTHINT column to some_value Tell DB2 to use the hint EXPLAIN(YES) and OPTHINT( some_value ) or SET CURRENT OPTIMIZATION HINT =' some_value '; Check Plan_Table and see if hint was used Slide 48 of 51

49 References Please Visit My DB2 for z/os Blog Slide 49 of 51

50 Thank you! Slide 50 of 51

51 Willie Favero DB2 SME Data Warehousing for System z Swat Team IBM Silicon Valley Laboratory Slide 51 of 51

52 Joins METHOD 0 - First table accessed 1 - Nested Loop Join 2 - Merge Scan Join 3 - Sort 4 - Hybrid Join JOIN_TYPE F - Full Outer Join L - Left Outer Join S - Star Join blank - Inner Join or No Join Slide 52 of 51

53 Join Processing TABLE T1 TABLE T2 INNER INNER TABLES TABLES or or NEW NEW TABLES TABLES Composite TABLE T3 OUTER OUTER TABLES TABLES or or COMPOSITE COMPOSITE TABLES TABLES Composite Slide 53 of 51

54 Nested Loop Join Outer Table Inner Table Small size Low filter factor TBNAME ACCESS_TYPE MATCHCOLS METHOD = 0 - composite table outer METHOD = 1 new table inner table SORTN_JOIN SORTC_JOIN Small Highly clustered Slide 54 of 51

55 Merge Scan Join A A A B B B D D E E A A B B B C D D E F TBNAME ACCESS_TYPE MATCHCOLS METHOD = 0 - composite table outer METHOD = 1 new table inner table SORTN_JOIN SORTC_JOIN Slide 55 of 51

56 DSN_STATEMNT_TABLE Same as Plan Table Reason for using Category B Column_name QUERYNO APPLNAME PROGNAME COLLID GROUP_MEMBER EXPLAIN_TIME STMT_TYPE COST_CATEGORY PROCMS PROCSU REASON SELECT INSERT UPDATE DELETE SELUPD DELCUR UPDCUR A or B (next slide) Milli-seconds Service Units Cost estimates for static and dynamic SELECT,INSERT,UPDATE and DELETE statements Slide 56 of 51

57 Cost Category Category B includes Uses UDF s Table has a trigger defined UPDATE, INSERT or DELETE WHERE predicate uses host variable, parameter marker or special register Subselect contains a HAVING clause Cardinality stats are zero RI constraint with DELETE rule of CASCADE or SET NULL Category A includes everything else Slide 57 of 51

58 DSN_FUNCTION_TABLE column name How DB2 resolves a function QUERYNO QBLOCKNO APPLNAME PROGNAME COLLID GROUP_MEMBER EXPLAIN_TIME SCHEMA_NAME FUNCTION_NAME SPEC_FUNC_NAME FUNCTION_TYPE VIEW_CREATOR VIEW_NAME PATH FUNCTION_TEXT Same as Plan Table Timestamp SU Scalar or TU - Table 1 st 100 bytes of Function text If function is referenced in view definition Slide 58 of 51

59 My Favorite Two??? Terms GETPAGE I/O How DB2 satisfies your request for data from a page What DB2 uses if page needed is NOT in the buffer pool Slide 59 of 51

60 The Cost of Doing Business milliseconds per random read. Why? DB2 OS/390 Services C h a n n e l C o n t r o l l e r C a c h e 1. Seek 2. Rotational Delay 3. Data Transfer Slide 60 of 51

61 How Much Prefetch? 4KB buffers 32 pages (64) 8KB 500+ buffers 16 pages (32) 16KB 250+ buffers 8 pages (16) 32KB 100+ buffers 4 pages (8) For an index scan that accesses 8 or more consecutive data pages, DB2 requests sequential prefetch at bind time. The index must have a cluster ratio of 80% or higher. Both data pages and index pages are prefetched. 600 prefetch engines (SRB tasks) Slide 61 of 51

62 Sequential Prefetch Sync XP Async P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P X Async P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P P X Look aside Buffer Pool Slide 62 of 51

63 Watch Out For Out of sequence index pages Data rows on the wrong page due to relocation (VarChar rows) Mis-match to SEGSIZE Slide 63 of 51

64 Relational Scans Read Segmented Table Space Read Simple Table Space QUERYNO TNAME ACCESSTYPE MATCHCOLS ACCESSNAME PRE 1 TAB1 R 0 S Slide 64 of 51

65 List Prefetch Access index pages Process only RID DB2 can apply index screening with multi column index key Sort RID in data page sequence Careful of high RID processing cost RID number restrictions 25% of table is accessed RID s take more than 50% of RID Pool Index order is not maintained Fetch data pages in (skip) sequential order using prefetch Slide 65 of 51

66 List-Prefetch Root Non-Leaf Non-Leaf Non-Leaf Non-Leaf Non-Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf RID SORT Data Data Data Data Data Data Prefetch Benefit To Data With Low Cluster Ratio Slide 66 of 51

67 Sequential Detection 5 of 8 pages read sequentially First determine if the page is sequential 8 pages tracked within P/2 (P=32?) Checking for 5 of 8 hits within the next 8 pages Show up as an index access in Explain. But just watch!!!!!!! Slide 67 of 51

68 Matching Index Scan INDEX ON T(C1,C2,C3) SELECT * FROM T WHERE C1 = 1 AND C2 = 5 AND C3 = 3 ; Root EXPLAIN ACCESSTYPE=I MATCHCOLS>0 How many columns from primary key were used for access? Non-Leaf Non-Leaf Non-Leaf Non-Leaf Non-Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf Data Data Data Data Data Data Slide 68 of 51

69 Matching Index Scan INDEX ON T(C1,C2,C3,C4) SELECT * FROM T WHERE C1 = 1 AND C4 = 5; Root Non-Leaf Non-Leaf Non-Leaf Non-Leaf Non-Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf Data Data Data Data Data Data IN-List Index Scan - Special Case of Matching Index Scan Slide 69 of 51

70 Non-Matching Index Scan Root Non-Leaf Non-Leaf Non-Leaf Non-Leaf Non-Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf Data Data Data Data Data Data Slide 70 of 51

71 List-Prefetch Root Non-Leaf Non-Leaf Non-Leaf Non-Leaf Non-Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf RID SORT Data Data Data Data Data Data Prefetch Benefit To Data With Low Cluster Ratio Slide 71 of 51

72 Multiple Index Access SELECT * FROM SAMPLE WHERE ( COCODE = L05 OR AVGSAL < ) AND COLLEGE = 03 COCODE COLLEGE AVGSAL Rids For Data Access Slide 72 of 51

73 MIA EXPLAIN QUERY NO TNAME ACCESS NAME ACCESS TYPE IX O MATCH COLS PRE COMP UJOG NEW UJOG MIXOP SEQ 0002 SAMPLE M L NNNN NNNN SAMPLE X1 MX y 01 NNNN NNNN SAMPLE X2 MX 01 S NNNN NNNN SAMPLE MU NNNN NNNN SAMPLE X3 MX y 01 S NNNN NNNN SAMPLE MI NNNN NNNN 5 Indexes: X1 on SAMPLE(COCODE) X2 on SAMPLE(AVGSAL) X3 on SAMPLE(COLLEGE) Slide 73 of 51

74 Index Only Access EXPLAIN PLAN SET QUERYNO = 1234 FOR DELETE FROM SOME_TABLE WHERE SOME_COL = Index XX1 on SOME_COL - CLUSTERRATION = 0% QUERYNO TNAME ACCESSNAME ACCESSTYPE INDEXONLY 1 TAB1 XX1 I Y Slide 74 of 51

75 Other Index Access One Fetch Access Best Possible Access. Only one row retrieved. ACCESSTYPE of 'I1' Equal Unique Index Second only to one-fetch access. A fully matched unique index where all matching predicates are equal. Index Look-Aside (always used) Next page... Slide 75 of 51

76 Index Only INDEX C1,C2,C3 Index Root SELECT C1,C2,C3 FROM Non Leaf s Leaf s X X X X X X... Data s Slide 76 of 51

77 Index Look-Aside Index Root Cached key Non Leaf s Cached key Leaf s Data s rows... Slide 77 of 51

78 Index Lookaside Root Non-Leaf Non-Leaf Non-Leaf Non-Leaf Non-Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf Leaf Data Data Data Data Data Data Slide 78 of 51

79 Direct Row Access What is it? No index, no table space scan When might it be used? ROWID column must be present Stage 1 processing Search condition on SELECT, UPDATE, or DELETE When will it Not be used? Row has moved Parallelism RID List processing Should have index on ROWID column just in case Slide 79 of 51

80 SELECT With ORDER BY... QUERY NO TNAME ACCESS NAME ACCESS TYPE IX O MATCH COLS PRE COMP UJOG NEW UJOG PLAN NO 0108 SAMPLE R S NNNN NNNN NNYN NNNN SAMPLE R S NNNN NNNN NNYN NNNN SAMPLE X1 I N 00 S NNNN NNNN 1 EXPLAIN PLAN SET QUERYNO = 108 FOR SELECT * FROM SAMPLE ORDER BY SOME_COL X1 CLUSTERRATIO Increased - 75%, 85%, and 90% Slide 80 of 51

81 SELECT With ORDER BY... QUERY NO TNAME ACCESS NAME ACCESS TYPE IX O MATCH COLS PRE COMP UJOG NEW UJOG PLAN NO 0114 TABBY XX3 I N 01 L NNNN NNNN 1 EXPLAIN PLAN SET QUERYNO = 114 FOR SELECT * FROM SAMPLE WHERE SOME_COL LIKE? The? acts like a host variable Slide 81 of 51

82 What Should Happen? SELECT col1, col2, col3 FROM some_table WHERE col2 > :host_var DB2 will estimate number of rows returned based on Catalog statistics some_table Entire Answer Set is Materialized Sorted Rows No index exist Application needs only 10 Rows Slide 82 of 51

83 What Should Happen? SELECT col1, col2, col3 FROM some_table Where col2 > :host_var OPTIMIZE FOR 10 ROWS DB2 will estimate number of rows returned based on value supplied by OPTIMZE Clause some_table What About OPTIMZE FOR 1 ROW? Rows Returned What about today? First rows are returned Quickly However The Entire Answer Could Be Returned Application needs only 10 Rows Control Number of rows for real? Slide 83 of 51

84 OPTIMZE FOR... QUERY NO TNAME ACCESS NAME ACCESS TYPE IX O MATCH COLS PRE COMP UJOG NEW UJOG PLAN NO 0120 S_TAB XX3 I 01 L NNNN NNNN S_TAB XX3 I N 01 NNNN NNNN 1 EXPLAIN PLAN SET QUERY = 120 FOR SELECT * FROM S_TAB WHERE COL1 = STUFF Index is not on COL1 EXPLAIN PLAN SET QUERY = 121 FOR SELECT * FROM S_TAB WHERE COL1 = STUFF OPTIMZE FOR 1 ROW Slide 84 of 51

85 OPTIMZE FOR... QUERY NO TNAME ACCESS NAME ACCESS TYPE IX O MATCH COLS PRE COMP UJOG NEW UJOG PLAN NO 0122 S_TAB XX3 I 01 L NNNN NNNN NNYN NNNN S_TAB XX3 I N 01 L NNNN NNNN 1 EXPLAIN PLAN SET QUERY = 122 FOR SELECT * FROM S_TAB ORDER BY LAST_COL EXPLAIN PLAN SET QUERY = 123 FOR SELECT * FROM S_TAB ORDER BY LAST_COL OPTIMZE FOR 1 ROW Slide 85 of 51

86 FETCH FIRST n ROWS ONLY Only the number of rows specified Slide 86 of 51

87 Partition Scan Change Prior to V4 V Read V5.1 and above Read PAGE_RANGE = Y --- Partition Scan Slide 87 of 51

88 EXPLAIN For Parallelism Buffer Pool Table Space Scan Degree and Access Parallel Group TNAME METHOD ACCESS_DEGREE ACCESS_PGROUP_ID JOIN_DEGREE JOIN_PGROUP_ID TAB Slide 88 of 51

89 EXPLAIN For Parallelism TAB1 TAB2 TNAME METHOD ACCESS_DEGREE ACCESS_PGROUP_ID JOIN_DEGREE JOIN_PGROUP_ID TAB1 TAB Slide 89 of 51

90 EXPLAIN For Parallelism TAB1 DB2 Sort Facility S o r t Physically Repartitioned TAB2 TNAME METHOD ACCESS_DEGREE ACCESS_PGROUP_ID JOIN_DEGREE JOIN_PGROUP_ID TAB1 TAB Slide 90 of 51

91 EXPLAIN For Parallelism Table Space Scan TNAME METHOD ACCESS_DEGREE ACCESS_PGROUP_ID JOIN_DEGREE JOIN_PGROUP_ID TAB Host Variables Slide 91 of 51

92 Choosing Parallelism DEGREE Option on BIND Package/Plan SET CURRENT DEGREE Special Register Slide 92 of 51

93 The Numbers Accounting Trace ACE Address Originating ACE Address Number of parallel task created CP time Getpages Sequential prefetch Originating Task One record for each Parallel Task ACE Address Originating ACE Address Number of parallel task created CP time Getpages Sequential prefetch Slide 93 of 51

94 Sysplex Query Parallelism Trace Accounting Trace Member name ACE Address Query originating member Originating ACE Address Number of parallel task created LUWID One record for each Parallel Task On different members Originating Task + More... Member name ACE Address Query originating member Originating ACE Address Number of parallel task created LUWID + More... Slide 94 of 51

95 More Numbers IFCID Issued for each parallel group Pgm name, statement number, and QBLOCKNO Parallel group number Planned bind, planned run, and actual degrees Reason for runtime degree Parallel Mode Low/High page/keey range for logical partitions Status Repeats for each logical partition Slide 95 of 51

96 Even More Numbers IFCID Issued for each parallel group Pipe creation/termination times Rows processed Subpipe creation/termination times Rows processed per subpipe Repeats for each subpipe Slide 96 of 51

97 The Last of the Numbers IFCID Issued for each parallel group - not used for I/O parallelism Group creation/termination times Parallel task creation/termination times CP seconds DB2 member and service units added in in Version 5 Repeats for each parallel task All three trace records are available from a Class 8 Performance Trace Slide 97 of 51

98 Some Trace Information (for the daring!) Performance Class 8 IFCID MVS Enclave services not available IFCID Additional Subpipe information IFCID Parallel group number and task create and terminate times Statistics Class 1 IFCID Extended fields to include CPU parallelism Accounting Class 1 IFCID Added number of parallel task Performance Class 3 IFCID disabled by RLF Slide 98 of 51

99 Order By Question QUERY NO TNAME ACCESS NAME ACCESS TYPE IX O MATCH COLS PRE COMP UJOG NEW UJOG PLAN NO 0106 S_TAB R S NNNN NNNN NNYN NNNN S_TAB XX4 I N 00 S NNNN NNNN 1 EXPLAIN PLAN SET QUERY = 106 FOR SELECT * FROM S_TAB ORDER BY LAST_COL EXPLAIN PLAN SET QUERY = 107 FOR SELECT * FROM S_TAB ORDER BY LAST_COL DESC XX4 has CLUSTERRATION = 99% Slide 99 of 51

100 Low Key Cardinality QUERY NO TNAME ACCESS NAME ACCESS TYPE IX O MATCH COLS PRE COMP UJOG NEW UJOG PLAN NO 0124 S_TAB R S NNNN NNNN S_TAB XX6 I N 01 L NNNN NNNN S_TAB R S NNNN NNNN 1 EXPLAIN PLAN SET QUERY = 124 FOR SELECT * FROM S_TAB WHERE SEX = M EXPLAIN PLAN SET QUERY = 125 FOR SELECT * FROM S_TAB WHERE SEX = M AND B_COL > 5000 QUERYNO=0126 WHERE SEX = M OR B_COL > 5000 Slide 100 of 51

101 Looks Good But... QUERY NO TNAME ACCESS NAME ACCESS TYPE IX O MATCH COLS PRE COMP UJOG NEW UJOG TS LOCK MODE 0124 S_TAB R S NNNN NNNN S EXPLAIN PLAN SET QUERY = 124 FOR SELECT * FROM S_TAB WHERE SEX = M Slide 101 of 51

102 Make it Work Correctly Catalog Statistics Must reflect production values Watch out for Default Values Dynamic vs Static Explain Uses "?" instead of actual value? always matches Slide 102 of 51

103 If EXPLAIN YES, Then Look! Validate Access Paths Selected Identify Ineffective SQL Index verses non-index access Stage 1 vs Stage 2 Predicates At Development Time Examine output form each SQL statement Check out all SQL in plan for Unit Test At Maintenance Time Need history EXPLAIN output Examine every statement Stuff changes Slide 103 of 51

104 What Else Can We Do With It? Workload Profile Will help understand SQL profile Queries that never run Scan of a small table Should maintain historical Plan_Tables Subset of Production SQL Develop Code/Queries Slide 104 of 51

105 The End Slide 105 of 51

106 References Please Visit My DB2 for z/os Blog Slide 106 of 51

107 Thank you! Slide 107 of 51

108 Willie Favero DB2 SME Data Warehousing for System z Swat Team IBM Silicon Valley Laboratory Slide 108 of 51