Performance and Tuning Guide: Volume 1 - Basics

Transcription

1 Performance and Tuning Guide: Volume 1 - Basics Adaptive Server Enterprise 12.5

2 DOCUMENT ID: LAST REVISED: July 2001 Copyright by Sybase, Inc. All rights reserved. This publication pertains to Sybase database management software and to any subsequent release until otherwise indicated in new editions or technical notes. Information in this document is subject to change without notice. The software described herein is furnished under a license agreement, and it may be used or copied only in accordance with the terms of that agreement. To order additional documents, U.S. and Canadian customers should call Customer Fulfillment at (800) , fax (617) Customers in other countries with a U.S. license agreement may contact Customer Fulfillment via the above fax number. All other international customers should contact their Sybase subsidiary or local distributor. Upgrades are provided only at regularly scheduled software release dates. No part of this publication may be reproduced, transmitted, or translated in any form or by any means, electronic, mechanical, manual, optical, or otherwise, without the prior written permission of Sybase, Inc. Sybase, the Sybase logo, ADA Workbench, Adaptable Windowing Environment, Adaptive Component Architecture, Adaptive Server, Adaptive Server Anywhere, Adaptive Server Enterprise, Adaptive Server Enterprise Monitor, Adaptive Server Enterprise Replication, Adaptive Server Everywhere, Adaptive Server IQ, Adaptive Warehouse, AnswerBase, Anywhere Studio, Application Manager, AppModeler, APT Workbench, APT-Build, APT-Edit, APT-Execute, APT-FORMS, APT-Translator, APT-Library, Backup Server, ClearConnect, Client-Library, Client Services, Data Pipeline, Data Workbench, DataArchitect, Database Analyzer, DataExpress, DataServer, DataWindow, DB-Library, dbqueue, Developers Workbench, Direct Connect Anywhere, DirectConnect, Distribution Director, E-Anywhere, E-Whatever, Embedded SQL, EMS, Enterprise Application Studio, Enterprise Client/Server, Enterprise Connect, Enterprise Data Studio, Enterprise Manager, Enterprise SQL Server Manager, Enterprise Work Architecture, Enterprise Work Designer, Enterprise Work Modeler, EWA, Financial Fusion, Financial Fusion Server, Gateway Manager, ImpactNow, InfoMaker, Information Anywhere, Information Everywhere, InformationConnect, InternetBuilder, iscript, Jaguar CTS, jconnect for JDBC, KnowledgeBase, MainframeConnect, Maintenance Express, MAP, MDI Access Server, MDI Database Gateway, media.splash, MetaWorks, MySupport, Net-Gateway, Net-Library, ObjectConnect, ObjectCycle, OmniConnect, OmniSQL Access Module, OmniSQL Toolkit, Open Client, Open ClientConnect, Open Client/Server, Open Client/Server Interfaces, Open Gateway, Open Server, Open ServerConnect, Open Solutions, Optima++, PB-Gen, PC APT Execute, PC DB-Net, PC Net Library, Power++, power.stop, PowerAMC, PowerBuilder, PowerBuilder Foundation Class Library, PowerDesigner, PowerDimensions, PowerDynamo, PowerJ, PowerScript, PowerSite, PowerSocket, Powersoft, PowerStage, PowerStudio, PowerTips, Powersoft Portfolio, Powersoft Professional, PowerWare Desktop, PowerWare Enterprise, ProcessAnalyst, Report Workbench, Report-Execute, Replication Agent, Replication Driver, Replication Server, Replication Server Manager, Replication Toolkit, Resource Manager, RW-DisplayLib, RW-Library, S-Designor, SDF, Secure SQL Server, Secure SQL Toolset, Security Guardian, SKILS, smart.partners, smart.parts, smart.script, SQL Advantage, SQL Anywhere, SQL Anywhere Studio, SQL Code Checker, SQL Debug, SQL Edit, SQL Edit/TPU, SQL Everywhere, SQL Modeler, SQL Remote, SQL Server, SQL Server Manager, SQL SMART, SQL Toolset, SQL Server/CFT, SQL Server/DBM, SQL Server SNMP SubAgent, SQL Station, SQLJ, STEP, SupportNow, Sybase Central, Sybase Client/Server Interfaces, Sybase Financial Server, Sybase Gateways, Sybase MPP, Sybase SQL Desktop, Sybase SQL Lifecycle, Sybase SQL Workgroup, Sybase User Workbench, SybaseWare, Syber Financial, SyberAssist, SyBooks, System 10, System 11, System XI (logo), SystemTools, Tabular Data Stream, Transact-SQL, Translation Toolkit, UNIBOM, Unilib, Uninull, Unisep, Unistring, URK Runtime Kit for UniCode, Viewer, Visual Components, VisualSpeller, VisualWriter, VQL, WarehouseArchitect, Warehouse Control Center, Warehouse Studio, Warehouse WORKS, Watcom, Watcom SQL, Watcom SQL Server, Web Deployment Kit, Web.PB, Web.SQL, WebSights, WebViewer, WorkGroup SQL Server, XA-Library, XA-Server and XP Server are trademarks of Sybase, Inc. 3/01 Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. All other company and product names used herein may be trademarks or registered trademarks of their respective companies. Use, duplication, or disclosure by the government is subject to the restrictions set forth in subparagraph (c)(1)(ii) of DFARS for the DOD and as set forth in FAR (a)-(d) for civilian agencies. Sybase, Inc., 6475 Christie Avenue, Emeryville, CA

3 Contents About This Book... xxxi CHAPTER 1 Overview... 1 Good performance... 1 Response time... 1 Throughput... 2 Designing for performance... 2 Tuning performance... 2 Tuning levels... 3 Identifying system limits... 8 Setting tuning goals... 8 Analyzing performance... 9 Normal Forms Locking Special Considerations CHAPTER 2 Networks and Performance Introduction Potential performance problems Basic questions on network performance Techniques summary Using sp_sysmon while changing network configuration How Adaptive Server uses the network Changing network packet sizes Large versus Default packet sizes for user connections Number of packets is important Evaluation tools with Adaptive Server Evaluation tools outside of Adaptive Server Server-based techniques for reducing network traffic Impact of other server activities Single user versus multiple users Improving network performance Isolate heavy network users iii

4 Contents Set tcp no delay on TCP networks Configure multiple network listeners CHAPTER 3 Using Engines and CPUs Background concepts How Adaptive Server processes client requests Client task implementation Single-CPU process model Scheduling engines to the CPU Scheduling tasks to the engine Execution task scheduling Adaptive Server SMP process model Scheduling engines to CPUs Scheduling Adaptive Server tasks to engines Multiple network engines Task priorities and run queues Processing scenario Housekeeper task improves CPU utilization Side effects of the housekeeper task Configuring the housekeeper task Measuring CPU usage Single-CPU machines Determining when to configure additional engines Taking engines offline Enabling engine-to-cpu affinity Multiprocessor application design guidelines CHAPTER 4 Distributing Engine Resources Algorithm for successfully distributing engine resources Algorithm guidelines Environment analysis and planning Performing benchmark tests Setting goals Results analysis and tuning Monitoring the environment over time Manage preferred access to resources Types of execution classes Predefined execution classes User-Defined execution classes Execution class attributes Base priority Time slice Task-to-engine affinity iv

5 Contents Setting execution class attributes Assigning execution classes Engine groups and establishing task-to-engine affinity How execution class bindings affect scheduling Setting attributes for a session only Getting information Rules for determining precedence and scope Multiple execution objects and ECs Resolving a precedence conflict Examples: determining precedence Example scenario using precedence rules Planning Configuration Execution characteristics Considerations for Engine Resource Distribution Client applications: OLTP and DSS Adaptive Server logins: high-priority users Stored procedures: hot spots CHAPTER 5 Controlling Physical Data Placement Object placement can improve performance Symptoms of poor object placement Underlying problems Using sp_sysmon while changing data placement Terminology and concepts Guidelines for improving I/O ierformance Spreading data across disks to avoid I/O contention Isolating server-wide I/O from database I/O Keeping transaction logs on a separate disk Mirroring a device on a separate disk Creating objects on segments Using segments Separating tables and indexes Splitting large tables across devices Moving text storage to a separate device Partitioning tables for performance User transparency Partitioned tables and parallel query processing Improving insert performance with partitions Restrictions on partitioned tables Partition-related configuration parameters How Adaptive Server distributes partitions on devices Space planning for partitioned tables Read-only tables v

6 Contents Read-mostly tables Tables with random data modification Commands for partitioning tables alter table...partition syntax alter table...unpartition Syntax Changing the number of partitions Distributing data evenly across partitions Using parallel bcp to copy data into partitions Getting information about partitions Using bcp to correct partition balance Checking data distribution on devices with sp_helpsegment 101 Updating partition statistics Steps for partitioning tables Backing up the database after partitioning tables Table does not exist Table exists elsewhere in the database Table exists on the segment Special procedures for difficult situations Clustered indexes on large tables Alternative for clustered indexes Problems when devices for partitioned tables are full Adding disks when devices are full Adding disks when devices are nearly full Maintenance issues and partitioned tables Regular maintenance checks for partitioned tables CHAPTER 6 Database Design Basic design Physical database design for Adaptive Server Logical Page Sizes Normalization Levels of normalization Benefits of normalization First Normal Form Second Normal Form Third Normal Form Denormalizing for performance Risks Denormalization input Techniques Splitting tables Managing denormalized data Using triggers Using application logic vi

7 Contents Batch reconciliation CHAPTER 7 Data Storage Performance gains through query optimization Query processing and page reads Adaptive Server pages Page headers and page sizes Varying logical page sizes Data and index pages Large Object (LOB) Pages Extents Pages that manage space allocation Global allocation map pages Allocation pages Object allocation map pages How OAM pages and allocation pages manage object storage 146 Page allocation keeps an object s pages together sysindexes table and data access Space overheads Number of columns and size Number of rows per data page Maximum numbers Heaps of data: tables without clustered indexes Lock schemes and differences between heaps Select operations on heaps Inserting data into an allpages-locked heap table Inserting data into a data-only-locked heap table Deleting data from a heap table Updating data on a heap table How Adaptive Server performs I/O for heap operations Sequential prefetch, or large I/O Caches and object bindings Heaps, I/O, and cache strategies Select operations and caching Data modification and caching Asynchronous prefetch and I/O on heap tables Heaps: pros and cons Maintaining heaps Methods Transaction log: a special heap table CHAPTER 8 Indexing for Performance How indexes affect performance vii

8 Contents Detecting indexing problems Symptoms of poor indexing Index limits and requirements Choosing indexes Index keys and logical keys Guidelines for clustered indexes Choosing clustered indexes Candidates for nonclustered indexes Other indexing guidelines Choosing nonclustered indexes Choosing composite indexes Key order and performance in composite indexes Advantages and disadvantages of composite indexes Techniques for choosing indexes Choosing an index for a range query Adding a point query with different indexing requirements Index and statistics maintenance Dropping indexes that hurt performance Choosing space management properties for indexes Additional indexing tips Creating artificial columns Keeping index entries short and avoiding overhead Dropping and rebuilding indexes CHAPTER 9 How Indexes Work Types of indexes Index pages Index Size Clustered indexes on allpages-locked tables Clustered indexes and select operations Clustered indexes and insert operations Page splitting on full data pages Page splitting on index pages Performance impacts of page splitting Overflow pages Clustered indexes and delete operations Nonclustered indexes Leaf pages revisited Nonclustered index structure Nonclustered indexes and select operations Nonclustered index performance Nonclustered indexes and insert operations Nonclustered indexes and delete operations Clustered indexes on data-only-locked tables viii

9 Contents Index covering Covering matching index scans Covering nonmatching index scans Indexes and caching Using separate caches for data and index pages Index trips through the cache CHAPTER 10 Locking Configuration and Tuning Locking and performance Using sp_sysmon and sp_object_stats Reducing lock contention Additional locking guidelines Configuring locks and lock promotion thresholds Configuring Adaptive Server s lock limit Configuring the lock hashtable Setting lock promotion thresholds Choosing the locking scheme for a tablel Analyzing existing applications Choosing a locking scheme based on contention statistics Monitoring and managing tables after conversion Applications not likely to benefit from data-only locking CHAPTER 11 Using Locking Commands Specifying the locking scheme for a table Specifying a server-wide locking scheme Specifying a locking scheme with create table Changing a locking scheme with alter table Before and after changing locking schemes Expense of switching to or from allpages locking Sort performance during alter table Specifying a locking scheme with select into Controlling isolation levels Setting isolation levels for a session Syntax for query-level and table-level locking options Using holdlock, noholdlock, or shared Using the at isolation clause Making locks more restrictive Making locks less restrictive Readpast locking Cursors and locking Using the shared keyword Additional locking commands lock table Command ix

10 Contents Lock timeouts CHAPTER 12 Reporting on Locks Locking tools Getting information about blocked processes Viewing locks Viewing locks Intrafamily blocking during network buffer merges Deadlocks and concurrency Server-side versus application-side deadlocks Server task deadlocks Deadlocks and parallel queries Printing deadlock information to the error log Avoiding deadlocks Identifying tables where concurrency is a problem Lock management reporting CHAPTER 13 Setting Space Management Properties Reducing index maintenance Advantages of using fillfactor Disadvantages of using fillfactor Setting fillfactor values fillfactor examples Use of the sorted_data and fillfactor options Reducing row forwarding Default, minimum, and maximum values for exp_row_size Specifying an expected row size with create table Adding or changing an expected row size Setting a default expected row size server-wide Displaying the expected row size for a table Choosing an expected row size for a table Conversion of max_rows_per_page to exp_row_size Monitoring and managing tables that use expected row size 278 Leaving space for forwarded rows and inserts Extent allocation operations and reservepagegap Specifying a reserve page gap with create table Specifying a reserve page gap with create index Changing reservepagegap reservepagegap examples Choosing a value for reservepagegap Monitoring reservepagegap settings reservepagegap and sorted_data options to create index Using max_rows_per_page on allpages-locked tables x

11 Contents Reducing lock contention Indexes and max_rows_per_page select into and max_rows_per_page Applying max_rows_per_page to existing data CHAPTER 14 Memory Use and Performance How memory affects performance How much memory to configure Caches in Adaptive Server Procedure cache Getting information about the procedure cache size Procedure cache sizing Estimating stored procedure size Data cache Default cache at installation time Page aging in data cache Effect of data cache on retrievals Effect of data modifications on the cache Data cache performance Testing data cache performance Configuring the data cache to improve performance Commands to configure named data caches Tuning named caches Cache configuration goals Gather data, plan, and then implement Evaluating cache needs Large I/O and performance Reducing spinlock contention with cache partitions Cache replacement strategies and policies Named data cache recommendations Sizing caches for special objects, tempdb, and transaction logs. 316 Basing data pool sizes on query plans and I/O Configuring buffer wash size Overhead of pool configuration and binding objects Maintaining data cache performance for large I/O Diagnosing excessive I/O Counts Using sp_sysmon to check large I/O performance Speed of recovery Tuning the recovery interval Effects of the housekeeper task on recovery time Auditing and performance Sizing the audit queue Auditing performance guidelines xi

12 Contents CHAPTER 15 Determining Sizes of Tables and Indexes Why Object Sizes Are Important to Query Tuning Tools for Determining the Sizes of Tables and Indexes Effects of Data Modifications on Object Sizes Using optdiag to Display Object Sizes Advantages of optdiag Disadvantages of optdiag Using sp_spaceused to Display Object Size Advantages of sp_spaceused Disadvantages of sp_spaceused Using sp_estspace to Estimate Object Size Advantages of sp_estspace Disadvantages of sp_estspace Using Formulas to Estimate Object Size Factors That Can Affect Storage Size Storage Sizes for Datatypes Tables and Indexes Used in the Formulas Calculating Table and Clustered Index Sizes for Allpages-Locked Tables Calculating the Sizes of Data-Only-Locked Tables Other Factors Affecting Object Size Very Small Rows LOB Pages Advantages of Using Formulas to Estimate Object Size Disadvantages of Using Formulas to Estimate Object Size CHAPTER 16 Maintenance Activities and Performance Running reorg on tables and indexes Creating and maintaining indexes Configuring Adaptive Server to speed sorting Dumping the database after creating an index Creating an index on sorted data Maintaining index and column statistics Rebuilding indexes Creating or altering a database Backup and recovery Local backups Remote backups Online backups Using thresholds to prevent running out of log space Dump striping Minimizing recovery time Recovery order Bulk copy xii

13 Contents Parallel bulk copy Batches and bulk copy Slow bulk copy Improving bulk copy performance Replacing the data in a large table Adding large amounts of data to a table Using partitions and multiple bulk copy processes Impacts on other users Database consistency checker Using dbcc tune (cleanup) Determining the space available for maintenance activities Overview of space requirements Tools for checking space usage and space available Estimating the effects of space management properties If there is not enough space CHAPTER 17 Adaptive Server Optimizer Definition Steps in query processing Working with the optimizer Object sizes are important to query tuning Query optimization Factors examined during optimization Preprocessing can add clauses for optimizing Converting clauses to search argument equivalents Converting expressions into search arguments Search argument transitive closure Join transitive closure Predicate transformation and factoring Guidelines for creating search arguments Search arguments and useful indexes Search argument syntax How statistics are used for SARGS Using statistics on multiple search arguments Default values for search arguments SARGs using variables and parameters Join syntax and join processing How joins are processed When statistics are not available for joins Density values and joins Multiple column joins Search arguments and joins on a table Datatype mismatches and query optimization Overview of the datatype hierarchy and index issues xiii

14 Contents Datatypes for parameters and variables used as SARGs Compatible datatypes for join columns Suggestions on datatypes and comparisons Forcing a conversion to the other side of a join Splitting stored procedures to improve costing Basic units of costing CHAPTER 18 Advanced Optimizing Tools Special optimizing techniques Specifying optimizer choices Specifying table order in joins Risks of using forceplan Things to try before using forceplan Specifying the number of tables considered by the optimizer Specifying an index for a query Risks Things to try before specifying an index Specifying I/O size in a query Index type and large I/O When prefetch specification is not followed set prefetch on Specifying the cache strategy In select, delete, and update statements Controlling large I/O and cache strategies Getting information on cache strategies Enabling and disabling merge joins Enabling and disabling join transitive closure Suggesting a degree of parallelism for a query Query level parallel clause examples Concurrency optimization for small tables Changing locking scheme CHAPTER 19 Query Tuning Tools Overview How tools may interact Using showplan and noexec together noexec and statistics io How tools relate to query processing CHAPTER 20 Access Methods and Query Costing for Single Tables Table scan cost Cost of a scan on allpages-locked table xiv

15 Contents Cost of a scan on a data-only-locked tables From rows to pages How cluster ratios affect large I/O estimates Evaluating the cost of index access Query that returns a single row Query that returns many rows Range queries with covering indexes Range queries with noncovering indexes Costing for queries using order by Prefix subset and sorts Key ordering and sorts How the optimizer costs sort operations Allpages-locked tables with clustered indexes Sorts when index covers the query Sorts and noncovering indexes Access Methods and Costing for or and in Clauses or syntax in (values_list) converts to or processing Methods for processing or clauses How aggregates are optimized Combining max and min aggregates How update operations are performed Direct updates Deferred updates Deferred index inserts Restrictions on update modes through joins Optimizing updates Using sp_sysmon while tuning updates CHAPTER 21 Accessing Methods and Costing for Joins and Subqueries Costing and optimizing joins Processing Index density and joins Datatype mismatches and joins Join permutations Nested-loop joins Cost formula How inner and outer tables are determined Access methods and costing for sort-merge joins How a full-merge is performed How a right-merge or left-merge is performed How a sort-merge is performed Mixed example Costing for merge joins xv

16 Contents Costing for a full-merge with unique values Example: allpages-locked tables with clustered indexes Costing for a full-merge with duplicate values Costing sorts When merge joins cannot be used Use of worker processes Recommendations for improved merge performance Enabling and disabling merge joins At the server level At the session level Reformatting strategy Subquery optimization Flattening in, any, and exists subqueries Flattening expression subqueries Materializing subquery results Subquery introduced with an and clause Subquery introduced with an or clause Subquery results caching Optimizing subqueries or Clauses versus unions in joins CHAPTER 22 Parallel Query Processing Types of queries that can benefit from parallel processing Adaptive Server s worker process model Parallel query execution Returning results from parallel queries Types of parallel data access Hash-based table scans Partition-based scans Hash-based index scans Parallel processing for two tables in a join showplan messages Controlling the degree of parallelism Configuration parameters for controlling parallelism Using set options to control parallelism for a session Controlling parallelism for a query Worker process availability and query execution Other configuration parameters for parallel processing Commands for working with partitioned tables Balancing resources and performance CPU resources Disk resources and I/O Tuning example: CPU and I/O saturation Guidelines for parallel query configuration xvi

17 Contents Hardware guidelines Working with your performance goals and hardware guidelines Examples of parallel query tuning Guidelines for partitioning and parallel degree Experimenting with data subsets System level impacts Locking issues Device issues Procedure cache effects When parallel query results can differ Queries that use set rowcount Queries that set local variables Achieving consistent results CHAPTER 23 Parallel Query Optimization What is parallel query optimization? Optimizing for response time versus total work When is optimization performed? Overhead costs Factors that are not considered Parallel access methods Parallel partition scan Parallel clustered index partition scan (allpages-locked tables) 540 Parallel hash-based table scan Parallel hash-based index scan Parallel range-based scans Additional parallel strategies Summary of parallel access methods Selecting parallel access methods Degree of parallelism for parallel queries Upper limit Optimized degree Nested-loop joins Examples Runtime adjustments to worker processes Parallel query examples Single-table scans Multitable joins Subqueries Queries that require worktables union queries Queries with aggregates select into statements xvii

18 Contents Runtime adjustment of worker processes How Adaptive Server adjusts a query plan Evaluating the effect of runtime adjustments Recognizing and managing runtime adjustments Reducing the likelihood of runtime adjustments Checking runtime adjustments with sp_sysmon Diagnosing parallel performance problems Query does not run in parallel Parallel performance is not as good as expected Calling technical support for diagnosis Resource limits for parallel queries CHAPTER 24 Parallel Sorting Commands that benefits from parallel sorting Requirements and resources overview Overview of the parallel sorting strategy Creating a distribution map Dynamic range partitioning Range sorting Merging results Configuring resources for parallel sorting Worker process requirements for parallel sorts Worker process requirements for select query sorts Caches, sort buffers, and parallel sorts Disk requirements Recovery considerations Tools for observing and tuning sort behavior Using set sort_resources on Using sp_sysmon to tune index creation CHAPTER 25 Tuning Asynchronous Prefetch How asynchronous prefetch improves performance Improving query performance by prefetching pages Prefetching control mechanisms in a multiuser environment 603 Look-ahead set during recovery Look-ahead set during sequential scans Look-ahead set during nonclustered index access Look-ahead set during dbcc checks Look-ahead set minimum and maximum sizes When prefetch is automatically disabled Flooding pools I/O system overloads Unnecessary reads xviii

19 Contents Tuning Goals for Asynchronous Prefetch Commands for configuration Other Adaptive Server performance features Large I/O Fetch-and-discard (MRU) scans Parallel scans and large I/Os Special settings for asynchronous prefetch limits Setting limits for recovery Setting limits for dbcc Maintenance activities for high prefetch performance Eliminating kinks in heap tables Eliminating kinks in clustered index tables Eliminating kinks in nonclustered indexes Performance monitoring and asynchronous prefetch CHAPTER 26 tempdb Performance Issues How management of tempdb affects performance Main solution areas for tempdb performance Types and uses of temporary tables Truly temporary tables Regular user tables Worktables Initial allocation of tempdb Sizing the tempdb Placing tempdb Dropping the master Device from tempdb segments Using multiple disks for parallel query performance Binding tempdb to its own cache Commands for cache binding Temporary tables and locking Minimizing logging in tempdb With select into By using shorter rows Optimizing temporary tables Creating indexes on temporary tables Creating nested procedures with temporary tables Breaking tempdb uses into multiple procedures CHAPTER 27 Cursors and Performance Definition Set-oriented versus row-oriented programming Example Resources required at each stage xix

20 Contents Memory use and execute cursors Cursor modes Index use and requirements for cursors Allpages-locked tables Data-only-locked tables Comparing performance with and without cursors Sample stored procedure without a cursor Sample stored procedure with a cursor Cursor versus noncursor performance comparison Locking with read-only cursors Isolation levels and cursors Partitioned heap tables and cursors Optimizing tips for cursors Optimizing for cursor selects using a cursor Using union instead of or clauses or in lists Declaring the cursor s intent Specifying column names in the for update clause Using set cursor rows Keeping cursors open across commits and rollbacks Opening multiple cursors on a single connection CHAPTER 28 Introduction to Abstract Plans Definition Managing abstract plans Relationship between query text and query plans Limits of options for influencing query plans Full versus partial plans Creating a partial plan Abstract plan groups How abstract plans are associated with queries CHAPTER 29 Abstract Query Plan Guide Introduction Abstract plan language Identifying tables Identifying indexes Specifying join order Specifying the join type Specifying partial plans and hints Creating abstract plans for subqueries Abstract plans for materialized views Abstract plans for queries containing aggregates Specifying the reformatting strategy xx

21 Contents OR strategy limitation When the store operator is not specified Tips on writing abstract plans Comparing plans before and after Effects of enabling server-wide capture mode Time and space to copy plans Abstract plans for stored procedures Procedures and plan ownership Procedures with variable execution paths and optimization Ad Hoc queries and abstract plans CHAPTER 30 Creating and Using Abstract Plans Using set commands to capture and associate plans Enabling plan capture mode with set plan dump Associating queries with stored plans Using replace mode during plan capture Using dump, load, and replace modes simultaneously set plan exists check option Using Other set options with abstract plans Using showplan Using noexec Using forceplan Server-wide abstract plan capture and association Modes Creating plans using SQL Using create plan Using the plan Clause CHAPTER 31 Managing Abstract Plans with System Procedures System procedures for managing abstract plans Managing an abstract plan group Creating a group Dropping a group Getting information about a group Renaming a group Finding abstract plans Managing individual abstract plans Viewing a plan Copying a plan to another group Dropping an individual abstract plan Comparing two abstract plans Changing an existing plan Managing all plans in a group Copying all plans in a group xxi

22 Contents Comparing all plans in a group Dropping all abstract plans in a group Importing and exporting groups of plans Exporting plans to a user table Importing plans from a user table CHAPTER 32 Abstract Plan Language Reference Keywords Operands Derived tables Schema for examples g_join hints i_scan in lru m_g_join mru nested nl_g_join parallel plan prefetch prop scan store subq t_scan table union view work_t CHAPTER 33 Using Statistics to Improve Performance Importance of statistics Updating Adding statistics for unindexed columns update statistics commands Column statistics and statistics maintenance Creating and updating column statistics When additional statistics may be useful Adding statistics for a column with update statistics Adding statistics for minor columns with update index statistics.. xxii

23 Contents 749 Adding statistics for all columns with update all statistics Choosing step numbers for histograms Disadvantages of too many steps Choosing a step number Scan types, sort requirements, and locking Sorts for unindexed or non leading columns Locking, scans, and sorts during update index statistics Locking, scans and sorts during update all statistics Using the with consumers clause Reducing update statistics impact on concurrent processes 752 Using the delete statistics command When row counts may be inaccurate CHAPTER 34 Using the set statistics Commands Command syntax Using simulated statistics Checking subquery cache performance Checking compile and execute time Converting ticks to milliseconds Reporting physical and logical I/O statistics Total actual I/O cost value Statistics for writes Statistics for reads statistics io output for cursors Scan count Relationship between physical and logical reads statistics io and merge joins CHAPTER 35 Using set showplan Using Basic showplan messages Query plan delimiter message Step message Query type message FROM TABLE message TO TABLE message Update mode messages Optimized using messages showplan messages for query clauses GROUP BY message Selecting into a worktable Grouped aggregate message xxiii

24 Contents compute by message Ungrouped aggregate message messages for order by and distinct Sorting messages Messages describing access methods, caching, and I/O cost Auxiliary scan descriptors message Nested iteration message Merge join messages Table scan message Clustered index message Index name message Scan direction messages Positioning messages Scanning messages Index covering message Keys message Matching index scans message Dynamic index message (OR strategy) Reformatting Message Trigger Log Scan Message I/O Size Messages Cache strategy messages Total estimated I/O cost message showplan messages for parallel queries Executed in parallel messages showplan messages for subqueries Output for flattened or materialized subqueries Structure of subquery showplan output Subquery execution message Nesting level delimiter message Subquery plan start delimiter Subquery plan end delimiter Type of subquery Subquery predicates Internal subquery aggregates Existence join message CHAPTER 36 Statistics Tables and Displaying Statistics with optdiag System tables that store statistics systabstats table sysstatistics table Viewing statistics with the optdiag utility optdiag syntax optdiag header information xxiv

25 Contents Table statistics Index statistics Column statistics Histogram displays Changing statistics with optdiag Using the optdiag binary mode Updating selectivities with optdiag input mode Editing histograms Using simulated statistics optdiag syntax for simulated statistics Simulated statistics output Requirements for loading and using simulated statistics Dropping simulated statistics Running queries with simulated statistics Character data containing quotation marks Effects of SQL commands on statistics How query processing affects systabstats CHAPTER 37 Tuning with dbcc traceon Tuning with dbcc traceon(302) dbcc traceon(310) Invoking the dbcc trace facility General tips for tuning with dbcc traceon(302) Checking for join columns and search arguments Determining how the optimizer estimates I/O costs Structure of dbcc traceon(302) output Table information block Identifying the table Basic table data Cluster ratio Partition information Base cost block Concurrency optimization message Clause block Search clause identification Join clause identification Sort avert messages Column block Selectivities when statistics exist and values are known When the optimizer uses default values Out-of-range messages Disjoint qualifications message Forcing messages Unique index messages xxv

26 Contents Other messages in the column block Index selection block Scan and filter selectivity values Other information in the index selection block Best access block dbcc traceon(310) and final query plan costs Flattened subquery join order message Worker process information Final plan information CHAPTER 38 Monitoring Performance with sp_sysmon Using When to run Invoking Fixed time intervals Using begin_sample and end_sample Specifying report sections for output Specifying the application detail parameter Redirecting output to a file How to use the reports Reading output Interpreting the data Sample interval and time reporting Kernel utilization Sample output Engine busy utilization CPU yields by engine Network checks Disk I/O checks Total disk I/O checks Worker process management Sample output Worker process requests Worker process usage Memory requests for worker processes Avg mem ever used by a WP Parallel query management Sample output Parallel query usage Merge lock requests Sort buffer waits Task management Sample output Connections opened xxvi

27 Contents Task context switches by engine Task context switches due to Application management Requesting detailed application information Sample output Application statistics summary (all applications) Per application or per application and login ESP management Sample output Housekeeper task activity Sample output Buffer cache washes Garbage collections Statistics updates Monitor access to executing SQL Sample output Transaction profile Sample output Transaction summary Transaction detail Inserts Updates and update detail sections Deletes Transaction management Sample output ULC flushes to transaction log Total ULC flushes ULC log records Maximum ULC size ULC semaphore requests Log semaphore requests Transaction log writes Transaction log allocations Avg # writes per log page Index management Sample output Nonclustered maintenance Page splits Page shrinks Index scans Metadata cache management Sample output Open object, index, and database usage Object and index spinlock contention xxvii

28 Contents Hash spinlock contention Lock management Sample output Lock summary Lock detail Deadlocks by lock type Deadlock detection Lock promotions Lock time-out information Data cache management Sample output Cache statistics summary (all caches) Cache management by cache Procedure cache management Sample output Procedure requests Procedure reads from disk Procedure writes to disk Procedure removals Memory management Sample output Pages allocated Pages released Recovery management Sample output Checkpoints Average time per normal checkpoint Average time per free checkpoint Increasing the housekeeper batch limit Disk I/O management Sample output Maximum outstanding I/Os I/Os delayed by Requested and completed disk I/Os Device activity detail Network I/O management Sample output Total network I/Os requests Network I/Os delayed Total TDS packets received Total bytes received Average bytes received per packet Total TDS packets sent Total bytes sent xxviii

29 Contents Average bytes sent per packet Reducing packet overhead Index xxix

30 Contents xxx

31 About This Book Audience This manual is intended for database administrators, database designers, developers and system administrators. Note You may want to use your own database for testing changes and queries. Take a snapshot of the database in question and set it up on a test machine. How to use this book This manual would normally be used to fine tune, troubleshoot or improve the performance on Adaptive Server. The Performance and Tuning Guide is divided into three books: Volume 1 - Basics Volume 2 - Optimizing and Abstract Plans Volume 3 - Tools for Monitoring and Analyzing Performance The following information is covered: Volume 1- Basics Chapter 1, Overview, describes the major components to be analyzed when addressing performance. Chapter 2, Networks and Performance, provides a brief description of relational databases and good database design. Chapter 3, Using Engines and CPUs, describes how client processes are scheduled on engines in Adaptive Server. Chapter 4, Distributing Engine Resources describes how to assign execution precedence to specific applications. Chapter 5, Controlling Physical Data Placement describes the uses of segments and partitions for controlling the physical placement of data on storage devices. Chapter 6, Database Design provides a brief description of relational databases and good database design. xxxi

32 Chapter 7, Data Storage describes Adaptive Server page types, how data is stored on pages, and how queries on heap tables are executed. Chapter 8, Indexing for Performance provides guidelines and examples for choosing indexes. Chapter 9, How Indexes Work provides information on how indexes are used to resolve queries. Chapter 10, Locking Configuration and Tuning describes the impact of locking on performance and describes the tools to analyze locking problems and configure locking. Chapter 11, Using Locking Commands describes the commands that set locking schemes for tables and control isolation levels and other locking behavior during query processing. Chapter 12, Reporting on Locks describes the system procedures that report on locks and lock contention. Chapter 13, Setting Space Management Properties describes how space management properties can be set for tables to improve performance and reduce the frequency of maintenance operations on tables and indexes. Chapter 14, Memory Use and Performance describes how Adaptive Server uses memory for the procedure and data caches. Chapter 15, Determining Sizes of Tables and Indexes, describes different methods for determining the current size of database objects and for estimating their future size. Chapter 16, Maintenance Activities and Performance describes the impact of maintenance activities on performance, and how some activities, such as recreating indexes, can improve performance. Volume 2 - Optimizing and Abstract Plans Chapter 17, Adaptive Server Optimizer explains the process of query optimization, how statistics are applied to search arguments and joins for queries. Chapter 18, Advanced Optimizing Tools describes advanced tools for tuning query performance. Chapter 19, Query Tuning Tools presents an overview of query tuning tools and describes how these tools can interact. xxxii

33 About This Book Chapter 20, Access Methods and Query Costing for Single Tables describes how Adaptive Server accesses tables in queries that only involve one table and how the costs are estimated for various access methods. Chapter 21, Accessing Methods and Costing for Joins and Subqueries describes how Adaptive Server accesses tables during joins and subqueries, and how the costs are determined. Chapter 22, Parallel Query Processing intoduces the concepts and resources required for parallel query processing. Chapter 23, Parallel Query Optimization provides an indepth look at the optimization of parallel queries. Chapter 24, Parallel Sorting describes the use of parallel sorting for queries and creating indexes. Chapter 25, Tuning Asynchronous Prefetch describes how asynchronous prefetch improves performance for queries that perform large disk I/O. Chapter 26, tempdb Performance Issues stresses the importance of the temporary database, tempdb, and provides suggestions for improving its performance. Chapter 27, Cursors and Performance describes performance issues with cursors. Chapter 28, Introduction to Abstract Plans provides an overview of abstract plans and how they can be used to solve query optimization problems. Chapter 29, Abstract Query Plan Guide provides an introduction to writing abstract plans for specific types of queries and to using abstract plans to detect changes in query optimization due to configuration or system changes. Chapter 30, Creating and Using Abstract Plans describes the commands that can be used to save and use abstract plans. Chapter 31, Managing Abstract Plans with System Procedures describes the system procedures that manage abstract plans and abstract plan groups. Chapter 32, Abstract Plan Language Reference describes the abstract plan language. Volume 3 - Tools for Monitoring and Analyzing Performance Chapter 33, Using Statistics to Improve Performance describes how to use the update statistics command to create and update statistics. Chapter 34, Using the set statistics Commands explains the commands that provide information about execution. xxxiii