Monitoring, Tuning, and Configuration

Transcription

1 Monitoring, Tuning, and Configuration Monitoring, Tuning, and Configuration Objectives Learn about the tools available in SQL Server to evaluate performance. Monitor application performance with the SQL Server Profiler. Work with the tools available in the Query Analyzer. Use the graphical showplan tools and the Index Tuning Wizard to optimize queries. Learn how data caching works in SQL Server. Use the System Monitor to monitor SQL Server system performance. Microsoft SQL Server 2000 Professional Skills Development 15-1

2 Monitoring, Tuning, and Configuration Evaluating Performance Many individuals think of performance tuning as the act of changing and adjusting server setting to tweak performance. While there is a lot to be said for knowing how to adjust the knobs with SQL Server, the majority of the performance snags in SQL Server are related to the way that your queries are written. Query bottlenecks often occur when Transact-SQL code is written in such a way that it goes against the way SQL Server is meant to perform. SQL Server 2000 provides a variety of tools that can be used to monitor performance and return information on how your applications are using system resources. As far as tuning the server itself goes, SQL Server 2000 actively monitors its own settings, pretty much freeing developers and DBAs from having to worry too much about tweaking server settings. This active monitoring allows developers and DBAs to concentrate on how Transact- SQL code is written, instead of worrying about server settings. Only when you ve reviewed your code and you re still having performance problems do you need to look at the server, or the hardware, or perhaps the network configuration. You can adjust SQL settings such as memory and processor throughput at the server level these are often referred to as adjusting the knobs. There are two parts to getting the most out of SQL Server understanding how to write queries and use indexes, and becoming familiar with the monitoring tools available in SQL Server 2000 to help you troubleshoot performance problems. Define Monitoring Goals Before you choose a performance monitoring tool or technique, you need to evaluate your monitoring goals. You may need to improve performance by monitoring response times for queries and adjusting indexes. Or perhaps you need to evaluate user activity in terms of security or for assessing ad hoc query impact on your server. Monitoring can also include troubleshooting server bottlenecks, debugging application components, or blocking/deadlocking issues. In order to evaluate performance effectively, you will need to establish a baseline by taking measurements over time and comparing the baseline statistics to current performance. Numbers far above or below the baseline may indicate areas in need of tuning or reconfiguration. SQL Server Monitoring Tools SQL Server provides system stored procedures as well as monitoring tools to help you monitor performance. For example, the following stored procedures 15-2 Microsoft SQL Server 2000 Professional Skills Development

3 Evaluating Performance and system functions can be used without having to resort to running a separate tool: sp_who returns information about current SQL Server connections and processes. You can also view this information in the Enterprise Manager. sp_lock returns information about locks being held, which is also available in the Enterprise Manager. sp_spaceused displays an estimate of the current amount of disk space used. sp_monitor displays statistics about CPU usage, I/O usage, and the amount of idle time since sp_monitor was last executed. DBCC statements, which can be used to check performance statistics as well as the logical and physical consistency of a database. Global system function, such which contains the amount of time the CPU has been executing SQL Server which contains the number of SQL Server connections or attempted connections, which contains the number of network packets occurring on SQL Server connections. In addition, SQL Server also provides the SQL Profiler, System Monitor, and assorted tools in the Query Analyzer. SQL Profiler The SQL Profiler enables you to monitor server and database activity. You can trace Transact-SQL statements, stored procedures, login activity, number of deadlocks, fatal errors, or other problems. You can also replay events captured, step-by-step, to examine exactly what happened. System Monitor The System Monitor (or Performance Monitor) is available only on Windows NT/Windows 2000 and can monitor either remote or local instances of SQL Server, tracking resource usage. The System Monitor collects counts rather than data about events. You can use it to track memory usage, number of active transactions, number of blocked locks, or CPU activity. You can also generate alerts from the System Monitor that are based on thresholds you set on specific counters. Current Activity Window in the Enterprise Manager The Enterprise Manager displays graphical information about current processes, providing an ad hoc view of current activity. Microsoft SQL Server 2000 Professional Skills Development 15-3

4 Monitoring, Tuning, and Configuration Error Logs The SQL Server error logs and the Windows application event log provide information about events in SQL Server Microsoft SQL Server 2000 Professional Skills Development

5 Monitoring with SQL Server Profiler Monitoring with SQL Server Profiler You can think of the SQL Server Profiler as a tool that snoops on the conversations between your applications and your SQL Server. The Profiler allows you to view the actual traffic between client and server, rather than just queries and their results. The Profiler is an interception utility, interposing itself between client and server, passing on all of their traffic in real time while maintaining a history of events that you can analyze or replay later. It accomplishes its tasks by providing a graphical interface to a set of stored procedures. TIP: The Profiler consumes a lot of system resources, so you don t want to leave it running full-time or at peak load time, unless you re debugging a specific problem. Profiler Terminology In order to use the Profiler effectively, you need to become familiar with some key concepts and terminology. The Profiler runs traces that record activity on your SQL Server. A trace is created from a template that defines the data you re interested in collecting. A trace collects data by monitoring the specific events, as defined the template. An event is an action generated within the SQL Server engine. For example, the start of a Transact-SQL batch triggers an event, as does acquiring a lock on a table. Creating a trace template involves specifying which events, data columns, and filters to use. An event class is a generic description of the type of event that was produced by the SQL Server engine. The Profiler displays the event classes and data columns while the trace is running. An event category refers to the way events are grouped together, with all lock event classes being grouped with the Locks event category, and the transaction event classes being grouped with the Transaction event category, and so on. In addition, you can also define a filter, which lets you include or exclude data. Trace files can become quite large if too many events are being monitored, and filters let you winnow down the trace to only record activity you are interested in. The data columns describe the data collected for each of the event classes captured in the trace. Because the event class determines the type of data collected, not all data columns are applicable to all event classes. Microsoft SQL Server 2000 Professional Skills Development 15-5

6 Monitoring, Tuning, and Configuration Getting Started with the Profiler When you define a trace template and launch a trace, the settings you defined in the template will determine the data that is captured by the trace. You can view trace data in real time in the Profiler window, or capture it to a file or database table, where it can be analyzed later. Try It Out! You can launch the Profiler from the Windows Start menu or from the Enterprise Manager. 1. From the Windows Start menu, choose Programs Microsoft SQL Server Profiler. This launches the Profiler in an empty window. 2. Choose File New Trace from the Profiler menu and log on to SQL Server Microsoft SQL Server 2000 Professional Skills Development

7 Monitoring with SQL Server Profiler 3. Fill in the dialog box shown in Figure 1. Name the trace TraceShark, choose the SQLProfilerStandard template and save the trace to a file named SharkTrace.trc. Figure 1. Defining trace properties and choosing the trace template. Microsoft SQL Server 2000 Professional Skills Development 15-7

8 Monitoring, Tuning, and Configuration 4. Click the Events tab and expand the Locks node in the Available event classes list, as shown in Figure 2. Add the Lock:Deadlock and Lock:Deadlock Chain event classes. These two event classes will help you determine the cause of deadlocking on your SQL Server. Figure 2. Adding lock event classes to a trace Microsoft SQL Server 2000 Professional Skills Development

9 Monitoring with SQL Server Profiler 5. Click the Data Columns tab. Add the DatabaseName column to the Selected data list, as shown in Figure 3. Figure 3. Adding the DatabaseName column to the trace. Microsoft SQL Server 2000 Professional Skills Development 15-9

10 Monitoring, Tuning, and Configuration 6. Click the Filter tab, expand the Like node, and type Query Analyzer to filter only Query Analyzer events, as shown in Figure 4. Figure 4. Setting a filter to trace only Query Analyzer events. 7. Click the Run button to start the trace. Switch to the Query Analyzer and execute a query. You should see the results as shown in Figure 5. Figure 5. The results of running a trace Microsoft SQL Server 2000 Professional Skills Development

11 Monitoring with SQL Server Profiler 8. If at any time you want to change the trace definition, stop the trace. Choose File Properties from the menu. You can add or remove items from the trace. Microsoft SQL Server 2000 Professional Skills Development 15-11

12 Monitoring, Tuning, and Configuration Tuning Queries Learning how to write efficient Transact-SQL code is fundamental to performance. You can have the most powerful hardware, the most RAM, and still come away with sub-par performance if you don t know how to write an efficient query. How Join Types Affect Performance SQL Server 2000 introduces new join types that boost performance over the benchmarks of earlier versions. A join type is an algorithm that SQL server uses internally to perform a join. In SQL Server 6.5, there was only one join type: The nested loop, which performed a join by searching on the inner table for each row of the outer table. A nested loop is similar to the programming algorithm of reading the first row in the outer table, and searching the inner table for a corresponding match. Although this works well for certain types of tables, it is not always the best mechanism for performing a join. SQL Server 2000 has two additional join types: A merge join can be used if the columns in both tables are sorted using the order of the join column. The tables are then merged, hence the name merge join. This type of join is usually chosen if there is a clustered index sorting the join columns for both tables. A hash join uses a complex set of algorithms to subdivide the data into hash buckets. These hash buckets are internal mechanisms that reduce access time by allowing a search from bucket to bucket between the tables. A hash join is the most efficient of the three join types. Join Hints There is no default join type in SQL Server The Query Optimizer examines each situation separately and determines the join strategy that best fits. The join strategies can be tweaked using join hints that change the join method. You can force a loop, hash, or merge join by specifying in the from clause the appropriate ANSI SQL syntax. Join hints are fully documented in Books Online under the sub-topic Hints in the Using Options in SQL Server topic Microsoft SQL Server 2000 Professional Skills Development

13 Tuning Queries Although you can provide join hints for the Query Optimizer, it is rarely necessary to do so. Most of the time SQL Server will pick the best join strategy for a particular query. However, if you feel that a hint is required, you can test forcing different merge strategies in order to maximize performance. You can provide a hint to the optimizer telling it which join to use. Note that when a join hint is used, it is placed between the key words of the join: SELECT invid, custno, zip, item FROM invoice INNER MERGE JOIN invoicedetail ON invoice.invid = invoicedetail.invid TIP: Join hints and optimizing queries are a waste of time for small tables. What difference is it going to make if a table scan is used in querying the Employees table in the Northwind database? Or for that matter, any of the tables in the Northwind database? The Query Optimizer doesn t waste resources by trying to figure out the best way to fetch small amounts of data good enough will do. This is why you always need to test your queries with a full load of sample data that closely approximates the volume of data you ll be dealing with when your application goes into production. Is there a right type of join that fits every query? Absolutely not that is what performance tuning is all about: Measuring performance by trying different strategies until you find the one that works best for your particular situation. Performance Issues with Joins and Subqueries A join is an operation that allows you to query two or more tables to produce a single result set incorporating rows and columns from each table. There are three basic types of joins: Inner joins combine tables by comparing values in columns that are common to both tables. Outer joins return combined rows that match the join condition. In addition to the rows that match the join condition, they return unmatched rows. Left- and right-outer joins return all rows from one of the tables, regardless if there is a matching row in the other table. Microsoft SQL Server 2000 Professional Skills Development 15-13

14 Monitoring, Tuning, and Configuration Full outer joins will in effect perform an outer join on both tables. In effect, a full outer join is the same as doing a left join on one table and a right on the other. It returns both matched rows and unmatched rows from both tables. Outer joins will almost always produce a result set with null values. Columns from the source table of the unmatched rows contain data, but columns from the other table contain null values. SQL Server not only has to provide the matched data in the result set, but it has to do a little extra work and provide the unmatched data as well, adding a performance penalty. The reason for this is that outer joins employ temporary tables internally, as do subqueries. The left join must use one temp table and the full outer join uses two temp tables. If you have to use a left join or a full outer join, you should question whether the database is designed correctly and the data meets basic data integrity standards. A subquery is often used as an alternative to outer joins because it can often save steps and the result is similar to the join. However, most of the time a join will most likely outperform a subquery. Limit the Number of Tables in a Join Limiting the number of tables in a join is a good idea because SQL Server usually makes more efficient use of indexes when the joins are between fewer tables. Performance drastically slows at around four joins if you find yourself writing queries with too many joins, consider using several smaller queries or temporary tables to hold intermediate result sets. Avoid Subqueries A subquery is a select statement nested inside of a select, update, delete, or inside another subquery. They are a logical way to query data as they often match the way we think, and can be used in place of an expression. However, they are often not the most efficient way of querying data. There are two types of subqueries: non-correlated and correlated. A non-correlated subquery has the following syntax: WHERE expression [NOT] IN (subquery) or WHERE expression comparison operator [ANY ALL] (subquery) Microsoft SQL Server 2000 Professional Skills Development

15 Tuning Queries The following SQL statement using a non-correlated subquery is not efficient SQL: SELECT trantype, amount FROM acct_tran WHERE acct_no IN (SELECT acct_no FROM acctmaster WHERE acctype = 'A') The reason this is not efficient is because the inner query has to be resolved before the outer query can be resolved. SQL Server has to resolve two separate queries in order to process the statement. A join is going to give you much better performance. Here s the subquery rewritten as a join: SELECT trantype, amount FROM acct_tran INNER JOIN acctmaster ON acct_tran.acct_no = acctmaster.acct_no WHERE acctype = 'A' Although using the subquery appeals more to the way we think than the join example, it s not optimized for the way SQL Server behaves. The join will give better performance here, especially with large data sets. Correlated subqueries use the following syntax: WHERE [NOT] EXISTS (subquery) Here s an example of a correlated subquery: SELECT pub_name FROM publishers WHERE EXISTS (SELECT * FROM titles WHERE pub_id = publishers.pub_id AND type = business ) Microsoft SQL Server 2000 Professional Skills Development 15-15

16 Monitoring, Tuning, and Configuration In order for the inner query to be resolved, it needs the results of the outer query first. When a correlated relationship exists between the inner query and the outer query, SQL Server has no choice but to process the query in a rowby-row basis. Processing anything a row at a time is always very slow. This query should be rewritten using a join: SELECT distinct pub_name FROM publishers,titles WHERE titles.pub_id = publishers.pub_id AND type = business Subqueries can be nested up to 32 levels deep, which could confer quite a substantial performance penalty if you were ever to attempt such a thing. The more levels that are nested, the greater the performance hit. Avoid subqueries in favor of joins wherever possible. Avoid Cursors Cursors in SQL Server are similar to recordsets in client applications they allow movement through a set of records. Cursors give you the ability to work with one row of data at a time, and then move on to the next row. While cursors do have their uses, SQL Server is optimized to work with data in sets, not individual rows. Unfortunately, many applications suffer from an overuse of cursors because the developers don t know how to work with data any other way. While cursor performance in SQL Server 2000 is much improved over earlier releases, they re not the best choice in terms of performance if any other alternatives exist. Consider this example: For each invoice header row, you want to find the associated invoice detail row that has the highest amount. The temptation would be to use a cursor to scroll through each invoice detail row, looking for the highest amount associated with an invoice (probably using variables to do the comparisons). You could use a cursor to perform this type of processing it would work, but it would be slower than using a query. Here s the equivalent Transact-SQL syntax for performing the same operation: SELECT invoice_id, max(amont) FROM invoice_detail GROUP BY invoice_id Microsoft SQL Server 2000 Professional Skills Development

17 Tuning Queries To take the example one step further, imagine that you want the highest invoice detail for each invoice, but only where the budget area in the invoice detail equaled the invoice header budget area. The temptation would be to again use a cursor to accomplish this task. A better solution would be to use a correlated subquery. Although correlated subqueries are inefficient compared to joins, they re a more efficient choice than using a cursor. Here s the correlated subquery that will accomplish the task: SELECT invoice_id, max(amount) FROM invoice_detail AS a WHERE EXISTS (SELECT invoice_id FROM invoice_header WHERE invoice_header.invoid = a.invoid AND invoice_header.budgetarea=a.budgetarea) GROUP BY invoice_id There will be times when functionality will outweigh response time and a cursor will be necessary to solve a particular problem. As a database designer/implementer, you will need to consider the features that are provided by the different types of cursors and the resulting impact on the performance of the application and the database engine. Cursor Alternatives Whenever possible, try to find an alternative for using cursors. Use temporary tables or the new Table data type for processing intermediate result sets. Microsoft SQL Server 2000 Professional Skills Development 15-17

18 Monitoring, Tuning, and Configuration Working with Indexes Understanding how indexes work is crucial to getting the best performance out of your SQL Server. What Is an Index? The best analogy for an index is a book. If you were to look up an item, you d turn either to the table of contents or the index, note the page number, and then simply locate the page and read the item you re interested in. A SQL Server index works the same way. Imagine a directory that points to different pages of a SQL Server database and imagine how much time this saves when trying to find a particular row. The structure used to create an index is called a b-tree structure. A b-tree structure basically works as a hierarchy. An index is created on columns of a table. The table that the index is created on is often referred to as the base table. An index can be created on one or many columns of a table. There are two types of nodes on a b-tree structure: leaf nodes and non-leaf nodes. A leaf node is the lowest level on the b-tree. Every other node is a nonleaf node. When a table doesn t have an index, SQL Server must perform a table scan, which can be enormously time-consuming, especially on large tables. If an index exists, the Query Optimizer will generally choose it if it offers a better performance plan than a table scan. Clustered and Non-Clustered Index Recommendations There are two index types in SQL Server: A non-clustered index works like the book analogy described earlier in this section. It is a separate structure that allows SQL Server to look up the location of the data being sought. A clustered index reflects the physical order of the data. A table can only have one clustered index, since it can only be stored in one physical order. If a non-clustered index is analogous to the index or table of contents of a book, then a clustered index is analogous to the page number Microsoft SQL Server 2000 Professional Skills Development

19 Working with Indexes Clustered indexes organize the data in the base table by the index key. This organization allows rows within a range to appear on the same page and results in rapid retrieval. Clustered indexes work best for range selections and for keys with a high density where you need to retrieve a large number of rows. An example of this is a State column on a table that holds the state a person lives in. There will be a limited number of distinct values stored in the column (50, to be precise). A query by state will potentially bring back a large number of rows and is therefore a good candidate for a clustered index. A non-clustered index presents the opposite situation. The data on the base table is not organized in the physical order of the index, so retrieving a large number of rows via a non-clustered index is of no value as the rows will not necessarily be stored on the same page. A non-clustered index is best for a table with high selectivity (many unique keys) where only a few (or even one) row needs to be retrieved. Data elements like social security numbers or employee IDs are perfect candidates for non-clustered indexes. Here are some recommendations for creating indexes: Normalize your database and use DRI (declarative referential integrity) to enforce relationships. DRI constraints use native machine code, which is faster than Transact-SQL code contained in stored procedures and triggers. In addition, create indexes on foreign key joins in the referenced table. This will improve join performance between the two tables. Keep clustered indexes as narrow as possible. In SQL Server, a nonclustered index will attempt to utilize a clustered index on a table, if there is one. To do this, the non-clustered index will need to store the key of the clustered index in its b-tree structure. The smaller the key, the less data that will need to be stored in the non clustered index, resulting in better performance. Watch out for disk-jam with clustered indexes. A clustered index on a sequentially incrementing key (such as an identity column) is a poor choice. Clustered indexes can cause disk-jam or hot spot problems as a clustered index will force a row to be written sequentially on the disk. Imagine a regional office for a large corporation that is entering invoices. Imagine that this table contains a regional office number column that is always filled in with the same number. If this table were to have a clustered index on the regional office number, the rows will be written sequentially to the disk ordered by this office number. A high volume of transactions will cause excessive action on that part of the disk, causing a disk jam. Set the fill factor and pad index options. So, what happens when an index page fills up? When a page containing data rows fills up, SQL server can go to the next available page. Not so for an index page. When a leaf-level index page fills up, the page will split in half, writing half of the index to a new page. This page splitting will incur a lot of overhead in terms of I/O so it is best to avoid a page from filling up. When you create indexes, you can specify how much of the page Microsoft SQL Server 2000 Professional Skills Development 15-19

20 Monitoring, Tuning, and Configuration to use as the index is being created. This index option is called fill factor. If you specify a fill factor of 50%, when the index is created, half of each page will be filled up. The pages will continue to fill, and will fill up until the page is completely full. The page will split again if a re-indexing does not occur, which will reorganize the index pages back to the fill factor of 50% again. Using a fill factor will only work if re-indexing is done on a regular basis. The pad index option is similar to fill factor, except it works with non-leaf level pages. When the pad index option is used, it will take the fill factor percentage specified in the fill factor option; specifying the pad index option without specifying a fill factor is useless. Avoid too many indexes. Indexes come at a price. Indexes need to be maintained as rows are inserted, deleted or updated from a table. The more indexes exist for a table, the more overhead a table has when the data changes. Indexes are accessed from the highest order column downward. If a table has an index by state and zip code, a read from the table by both those columns (i.e., in the WHERE clause) will result in the optimizer using the index. A read by state will also use the index. A read by zip code will not use that particular index, because it is not starting in the column in the highest position. This brings you to the next major issue, one of how the optimizer decides which index to use. How the Optimizer Decides on an Index The SQL Server Query Optimizer will choose the index it feels will bring results back as fast as possible, utilizing a complex set of algorithms to do so. Each index contains a distribution page, containing a sample space of each index. The Query Optimizer will look at this sample to determine which index is most efficient. Unless you provide compiler hints in the query, SQL Server will automatically select the index it feels is best, based on the statistics for that table Microsoft SQL Server 2000 Professional Skills Development

21 Working with Indexes As tables are updated, rows are added and deleted, the statistics become out of date. SQL Server automatically detects when statistics are out of date and updates them automatically. You can change the server configuration using the Enterprise manager or with the UPDATE STATISTICS Transact-SQL command. Figure 6 shows the Shark Properties dialog box, with the Auto create statistics and the Auto update statistics options turned on. It is recommended that you leave them on. Figure 6. The Shark database properties showing the Auto update statistics option selected. Using Graphical Showplan The Query Analyzer ships with a set of tools known as the graphical showplan, that shows you a graphical representation of a query execution plan. The execution plan is the series of steps that the query optimizer takes when determining the best way to execute a query. The optimizer picks the best path for execution possible, but it can only work with what actually exists in the Microsoft SQL Server 2000 Professional Skills Development 15-21

22 Monitoring, Tuning, and Configuration table. In other words, if there are no indexes, then the optimizer must employ a table scan, which means traversing all of the rows until it finds the requested data. Needless to say, a table scan is much more CPU-intensive and time consuming than accessing the data through an index. The graphical showplan provides the following information: Displays the data retrieval methods chosen by the query optimizer. Shows output using icons representing the execution of specific statements and queries. These icons are documented in Books Online under the topic Graphically Displaying the Execution Plan Using SQL Query Analyzer. Provides statistical information on performance measured in terms of CPU cost. Try It Out! See Tuning.SQL Follow these steps to create a table and some sample data to analyze: 1. Open the Query Analyzer and create the test table: CREATE TABLE tbltest (EmpID int identity, EmpStuff char(20), EmpZip char(5)) 2. Run the following script to create the sample data: int = 0 < 5000 BEGIN INSERT INTO tbltest VALUES('aaaaaaaaaa','10003') INSERT INTO tbltest VALUES('aaaaaaaaaa','10103') INSERT INTO tbltest VALUES('aaaaaaaaaa','10245') INSERT INTO tbltest VALUES('aaaaaaaaaa','16124') INSERT INTO tbltest VALUES('aaaaaaaaaa','20873') +1 END Microsoft SQL Server 2000 Professional Skills Development

23 Working with Indexes 3. Create the following queries: SELECT EmpID, EmpStuff FROM tbltest WHERE EmpID = 250 SELECT EmpID, EmpStuff FROM tbltest WHERE EmpZip = '10003' 4. Highlight both queries by dragging the mouse over them and choose Query Display Estimated Execution Plan from the menu or press the CTRL+L key. This displays the results shown in Figure 7. Drag the mouse over the Table Scan icon to display the statistics for the query. Note the CPU cost and the Subtree cost. The CPU cost represents the estimated cost for all CPU activity, and the Subtree cost is the total cost to the query optimizer in executing the operation and all operations preceding it in the same subtree. Figure 7. The graphical showplan for a table scan. The table does not have either a primary key or any other indexes to improve performance, forcing SQL Server to perform a table scan in order to retrieve the results. Microsoft SQL Server 2000 Professional Skills Development 15-23

24 Monitoring, Tuning, and Configuration Indexed Views Indexed views are a new feature in SQL Server They ve been around for a while in other relational database management systems, like Oracle, where they are known as materialized views. In SQL Server 2000, an indexed view consists of a view with a unique clustered index, which materializes the view as index pages in a clustered index reflect the physical ordering of the data. TIP: Once you create the clustered index, you can create additional, non-clustered indexes on the view. When to Use Indexed Views Indexed views are not for every application you need to weigh the pros and cons of using indexed views: If your data is relatively static and you need to process or aggregate many rows, then consider using indexed views. Consider indexed views only when the improved speed in retrieving the results outweighs the increased overhead of data modifications. Indexed views are not appropriate for a high-volume OLTP system. The overhead of an indexed view is excessive for a query that does not involve any aggregations or joins. Don t use indexed views for simple queries. If the result set of an aggregate query returns almost as many rows as the base table, then don t use an indexed view. A view with expanding joins in which the result set is larger than the original data stored in the base tables is also not a good candidate for an indexed view. Indexed views are more complex to maintain than indexes on tables. After the index is created, modifications to the table schema are not permitted. Once you ve determined where indexed views may be useful, you then need to consider the hard-and-fast requirements for creating indexed views. Indexed View Requirements When you create a view that you intend to use as an indexed view, it must meet the following requirements: Microsoft SQL Server 2000 Professional Skills Development

25 Indexed Views The ANSI_NULLS and QUOTED_IDENTIFIER options must be set to ON when the view is created. The view can only reference base tables not other views. Base tables referenced in the view must be in the same database as the view, and both must have the same owner. Use the WITH SCHEMABINDING option in the view definition so that the underlying tables cannot be altered or dropped as long as the view or function exists. Reference all table names and user-defined functions with the two-part naming convention, such as dbo.tblcustomer for the tblcustomer table. All functions used by the view must be deterministic, meaning that the function always returns the same result set each time it s called with the same set of input values. For example, SUM(2+2) is deterministic, whereas GETDATE() is not. Columns listed in the view must be explicitly named. The * syntax specifying all columns is not allowed. Repeating the same column name is not allowed unless the second occurrence is part of an expression. For example, SELECT Col1, SUM(Col1) AS Total is legal because even though Col1 is listed twice, the second occurrence is part of an expression. Derived tables, subqueries, outer joins, or self-joins are not allowed. The following Transact-SQL statements or operators are not allowed: ROWSET, UNION, ROWSET, TOP, ORDER BY, DISTINCT. The COUNT() function is not allowed, although COUNT_BIG() can be used. The following aggregate functions are also not allowed: AVG, MAX, MIN, STDEV, STDEVP, VAR, and VARP. Try It Out! Follow these steps to create an indexed view on the tbltest table created earlier in this chapter. Microsoft SQL Server 2000 Professional Skills Development 15-25

26 Monitoring, Tuning, and Configuration 1. Open the Query Analyzer if it is not already open, and create the following view. Note that the ANSI_NULLS and QUOTED_IDENTIFIER options are being explicitly set to ON (even though that is the default for the Query Analyzer) and that WITH SCHEMABINDING is used in the view definition. SET ANSI_NULLS ON SET QUOTED_IDENTIFIER ON GO CREATE VIEW vwindexedview WITH SCHEMABINDING AS SELECT EmpZip, COUNT_BIG(EmpZip) AS NumZips FROM dbo.tbltest GROUP BY EmpZip GO 2. The next step is to create the clustered index for the view. Note that all of the options are being explicitly specified, and that the clustered index has its UNIQUE property set. The column chosen for the index must contain unique values and be part of the GROUP BY clause in an aggregate query. SET ANSI_NULLS ON SET ANSI_PADDING ON SET ANSI_WARNINGS ON SET ARITHABORT ON SET CONCAT_NULL_YIELDS_NULL ON SET QUOTED_IDENTIFIER ON SET NUMERIC_ROUNDABORT OFF GO CREATE UNIQUE CLUSTERED INDEX ixtestclustered ON dbo.vwindexedview (EmpZip) GO Microsoft SQL Server 2000 Professional Skills Development

27 Indexed Views 3. The indexed view is now created, and ready to go. Type the following statement in the Query Analyzer and examine the query plan generated. It should look something like that shown in Figure 8. SELECT EmpZip, NumZips FROM vwindexedview WHERE EmpZip = '33404' Figure 8. Displaying the index plan for the clustered index seek on the indexed view. 4. Next, highlight the following query in the Query Analyzer and press CTRL+L to display the estimated execution plan. Note that this query does not refer to the view at all it references only the base table. SELECT EmpZip, COUNT_BIG(EmpZip) FROM dbo.tbltest WHERE EmpZip = '11201' GROUP BY EmpZip Microsoft SQL Server 2000 Professional Skills Development 15-27

28 Monitoring, Tuning, and Configuration Note that the estimated query plan utilizes the indexed view even though it was not explicitly named in the SELECT statement. The query optimizer considers any and all indexes defined in the database when executing a query, regardless of whether the index was defined on a view or on a base table. If the optimizer estimates that using the indexed view provides the lowest-cost access mechanism, the optimizer chooses the indexed view, similarly to the way it chooses base table indexes even though they are not directly referenced in a query. The optimizer may even choose to utilize the indexed view when it contains columns that are not referenced by the query, if the view offers the lowest-cost option for covering one or more of the columns specified in the query. However, if the query contains columns that are not referenced in the indexed view, then the indexed view will not be used. TIP: You can use an inline table-valued function as a wrapper around an indexed view. Indexed views do not support parameters, however an inline function can be defined with parameters that reference a view. This results in optimal performance and flexibility. The complex aggregations and joins are done once, at CREATE INDEX time, and all subsequent queries referencing the inline function filter rows from this pre-built result set Microsoft SQL Server 2000 Professional Skills Development

29 Using the Index Tuning Wizard Using the Index Tuning Wizard The Index Tuning Wizard can help you optimize the performance of your application by analyzing your indexes and helping you fine-tune resource usage, response time, and data availability. You don t have to know everything there is to know about index internals to benefit from using the Index Tuning Wizard. You don t need an expert understanding of hardware platforms and components, or how end-user applications interact with the relational engine. The Index Tuning Wizard helps you select the best mix of indexes, allowing you to tune the database for a small set of problem queries without changing index configuration. However, the Index Tuning Wizard is not designed to be the ultimate tool for developing an indexing schema. Regardless of how efficient a tool may be, you, as the developer have the final say on which indexes will be used. Here s what the Index Tuning Wizard can do: Recommend or verify the optimal index configuration for a database. Provide an analysis of index usage on current indexes. Provide an analysis of performance improvement for the 100 most expensive queries and table participation in a workload. Recommend ways to tune the database for a small set of problem queries. Specify criteria to take into account when the Index Tuning Wizard evaluates a workload. For example: Maximum queries to tune, maximum space for recommended indexes, and maximum columns per index. The Index Tuning Wizard produces recommendations based on capturing and analyzing a workload. Capturing a Workload A workload consists of an SQL script or a SQL Profiler trace saved to a file or table containing SQL batch or remote procedure call (RPC) event classes and the Event Class and Text data columns. The batch consists of a set of SQL statements that you build to reflect a typical mix of statements in the system. You can capture this batch by using the SQL Server Profile and creating a trace. The SQL Server Profiler produces a script that contains the actual SQL statements that are processed on the system during a period of time that reflects the typical load on the system. You need to run the Profiler to collect the trace file when there is significant server load or at times of peak bottleneck, or at a point when bad performance has been noted. In addition, Microsoft SQL Server 2000 Professional Skills Development 15-29

30 Monitoring, Tuning, and Configuration SQL Server 2000 allows you to run the Index Tuning Wizard directly from the Query Analyzer, and analyze only a single query or a series of queries. Analyzing the Workload The Index Tuning Wizard then runs the workload file and analyzes how effectively the existing indexes meet the processing load. The wizard provides this analysis in the form of a variety of reports which that an Index usage report (recommended configuration or current configuration), Table analysis report, Query cost report, Workload analysis report, and Tuning summary report. Providing an Indexing Recommendation After the analysis portion is complete, the Index Tuning Wizard provides a recommendation on exactly which indexes the query optimizer should use to maximize performance via SQL Server statements. These statements can be executed to create new, more effective indexes or drop existing indexes that are ineffective. You can choose among the following options: Implement the changes immediately. Schedule a SQL Server job that can be scheduled to execute at a later time. Save a script to be executed manually. Keep in mind that the wizard does not recommend indexes in the following situations: Tables that are already referenced by cross-database queries. System tables. Columns already referenced by primary key or unique constraints. There is not enough data in the tables being sampled. The suggested indexes do not offer enough projected improvement in query performance over existing indexes. These restrictions make it difficult to demo the Index Tuning Wizard on small sample databases like Northwind and pubs because there simply isn t enough data or activity on those databases to be meaningful. Try It Out! Follow these steps to test the Index Tuning Wizard: Microsoft SQL Server 2000 Professional Skills Development

31 Using the Index Tuning Wizard 1. Launch the Index Tuning Wizard by highlighting the two queries used in the graphical showplan example and choosing Query Index Tuning Wizard from the menu. Click Next to bypass the introductory dialog box. Select the database, as shown in Figure 9. Clear the checkboxes for Keep all existing indexes (there aren t any) and Add indexed views (also not applicable). Select Thorough for the Tuning mode option and click Next. Figure 9. Setting up the options for the Index Tuning Wizard. 2. Leave the SQL Query Analylzer selection option selected and click Next. Microsoft SQL Server 2000 Professional Skills Development 15-31

32 Monitoring, Tuning, and Configuration 3. Select only the tbltest table, as shown in Figure 10. Click Next for the wizard to complete its analysis. Figure 10. Selecting the tbltest table to tune Microsoft SQL Server 2000 Professional Skills Development

33 Using the Index Tuning Wizard 4. You will then see a dialog box presenting the Index Tuning Wizard s recommendations, as shown in Figure 11. The wizard recommends a clustered index on EmpZip and a non-clustered covering index on EmpID and EmpStuff. Figure 11. The Index Tuning Wizard presents its conclusions. Microsoft SQL Server 2000 Professional Skills Development 15-33

34 Monitoring, Tuning, and Configuration 5. Click the Analysis button and choose Query Cost Report from the menu, as shown in Figure 12. This shows you the percent improvement in cost for both queries if the recommendations are accepted. Look at some of the other reports, such as Workload Analysis and Tuning Summary. Click Close and then click Next. Figure 12. Displaying the Query Cost Report Microsoft SQL Server 2000 Professional Skills Development

35 Using the Index Tuning Wizard 6. Click the Next button to advance to the scheduling dialog box. Choose to apply the changes now, as shown in Figure 13. Figure 13. Executing the recommendations. 7. Instead of applying the changes now, save the changes to a script file named IndexChanges in the same folder as the trace file and click Next and then click Finish to end the wizard. 8. Return to the Query Analyzer and choose CTRL + L on the two highlighted queries to show the estimated execution plan, as shown in Figure 14. The first query plan uses an index seek, and the second query uses a clustered index seek. Note the gain in the cost of the queries as compared with the estimated plans before the indexes were created. Microsoft SQL Server 2000 Professional Skills Development 15-35

36 Monitoring, Tuning, and Configuration Figure 14. Using an index seek to quickly locate rows of data. Run the Index Tuning Wizard regularly, especially if the database increases in size. You may need to incorporate new queries into a workload file by creating a trace for the wizard to run. You ll want to capture as much significant activity as possible, however the more data you capture, the longer the wizard will take to run. If that is the case, you can always run the wizard several times with smaller batches Microsoft SQL Server 2000 Professional Skills Development

37 Data Caching Data Caching Creating indexes and tuning queries is important, but so is maximizing resources in SQL Server such as memory usage and allocation, and disk usage in SQL Server. Reading and writing information to a database will normally go directly to disk. This however is slower than retrieving data from memory as disk access is considerably slower than memory access. The SQL Server architecture is designed in such a way that the disk does not have to be accessed as often and hence retrieval speed is maximized. SQL Server accomplishes this by employing a data cache and a procedure cache. The Data Cache When data is first retrieved, it is retrieved from disk and placed into the data cache. When it is called a second time, it is retrieved straight from the data cache so that the disk does not have to be accessed a second time. The first retrieval will incur a lot more overhead than the second retrieval, and all subsequent retrievals. In fact, the more data there is in the cache, the better the system will run because there will be less file I/O. When performing an update, delete, or insert the changes are recorded in the data cache. Eventually, changes that are made to data cache are committed to the disk during the CHECKPOINT process. While reading and writing straight from memory produces far better performance, what happens if the system crashes? What happens to all those updates, inserts, and deletes? They are sitting in memory, not on the disk. The Transaction Log SQL Server maintains a transaction log for those changes that have occurred in memory, but have not been committed to disk. It s quicker to write to the log than it is to write to disk, which is where a lot of the performance gain with SQL Server occurs. The database structure is complex, and the write to the actual database can be lengthy. Writing to the log, on the other hand, is a fairly quick process, as the log format is a lot simpler. If the system crashes, SQL Server can recover information from the log that has not yet been committed to disk. Memory Usage What happens when the data cache gets filled up? Data in the data cache is stored in pages using the same format in disk and a collection of the rows in the cache is known as the memory buffer pool. The memory buffer pool is scanned by a background process known as the lazywriter, which after Microsoft SQL Server 2000 Professional Skills Development 15-37

38 Monitoring, Tuning, and Configuration scanning, flushes the oldest buffers and dirty buffers to disk. This process of flushing out the memory buffer pool to make room translates into overhead. When you are optimizing SQL Server, you must make sure there is sufficient memory available to hold a large amount of data in cache without it being flushed constantly. The Procedure Cache SQL Server caches compiled query plans in memory in the procedure cache for reuse for both ad hoc queries and stored procedures. Subsequent executions of the query or the stored procedure can take advantage of the cached plan and will execute faster. What this means in terms of performance is that the first person to execute the procedure will incur the overhead, while subsequent users will not. The amount of memory in your server can drastically affect the server performance in its ability to cache data and performance plans. How much memory is enough? Different types of applications will have different memory requirements. You ll learn in the next section how to use the Windows NT/Windows 2000 Performance Monitor (known as the System Monitor in Windows 2000) to tell if your system is running at its optimal level Microsoft SQL Server 2000 Professional Skills Development

39 The System Monitor The System Monitor The Windows 2000 System Monitor isn t actually part of SQL Server, although it can be used to monitor SQL Server performance. It s a graphical tool used to chart the activity on a SQL server, real time, as it is happening. The System Monitor has been renamed from its predecessor, the Performance Monitor in Windows NT 4. The SQL Server features discussed here for the System Monitor also apply to the NT 4 Performance Monitor, although some of the terminology has been changed. The System Monitor measures performance by the usage of objects and counters. The System Monitor allows you to monitor processors, memory, cache, threads, and processes. Each System Monitor object has counters associated with it that can be used for monitoring performance. You can use these counters to monitor the following SQL Server activities: SQL Server I/O SQL Server memory usage SQL Server user connections SQL Server locking Replication activity The System Monitor allows you to perform the following activities: Monitor any number of computers. Create log files containing data about the objects and computers being monitored. Add alerts and notifications when certain conditions exist. View and update charts that reflect current activity. Export data gathered from System Monitor charts, logs, alert logs, and reports. Run applications contingent on user-defined counter values. View current activity reports, or create reports from existing log files. View current activity reports, or create reports from existing log files. Save individual chart, alert, log, or report settings, or the entire workspace setup for reuse when needed. Objects and Counters The System Monitor consists of counters and objects. A System Monitor counter is an object that is added to a System Monitor chart. Objects are Microsoft SQL Server 2000 Professional Skills Development 15-39