Database Maintenance Essentials

Transcription

1 Database Maintenance Essentials Brad M McGehee Director of DBA Education Red Gate Software

2 What We Are Going to Learn Today 1. Managing MDF Files 2. Managing LDF Files 3. Managing Indexes 4. Maintaining Statistics 5. Checking for Corruption 6. Creating Backups That Will Restore 7. Managing Maintenance Jobs Not every recommendation may be appropriate for your environment. Be sure to test in a non-production environment before trying these recommendations out.

3 Managing MDF & LDF Files There is a myth that MDF and LDF files manage themselves. In reality, for optimal performance, DBAs must take full responsibility for managing them, as we will see in the next few slides.

4 Managing MDF Files When creating a new database, it is important for it be pre-sized to its future expected size about one year ahead. Even though only a fraction of the database will be used after the database is in production, that s not a problem. Your guesstimate will probably not be accurate, and that s OK. If you monitor the amount of data growth, you will soon see how accurate your guesstimate was. If you underestimated, you can manually grow the database to a larger size using the trending information you have collected as you have observed data growth over time. If you overestimated the database s size, that s not a problem either. Just let the data continue to grow inside the database, and eventually your application will use it. Don t shrink it.

5 Managing MDF Files While there are many benefits for pre-sizing databases, there are two important ones you need to be aware of. First, by pre-sizing databases, SQL Server won t have to depend on autogrowth to grow the database for you, preventing the performance hit when autogrowth occurs. Second, each autogrowth can contribute to physical file fragmentation, which can hurt IO performance. I have seen databases with 10s of thousands of autogrowths, creating serious physical file fragmentation. Regularly monitor database growth, and if you see that empty space is running out, then manually grow the file to its estimated size a year from now, & repeat as needed.

6 Managing LDF Files LDF files, like MDF files, should be presized to their expected size; and at any one time, they may only be partially full, which is OK. Calculating the estimated log size for a newly created database is difficult to calculate, as there are so many factors that affect the size of LDF files. (e.g. T-log backup schedule, level of activity, recovery model, etc.) I use a guesstimate of around 5-10% of the expected size of the database files one year from now. If I guesstimate that the total size of the MDF and NDF files will be 500GB one year from now, then I would estimate the LDF file to be between 25 GB and 50 GB. A guesstimate is better than letting autogrowth grow the LDF file for you.

7 Managing LDF Files If you incorrectly guess the LDF file size of a database, which you probably will, there are two possible ways to deal with it: Regularly monitor log growth, and if you see that empty space is running out in the log file, or autogrowths are occurring, then manually grow the file as needed so autogrowth doesn t have to occur. If you overestimated the log file s size, that s not a problem either. Unless it is a huge amount of unused space, I would generally not reduce the size of the log, leaving it at its overestimated size. On rare occasions, LDF files will grow wildly. If so, first determine the cause and fix it. If the growth was so wild that the LDF size is much greater than needed, and you don t expect more wild growth spurts, then you may want to consider manually shrinking it.

8 Managing LDF Files While there are many benefits for pre-sizing transaction logs, there are three important ones you need to be aware of. First, by pre-sizing databases, SQL Server won t have to depend on autogrowth to grow the database for you, preventing the performance hit when autogrowth occurs. Second, each autogrowth can contribute to physical file fragmentation, which can hurt IO performance. Third, each autogrowth that occurs creates what are called virtual log files (VLFs). Too many VLFs in a log file can cause performance issues. By preventing autogrowths, you also prevent having too many VLFs. See my presentation, Inside the SQL Server Transaction Log, for more information on managing VLFs.

9 Using SQL Monitor to Help You Out Managing MDF and LDF Files Drill down to instance level to find disk sizes and available space Drill down to database level to find file size information Disk space alert Performance Monitor Counters that measure database Total Size, Data Size, Log Size, Log Space Used Create custom metric, for example, to tell you if log file grows wildly Demo

10 Regularly Review Indexing Needs Indexing Needs Change Over time, data in databases, and the use of the data, often changes. This means that the current indexing scheme may need to be changed to keep up with the data and data usage changes. For examples, indexes may need to be added, modified, or removed for optimal query performance. You need to proactively monitor your servers to see if their indexing needs are properly meet. SQL Server will not do this for you.

11 Identify Missing Indexes Other than manual index tuning, which is the ideal solution, one way to identify missing indexes is to use Profiler/SQL Trace to capture a trace file, and then use the Database Engine Tuning Advisor (DTA) to analyze the trace to look for index recommendations. When capturing a Profiler/SQL Trace, use the Tuning template and capture data over a representative time frame. Run the DTA against the trace data, review recommendations, and then add appropriate indexes. Do so regularly, as dictated by needs. Note: you can use sys.dm_db_missing_index_details to help identify missing indexes, but it has many limitations, and I don t recommend using its recommendations unless you really know what you are doing.

12 Identify Unused Indexes Most databases have one or more indexes that were created because they seemed that they might be useful, but they have ended up not being used. Because indexes need to be maintained when data changes in a table, maintaining indexes that are not used are a waste of resources. Periodically, identify unused indexes and remove them. Use the sys.dm_db_index_usage_stats DMV to help you identify unused indexes. Keep in mind that the data in this DMV is cleared out each time SQL Server is restarted, so only run this DMV after the server has been up and running for quite some time.

13 Identify Duplicate Indexes For many different reasons, it is possible for the duplicate indexes to be created using different names. This wastes resources and duplicate indexes should almost always be removed. See the following URL for sample scripts:

14 Index Maintenance: Fragmentation Index Fragmentation Hurts Performance Over time, as indexes are subjected to data modifications, gaps in data on pages develop (internal fragmentation), and the logical ordering of the data no longer matches the physical ordering of the data (external fragmentation). Together, this is referred to as index fragmentation. This is a normal behavior, but must be regularly addressed. Heavily fragmented indexes can lead to poor query performance, especially if scans occur regularly. This is because less data can fit into the data cache and because more disk I/O is required. Because of this, it is important that DBAs regularly detect and remove index fragmentation from their databases on a regular basis.

15 Index Fragmentation Maintenance There are three ways to remove fragmentation from an index: Reorganize: online (Standard and Enterprise Edition) Rebuild: offline (Standard and Enterprise Edition) Rebuild: online (Enterprise Edition Only) Each option has its pros and cons. You must select the option(s) which work best for your environment. I can t tell you what will work best for your environment.

16 What is the Best Way to Defragment Indexes Using the Fewest Resources? There is no easy answer, or one answer fits all. BOL makes these recommendations: If number of pages in an index is < 1000, then leave alone. If avg_fragmentation_in_percent (external) is less <5%, then leave alone. If avg_fragmentation_in_percent (external) is >5% and <30%, consider REORGANIZE. This can be faster for less fragmented indexes. If avg_fragmentation_in_percent (external) >30%, consider REBUILD or REBUILD ONLINE. This can be faster for more fragmented indexes. These recommendations were made up by Paul Randal as a general recommendation, but it is often not always the best solution.

17 My Recommendations Prefer REBUILD over REORGANIZE. If you have a maintenance window, or have Enterprise Edition that allows an online REBUILD, then use REBUILD to defragment indexes. If you don t have a maintenance window, or a very short one, and you don t have Enterprise Edition, consider using a combination of REBUILD and REORGANIZE. Experiment with the thresholds as described in BOL, and reduce them if doing so doesn t greatly increase the time required to run them. I use 100 pages & 20% for my tradeoffs instead of 1000 pages & 30%. Don t defrag indexes than don t need it, or have less than 5% external fragmentation. Run REBUILD or REORGANIZE as often as possible, daily if possible. This spreads out the resources needed for defragging over time and keeps indexes healthier for longer. If you have huge databases, your only option might be to use REORGANIZE, and schedule it to run a pre-defined amount of time each day. Use pre-written scripts, as the will save you a lot of time. ola.hallengren.com

18 Using SQL Monitor & SQL Index Manager to Help Manage Indexes Used SQL Monitor Fragmented Indexes alert to identify highly fragmented indexes. Use SQL Index Manager to help determine what indexes need maintenance. Demo

19 Maintaining Statistics Defragging index fragmentation and updating statistics are closely related and must be considered together. Keep the following in mind when creating statistics updating jobs: If you use REORGANIZE, then you must separately update index and column statistics. UPDATE STATISTICS table_name WITH FULLSCAN, ALL If you use REBUILD, index statistics are automatically updated using FULLSCAN, but you must separately update column statistics. UPDATE STATISTICS table_name WITH FULLSCAN, COLUMNS

20 Checking for Corruption DBCC CHECKDB checks the logical and physical integrity of a database. Ideally, the command should be run before a full database backup is made to identify problems before the backup occurs, although this is not always possible. If a problem is detected, you want to identify, and correct it, as soon as possible. DBCC CHECKDB has some very limited fixing ability, but it should not be counted upon, and only used by experts, as bad data is dropped. Running DBCC CHECKDB is resource-intensive and potentially time-consuming, and should be run during slow times on the server. If you don t have a large enough window to run DBCC CHECKDB before backing up, restore database to another server and run DBCC CHECKDB there.

21 Using SQL Monitor and SQL Backup Pro to Help Avoid Corrupted Databases Use these SQL Monitor alerts: Integrity Check Overdue Page Verification (Ensure it is enabled. Checksum is the best option.) Demo For large databases, use SQL Backup Pro to automate running DBCC CHECKDB on restored databases on a test server in order to prevent the workload caused by DBCC CHECKDB from affecting the production server. You can also set reminders to do this, just in case you forget. Demo later after backup discussion.

22 Creating Backups That Will Restore Production databases should use the Full Recovery model. Create a job to perform full backups daily on all system and user production databases, plus log backups at least hourly (or a similar schedule that best meets your HA needs). Always backup using RESTORE WITH VERIFYONLY to help verify backup integrity. But this is not a guarantee the backup is good. Restore backups to verify that you can restore your databases. Store backups securely (physically & encrypted), and off-site. If you have a limited backup window, or have limited disk space, use backup compression. (SQL Backup Pro automatically compresses backups, saving both disk space and reducing backup time.)

23 Using SQL Monitor and SQL Backup Pro to Help with Backups Use SQL Monitor s Backup Overdue and Log Backup Overdue to help ensure that all data is backed up as you expect. SQL Backup Pro s abilities to: Compress backups to save space and reduce backup time Encrypt databases to meet security needs To automatically test the quality of backups by automatically restoring to another server (previously demoed) Demo Backup & Automatic Restore Features

24 Managing Maintenance Jobs As much as practical, keep maintenance plans the same from instance to instance. Don t duplicate maintenance tasks (e.g. Rebuild indexes, then update index statistics immediately thereafter). Schedule jobs so that they do not overlap one another. Schedule database maintenance tasks during down times or during the least busy time of the day. Monitor jobs to ensure they are working correctly. Don t over-maintain your databases. Find the right balance.

25 Use SQL Monitor to Ensure Jobs Run Successfully Use the SQL Monitor alerts Job Failed or Job Duration Unusual to help ensure that your jobs all run as expected. If either of these alerts occur, you can be notified automatically by .

26 Take Aways From This Session Implementing optimal maintenance plans can greatly affect a SQL Server instances: Availability Performance Database maintenance is an on-going task that never ends. Automate as much as possible using the tools described here to help free up your time for more interesting tasks. A Challenge to You: When you get back to work, evaluate all of your SQL Server instances to ensure that all appropriate maintenance tasks are being performed, and are being performed optimally.

27 Find Out More Free Presentation Downloads at How to Monitor Your SQL Server for Performance and High Availability How to Defragment Indexes for Peak Performance Inside the SQL Server Transaction Log Find Out More About the Following Products: Click Here for a free 14-day trial of the Red Gate SQL Server Toolbelt