Enterprise Performance Tuning: Best Practices with SQL Server 2008 Analysis Services. By Ajay Goyal Consultant Scalability Experts, Inc.

Transcription

1 Enterprise Performance Tuning: Best Practices with SQL Server 2008 Analysis Services By Ajay Goyal Consultant Scalability Experts, Inc. June 2009

2 Recommendations presented in this document should be thoroughly tested before being implemented in a production environment. While recommendations are listed in an itemized format, they should not be considered standalone. All recommendations have dependencies and implementation of any recommendation in isolation can be detrimental to performance. Information in this document is subject to change without notice. Complying, with all applicable copyright laws is the responsibility of the user. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without the express written permission of Scalability Experts Inc. If, however, your only means of access is electronic, permission to print one copy is hereby granted. Scalability Experts Inc. may have patents, patent applications, trademarks, copyright, or other intellectual property rights covering subject matter in this document; except as expressly provided in any written license agreement from Scalability Experts Inc., the furnishing of this document does not give you any license to these patents, trademarks, copyright, or intellectual property. Copyright 2008 by Scalability Experts Inc. All rights reserved. Microsoft BackOffice, MS-DOS, Windows, Windows NT, Windows 2008, SQL Server 2008 are either registered trademarks or trademarks of Microsoft corporation in the USA and /or other countries. Other product and company names mentioned herein may be trademarks of their respective owners.

3 Contents Introduction... 4 Best practices for Cube Design... 4 Designing Cube Dimensions... 4 Use Attribute Relationships... 4 Use Hierarchies Effectively... 4 Use Numeric keys for dimensions... 5 Use Usage based optimization... 5 Set AttributeHierarchyEnabled to False... 6 Optimizing SQL Source queries... 6 Designing Partitions... 6 Best Practices for SSAS Configuration... 7 SSAS configuration... 7 Conclusion... 9

4 Introduction SQL Server 2008 Analysis Services provides many architectural enhancements that provide better Cube processing and query performance when compared to SQL Server 2005 Analysis Services. This whitepaper focuses on tips and best practices that help when deploying a large data warehouse. Best practices for Cube Design An efficient cube design ensures that the cube is processed faster and that the user queries provide a good response time. Based on industry experience, these best practices ensure that we see good performance increases in cube processing as well as query executions. Designing Cube Dimensions When designing Cube Dimensions for large data warehouses, it is common to have a few large dimensions with many millions of members. Optimizing these dimensions ensure that SSAS does not spend too much time in aggregating attributes that no one uses. Creating an efficient Cube includes creating aggregations and hierarchies only on the attributes where it makes sense. While this may seem obvious, it helps to take a second look - especially on large data warehouses - as every little change can result in large storage savings and improving performance. Use Attribute Relationships Define cascading attribute relationships (for example, Year Month Day) when designing dimensions. Attribute relationships help improve performance by 1. Creating indexes. The data is persisted on disk, so Analysis Services does not have to go through key attribute to build cross products 2. Aggregations are built during Cube processing. These can be reused for related user queries. While designing attribute relationships, consider if the relationship is going to be flexible or rigid. In a flexible relationship, members can move around during dimension updates. If the relationship is rigid, attribute relationships are guaranteed not to change every time the dimension is processed. An example for rigid dimension would be Location, where a city will always be in a particular state and country every time the dimension is processed. However, the relationship between a customer and Location can be flexible, as the customer can move to a different location. It is important to perform a Process Index after performance a Process Update on Dimension with flexible relationship. This is because Process Update may result in all aggregations to be dropped if the dimension contains any updates. With no aggregations available, query performance will be adversely affected. Use Hierarchies Effectively In a Natural Hierarchy, all participating attributes have direct or indirect relationship with the top most level attribute in the Hierarchy. For example, store, city and state are related to country attribute which will be the highest attribute in the Hierarchy. An advantage of creating Natural Hierarchies is that the data is materialized to disk, which improves query performance. Natural Hierarchies are automatically

5 considered as candidates for aggregations. Unnatural Hierarchies are sometimes used to compare unrelated attributes. For example, an analyst may want to get sales by City and Gender. An Unnatural Hierarchy must contain at least two attributes that do not have any relationships defined. Analysis Services does not materialize such hierarchies to disk and needs to calculate these on the fly at runtime. In SQL Server 2008, use the attribute designer to create cascading attribute relationships to take advantage of natural hierarchies. Use Numeric keys for dimensions It is common to have a few dimensions on a large data warehouse to have 1 Million+ members. On such dimensions, use numeric key columns instead of string or composite columns. Processing numeric keys takes up lesser storage space and is much faster to process than string or composite columns. As a best practice, continue using surrogate keys in relational tables for more efficient indexing. You can still use a string column as the attribute name, while still using numeric key column for the attribute key. Use Usage based optimization If usage-based optimization is used correctly, you can consistently reduce the length of long-running queries by using usage-based optimization to create smart aggregations, as shown in Figure 1. Figure 1: Query performance with Usage Based Optimization When using usage-based optimization, you should set the aggregation level to 100% (which is now the default). You should lower it only if processing time is unacceptable or aggregation size increases above the threshold (for example, 1.5 times the size of the facts). Enable the Query Log in Analysis Services Server Properties to save the user query statistics to a table in SQL Server database. Analysis Services will log query data to this table that will later be used by the Usage based optimization wizard. Once you have enough data collected, in the SQL Server Management

6 Studio, right-click on the partition that you want to define Usage based optimization, and select the Usage Based Optimization Wizard (UBO Wizard). Be sure to optimize the long running queries using the UBO Wizard. For more information on Usage based optimization, refer to this technical note: Reintroducing Usage-Based Optimization in SQL Server 2008 Analysis Services Set AttributeHierarchyEnabled to False Consider setting the AttributeHierarchyEnabled to False if the Attribute contains one value for each distinct value of the key attribute. For example a dimension may have more than a million customers with unique telephone numbers and addresses. These attributes are not a good candidate for group by clause and are mostly there for reference. Enabling attribute hierarchies on high cardinality attributes generally causes poor query performance. The result will also not be very useful. Alternatively, consider setting AttributeHierarchyOptimized to NotOptimized and setting GroupingBehavior to DiscourageGrouping if you still want to browse such attributes and want to set AttributeHierarchyEnabled to True. Optimizing SQL Source queries Dimension processing can create a large number of queries to the source relational table based on the number of attributes for each dimension. Dimension tables are not as frequently updated as fact tables and so we can afford to have more indexes on these tables without affecting inserts or updates. To optimize these queries, use the Database Engine Tuning Advisor on a Profiler trace of a Dimension query. Target long-running queries for large data warehouses and add every suggested index to the table. Remember not to use this approach for Fact tables as it will adversely affect the ETL bulk load performance on the Fact tables. Designing Partitions Partitions help divide data in a measure group into separate physical units. Each Partition can be placed on separate disk that helps in improving the overall processing performance and query performance. Keep partitions fewer than a few thousands. Having too many partitions will increase the cost of metadata operations. Aim to have less than 2,000 partitions for a measure group. At the same time, avoid designing partitions that contain fewer than two million rows. On a large data warehouse, fact tables can have 1+ billion rows, and having a small partition can result in too many partitions being created Process partitions in parallel. To get the most out of the hardware, process multiple partitions in parallel. When processing the cube, do not use the default Let the server decide setting for parallelism. From the performance testing, the ideal setting is twice the number of processors in the server. For example, if the server has four dual-core processors installed, set the number of parallel tasks to 16.

7 Have enough physical memory installed to prevent temp file usage. If the server does not have enough physical memory installed for processing aggregations, SSAS will have to use temp files for processing. This can increase the processing time. Target Partition sizing. The general recommendation is to target 20 million rows or 250 MB per partition. With a large data warehouse, this will result in too many partitions. Experiment with partition sizes between 500 MB to 1 GB. For even larger data warehouses, aim at 3 GB per partition. Spread partitions on multiple spindles. Keep each partition on separate disks. When using SAN for storage, ensure each LUN is spread across several disks. This ensures maximum IOPS available for performing queries. Identify aggregation considerations for multiple partitions. A large data warehouse typically contains historical records for a long period of time. Measure groups when partitioned on time, are a good candidates for having different aggregation design for different partitions of the same measure group. For example, as new data is loaded into newer partitions, they are the most heavily queried partitions. You can define more aggregations on these new partitions while decreasing the number of aggregations on older partitions. This saves disk space and keeps the size of the cube small. Use Query Binding and partition slicing Ensure that the query that is defined for each partitioned is optimized for best execution plan. If the underlying tables are also partitioned, this will ensure optimal performance while retrieving data for cube performance. For ROLAP and Proactive caching, manually specify the Partition slice in the properties of each partition. Best Practices for SSAS Configuration SSAS configuration For large data warehouses, keep Analysis Services on a dedicated server, separate from the Relational Star Schema. This reduces contention between SQL Server and Analysis Services for hardware resources. For best performance, choose a 64-bit platform so that SSAS can use more than 3 GB memory. Having multiple processors ensures parallelism that gives faster cube processing and a better query response time. To help sizing the initial hardware requirements, each hardware vendor may have reference architecture for their hardware. This can be useful when deciding on the server hardware required for the data warehouse deployment.

8 Threadpool\Query\MaxThreads Set this property to two times the number of processors on the server. If you have four Dual-core processors, set the value to 16. This setting may not improve the performance on individual queries, but it allow more queries to be run concurrently. Threadpool\Process\MaxThreads Set this property to ten times the number of processors on the server. The default value of 64 should be used if the server has fewer than eight processors. For example, if you have a 4 Dual-core processor server, set the value to 80 (4 Processors x 2 Cores x 10). This property controls the number of threads used by the storage engine during querying operations as well as during processing operations. Preallocate memory Preallocate memory setting can be found in the msmdsrv.ini file. It is used to pre-reserve memory for Analysis Services when the service is started. For servers where Analysis Services is used along with other services, using Preallocate can provide for more stable memory configuration. It has most impact on Windows Server 2003 where processing time was reduced by up to 50% in testing. Windows Server 2008 has better memory management that makes using Preallocate optional. If you use memory preallocation with SQL Server 2008 (or SQL Server 2005), use a value that is low enough to ensure that sufficient memory remains for other processes on the computer (avoiding paging) and high enough for Analysis Services (use the peak value for the Process: Private Bytes counter for the msmdsrv instance to establish this value). If you set the memory preallocation value too high, you will starve other external processes on the server and the result will be memory paging. If you are running the SQL Server relational engine on the same server as Analysis Services, you should also limit the maximum amount of memory used by the SQL Server relational engine. While there is no hard limit for how much memory you should leave for the operating system, on a server with 16 GB or more memory, 10% to 20% is generally sufficient if no other services are running on the same server. Set the memory preallocation to a value that is high enough to ensure that Analysis Service preallocates sufficient memory for Analysis Services query and processing activity on the server such that it does not need to request any additional address space. To determine this value, monitor the peak value for the Process: Private Bytes counter found in the Performance Monitor tool for the msmdsrv instance. The peak value establishes the maximum value for memory preallocation that you need to set. When you cannot set the preallocation value this high, set it as high as possible without starving other processes and the operating system. For more information on Preallocate setting, refer to this technical note: Running Microsoft SQL Server 2008 Analysis Services on Windows Server 2008 vs. Windows Server 2003 and Memory Preallocation: Lessons Learned

9 CoordinatorBuildMaxThreads This value indicates the maximum number of threads used to build indexes. Experiment with a value of - 4. This creates 4 threads per core. If it results in very high CPU usage and affects query performance, reduce the value to -2. Conclusion This guide provides a starting point for configuring Analysis Services for large scale data warehouse deployments. Each environment has specific requirements and could use best practices not mentioned in this document. For more information refer to these documents: Best Practices for Data Warehousing with SQL Server 2008 Scaling Up Your Data Warehouse with SQL Server 2008 Large data warehouses need special emphasis on cube design and reducing processing performance to make them useful for end users. Following these best practices and continuous monitoring will ensure maximum performance gains for most environments.