SQL Server Enterprise Edition Kathi Kellenberger is a Sr. Consultant with Pragmatic Works. She is author of Beginning T-SQL 2008 and co-author of Beginning T-SQL 2012, SQL Server MVP Deep Dives and Professional SQL Server 2005 Integration Services. Kathi enjoys speaking and writing about SQL Server. When not working with SQL Server, Written By: Kathi Kellenberger she enjoys spending time with family and friends and singing.
Contents Security 4 Transparent Data Encryption 4 Extensible Key Management 4 Fine Grained Auditing 4 Performance 5 Hardware Limits 5 Table Partitioning 6 Data Compression 6 Columnstore Index 8 Star Join Query Optimizations 8 Change Data Capture 9 Resource Governor 9 Integration Services Advanced Tasks, Transforms, and Adapters 10 Peer-to-Peer Replication 10 High Availability 11 Failover Clustering 11 High Performance Database Mirroring 11 AlwaysOn Availability Groups 13 Fast Recovery 14 Database Snapshots 14 Mirrored Backups 15 Online Index Rebuild 15 Online Page and File Restore 15 Conclusion 16
SQL Server is offered in several editions. Standard Edition is perfect for departmental databases that do not have high performance or availability requirements. Enterprise Edition, on the other hand, is intended for enterprise-wide databases and data warehouses. Database administrators face the challenge of keeping costs low while providing the required features and functionality for enterprise databases running on SQL Server. Database administrators may wonder which features make Enterprise Edition the smartest decision. The unique features of Enterprise Edition support three goals: Security, Performance, and Availability. Unlike other database platforms, the editions of SQL Server are complete. With SQL Server, you will not pay extra for features like database compression, table partitioning, or clustering when purchasing Enterprise Edition. In addition to the features supporting security, performance, and availability, SQL Server Enterprise also includes every feature available in the Business Intelligence Edition including the new BISM Tabular model and the exciting new data visualization tool, PowerView. www.pragmaticworks.com PAGE 3
Security The media reports data breaches on a weekly basis. SQL Server is the most secure database platform available today with the least vulnerabilities of any major database platform since 2002. In addition to the stellar security record of all the editions, companies using Enterprise Edition can take advantage of three additional security features. Transparent Data Encryption Transparent Data Encryption (TDE) encrypts data at rest, a requirement of many privacy regulations and laws. This means that the data in the database files is encrypted. TDE protects the data in case of the theft of backup tapes, backup files, snapshots or the database files themselves. When a database has been encrypted with TDE, a thief cannot perform a restore or attach the database files without the database encryption key and certificate used when enabling TDE. TDE requires no changes to applications, reports or connection strings and is easy to implement. The data is automatically encrypted as the pages are written to disk and decrypted as the pages are read from disk to memory. TDE also encrypts the tempdb files on a server where TDE is enabled on at least one database. Extensible Key Management The most convenient way to store certificates and keys when using TDE is the on the database server. Unfortunately, that is the riskiest way to store the keys. Keys stored on the database server can be compromised. Hardware Software Modules (HSM) are available from hardware vendors to store and manage keys. Extensible Key Management allows HSM to register with SQL Server which allows the keys used to encrypt databases to be stored and managed in HSM. In addition to the storage and management of the keys, you can also take advantage of advanced features available with HSM such as key rotation and bulk encryption. Fine Grained Auditing In order to meet auditing requirements for your database, SQL Server provides SQL Server Audit to record events at the server and database levels. SQL Server Audit is highly configurable enabling you to meet audit requirements set by government standards or those of your company. SQL Server Audit is built on Extended Events, a very lightweight, scalable technology for recording SQL Server events. Beginning with SQL Server 2012, each edition of SQL Server supports Basic Auditing, auditing events at the instance level. SQL Server Enterprise Edition features Fine Grained Auditing, additional auditing at the database level. SQL Server Audit can capture information about logons, permission changes, object access, role membership changes, data manipulation and access, and much more. The audit information can be saved in files or the Windows Event log. www.pragmaticworks.com PAGE 4
Performance SQL Server Enterprise Edition is ready for the largest, most demanding workloads. The SQL Server Customer Advisory Team (SQL CAT) at Microsoft s headquarters in Redmond, assists customers with the largest database projects in the world. The SQL CAT s site contains white papers, best practices and case studies so that everyone can take advantage of the techniques they use to achieve success. Table 1 contains the top statistics from SQL CAT as presented at the 2012 PASS Summit. Category Largest single database Largest table Biggest total data, 1 application Highest database transactions per second 1 server (from Perfmon) Fastest I/O subsystem in production (SQLIO 64k buffer serial read test) Fastest real time cube Data load for 1 TB Largest MOLAP cube Metric 350 TB 1.5 trillion rows 88 PB 130,000 18 GB/sec Millisecond latency 30 minutes 24 TB Table 1: The top statistics from SQL CAT Hardware Limits Today s hardware is reaching heights unimaginable just a few years ago. SQL Server Enterprise Edition supports the most modern hardware. Table 2 compares the hardware limits among Standard Edition, Business Intelligence Edition and Enterprise Edition. Feature Standard Business Intelligence Enterprise Compute Capacity Limited to lesser of 4 sockets or 16 cores Limited to lesser of 4 sockets or 16 cores Maximum Memory 64 GB 64 GB OS Max Table 2: Hardware limits OS Maximum when licensed by core. (20 when licensed by CAL) www.pragmaticworks.com PAGE 5
Table Partitioning Today s data warehouses commonly reach tens to hundreds of terabytes. Companies demand smaller and smaller windows for loading new data into those data warehouses. Table partitioning is a way to meet those demands. New data is loaded into a separate table. Then with just a metadata operation, the separate table can be added to the main table as a partition. Even with hundreds of millions of rows, the data can be added to the main table in milliseconds. There are two levels of compression: row and page. Row level compression stores fixed-width data types in a variable-length format. When compressed, trailing spaces are removed from char columns and fixed width columns storing numbers take up only the bytes required to store the value. For example, the number 1 in an INT column will only take up one byte instead of four. Figure 1 illustrates row level compression. With table partitioning, tables and indexes can be partitioned with no changes to the reports, processes, or applications that access the data. Partitions can be stored on different volumes. If the data is partitioned by date, older, less accessed data can be stored on slower disks, while the most recent, most used data can be stored on newer generation storage, such as Violin memory drives. Table partitioning is a great tool for managing large volumes of data, but it can also improve performance. When the optimizer determines that all of the rows to satisfy a query reside on a subset of the partitions, the other partitions are ignored thus reducing I/O. Data Compression One of the most popular Enterprise Edition performance features can also decrease costs. Database Compression reduces storage requirements and improves performance especially when applied to workloads with a low ratio of writes to reads. Performance can be improved by compression because the data is stored in a smaller number of pages and, therefore, less I/O is required when reading data into the buffer. The data is compressed and uncompressed in memory. Figure 1: Row level compression Unicode characters with nchar and nvarchar datatypes take up two bytes even if the language used does not require two-byte characters,plus one byte for Unicode metadata. When row level compression is applied in this situation, Unicode characters are stored as just one byte. Row level compression is the mildest level of compression. For even greater compression, page level compression can be used. Page level compression automatically includes row level. In addition, row level encryption adds two levels of compressions: prefix compression and dictionary compression. www.pragmaticworks.com PAGE 6
Prefix compression works by replacing repeating strings at the beginning of each value with a reference to an anchor tag. Figure 2 shows an example of data before page compression is applied. based array that is anchored in the compression information in the header of the page. Figure 4 shows how this data looks after dictionary compression. For example, the value 4b appears twice in the data. That value is saved in the compression information structure and is replaced with the value 0. Figure 2: Data with repeating prefixes After applying prefix compression, the data looks like Figure 3. The widest value from each column is saved in the anchor tag in the compression information structure created when compression is applied. The values are then replaced with the number of characters that match the widest value and the remaining characters. For example, if the widest value is aaabcc, the first three characters of aaaccc match, therefore, aaaccc is replaced with 3ccc. Figure 4: After dictionary compression While compression provides much better performance when reading data, CPU performance is decreased, especially with page compression, when updating data. Page compression should be applied to data that has very few updates. SQL Server provides dynamic management views and functions to help you determine which tables will benefit from compression. Figure 3: After prefix compression Compression can be applied to tables, both heaps and clustered indexes, non-clustered indexes, and individual partitions. This allows you to determine, at a very granular level, how to apply compression to attain the highest benefits with the best improvements to performance. Dictionary compression is applied after prefix compression by replacing repeating values anywhere in the page with a zero Columnstore Index www.pragmaticworks.com PAGE 7
Columnstore Index is the most exciting SQL Server 2012 feature for data warehousing. This new type of index, built on xvelocity in-memory technology, takes the complexity away from designing non-clustered indexes and provides amazing performance improvements over conventional non-clustered indexes. When using a Columnstore index, no other non-clustered indexes are needed. All eligible columns in the table, determined by data type, can be included in the column store index in any order. Columnstore indexes store the data in a very different way than traditional row stores. Each page contains the data from just one column. By storing one column on a page, there is a high chance of repeating values, especially for a fact table which usually contains many repeating values in a column, and a high degree of compression. This allows the Columnstore index, even if it includes every column from the table, to take up less space than the original table. Figure 5 shows the difference between traditional row store and Columnstore data. When accessing the data, only the pages that contain columns involved in the results are read, decreasing the I/O by tens or hundreds of times. SQL Server uses a new batch processing mode when retrieving data from a column store index which reduces the CPU requirements for the query. Column Store Indexes provide ease of development, advanced performance gains, and take up less space on disk than comparable non-clustered indexes. No changes to queries, applications or reports are needed; SQL Server automatically takes advantage of column store indexes. You will see the biggest performance improvement for queries that summarize the data over all of the rows and retrieve only some of the columns. Star Join Query Optimizations Another feature targeted especially for data warehouses is star join query optimizations. Typical data warehouse queries join fact tables to dimension tables with filters applied to the dimensions. This optimization uses bitmap filters based on the dimensions to filter out non-qualifying rows from the fact table before joining thus improving query performance by approximately 20%. Figure 5: Row vs. column store www.pragmaticworks.com PAGE 8
Change Data Capture Change Data Capture (CDC) records incremental changes in data that can then be used to keep data warehouse databases up to date. CDC is very low impact, monitoring the log for transactions and storing the differences in special shadow tables. To use the CDC rows in the ETL process, SSIS has tasks and components built especially to retrieve and manage the rows in the shadow tables. Figure 6 illustrates how CDC works. Resource Governor Resource Governor guarantees predictable performance of high priority workloads. When configured, different workloads based on users, databases or applications are assigned CPU and memory limits. For example, Resource Governor can be used to decrease the effects of runaway queries. Figure 7 shows a possible configuration for a server with priority given to executive reports. In SQL Server 2012 a cap feature was added to Resource Governor. The previous version would only apply the Min and Max settings if the server resources were at 100% utilization and all allocation Resource Groups were currently occupied by user transactions. Using Resource Caps in SQL Server 2012, you can ensure that a Resource Group will not exceed the amount of allocated Memory or CPU. Figure 6: Change Data Capture In addition to CDC for SQL Server databases, new with SQL Server 2012 is Oracle Change Data Capture by Attunity. This allows you to continuously copy data from Oracle databases. Figure 7: Resource Governor www.pragmaticworks.com PAGE 9
Integration Services Advanced Tasks, Transforms, and Adapters SQL Server Integration Services (SSIS), the SQL Server ETL tool, has a wealth of tasks, transforms, and adapters decreasing the need to create custom scripts. Enterprise Edition contains several advanced transforms, tasks, and adapters meant for enterpriselevel ETL. Table 2 lists the Enterprise Edition tasks, transforms, and adapters. Tasks Transforms Adapters Persistent (high performance) lookups Persistent (high performance) lookups High performance Oracle destination Data mining query transformation Data mining query transformation High performance Teradata destination Fuzzy grouping and lookup transformations Term extractions and lookup transformations Fuzzy grouping and lookup transformations Term extractions and lookup transformations SAP BW source and destination Data mining model training destination adapter Persistent (high performance) lookups Dimension processing destination adapter Data mining query transformation Partition processing destination adapter Fuzzy grouping and lookup transformations Change Data Capture components by Attunity Connector for Open Database Connectivity (ODBC) by Attunity Table 2: Advanced Transforms and Adapters Peer-to-Peer Replication Peer-to-peer replication is a scale-out solution involving databases on multiple servers called nodes. For example, databases for inserts and updates may be spread geographically or load balanced. All transactions are performed locally and then replicated to the other servers. This is also beneficial as a disaster recovery solution. www.pragmaticworks.com PAGE 10
High Availability Database outages cost companies millions of dollars in lost sales and productivity. SQL Server Enterprise Edition has several features to ensure that SQL Server databases attain a very high level of uptime, up to 99.999%. The features can be used independently or combined to provide complete high availability and disaster recovery protection. Failover Clustering Failover clustering is hardware redundancy providing automatic failover at the instance level in case of a server failure. A SQL Server Failover Cluster is composed of one or more SQL Server instances hosted on two or more clustered Windows nodes and shared storage. Enterprise Edition supports the number of nodes allowed by the version of Windows where it is installed. Standard Edition supports two node clusters. Figure 8 shows a possible cluster configuration. High Performance Database Mirroring Database Mirroring (DM) has been deprecated, but it is still being used, especially for pre-2012 SQL Server instances. DM works by keeping a copy of the database, called a mirror, synchronized with the principal, or active, database. The transactions are automatically copied to the mirror in real or almost real time. DM can be configured in three ways: high-safety, high-safety with a witness, and high-performance. The two high-safety options are also available with Standard Edition. High-safety means that the transactions must be committed on the mirror before they are committed on the principal guaranteeing that the mirror is always in sync. When high-performance mode is used with a witness, failover is automatic. Without a witness server, failover is manual only. Figure 9 shows Database Mirroring configured in High Performance with a Witness. Figure 8: Failover Clustering By using a virtual server name, clients can automatically reconnect after a failover. During a failover, the instance goes down on the first node, the second server gains ownership of the storage, and comes up on the second node. Figure 9: Database Mirroring in High Performance with a Witness mode www.pragmaticworks.com PAGE 11
Enterprise Edition also supports a third mode, High Performance. In this mode, the mirrored database keeps up as best as it can, and transactions are committed on the principal first. High Performance mode is perfect for disaster recovery and is often combined with SQL Server Failover Clustering. Failover is manual only in High Performance mode and data loss is possible when failing over. Figure 10 shows how High Performance mode is configured. Figure 10: Database Mirroring in High Performance Mode Besides being deprecated, there are several limitations to database mirroring as listed below: Mirroring must be configured individually for each database. Only one mirrored copy is allowed. A special connection string is required for applications to automatically reconnect after a failover. Reports cannot be offloaded to the mirror. A workaround to this limitation can be done with Database Snapshots. Mirroring reserves one CPU Thread per Mirrored Database, this can limit the number of databases that can participate in Mirroring creating scalability issues. www.pragmaticworks.com PAGE 12
AlwaysOn Availability Groups AlwaysOn Availability Groups is the replacement for Database Mirroring. It has been reengineered to provide better performance and much greater functionality than Mirroring. With AlwaysOn Availability Groups, multiple databases in a group can be replicated to four secondary nodes. Each replica is configured individually allowing both high availability with automatic failover and disaster recovery. Each secondary can be configured for synchronous or asynchronous commit. Synchronous commit means that the transactions must be committed on the secondary before being committed on the primary. Up to three nodes may be configured for synchronous commit, the primary and two secondaries. The primary and one secondary can be configured for automatic failover. Additionally, secondary replicas are available to offload reporting, transaction log backups and copy-only full backups. Here is a list of the benefits of AlwaysOn Availability Groups: Multiple databases failover as a groups. Groups of databases are configured together. Up to four secondary copies can be configured. A virtual name, called a listener always points to the primary database. Secondary copies can be used to offload reporting, transaction log backups and copy-only full backups. With a special connection string, read-only connections can be automatically routed to the readable secondary. Each secondary is configured separately to achieve both high availability and disaster recovery. Instead of a witness server, AlwaysOn Availability Groups use Windows Clustering to control automatic failover. AlwaysOn Availability Groups more efficiently manages Thread usage and can Scale to a much greater rate than Database Mirroring Figure 11 shows a possible configuration with four nodes. Figure 11: AlwaysOn Availability Groups www.pragmaticworks.com PAGE 13
Fast Recovery After a restart of SQL Server, a crash, or a failover, the databases must recover. Recovery has a Redo phase where all committed transactions in the log are applied to the database. During the Redo phase, no connections to the database are possible. The next phase is the Undo phase where uncommitted transactions are rolled back. In Enterprise Edition only, data access is allowed during the Undo phase. This means less downtime since applications can reconnect to the databases more quickly as shown in Figure 12. Database Snapshots Database backups are an important line of defense for the database administrator especially when data is accidently deleted or changed. In addition to database backups, it is possible to create a database snapshot that can be used to easily revert back to a previous state. This is beneficial for saving the data at a point in time before a large data load or some other risky operation so that you can get the database back to before the operation very quickly. They are beneficial for any reason that you want to quickly save the database exactly how it looks at a particular point in time. Database snapshots are readable, so they can also be used for querying or reporting. Database snapshots work by creating a sparse file with pointers pointing at the actual database pages. Over time, as the database is updated, the database snapshot copies the original data pages to the snapshot. Figure 13 demonstrates how the snapshot file looks to start and then after changes to the database have taken place. Figure 12: Fast Recovery Figure 13: Database snapshots www.pragmaticworks.com PAGE 14
Mirrored Backups SQL Server Enterprise Edition supports mirrored backups as extra insurance in case of a problem with a backup location. You can create up to four backups as shown in Figure 14 at one time decreasing the risk of a restore failure. Online Index Rebuild Removing fragmentation from non-clustered indexes is a very important maintenance task affecting how well the database performs. Traditionally, rebuilding the index has required that the table be unavailable to applications while the index is being rebuilt. With Enterprise Edition, you can specify that the index is rebuilt while online increasing the availability of applications. Online Page and File Restore Figure 14: Mirrored backups File and page restores are done while the rest of the database is online with Enterprise Edition. If the database has been created with multiple files, online restores can be done as long as the primary file group stays online and file group backups have been performed. In the case of page corruption, after determining the corrupt page, just the page can be restored, avoiding the downtime and possible data loss of restoring the entire database. www.pragmaticworks.com PAGE 15
Conclusion For departmental databases, SQL Server Standard Edition works great. When your databases require high performance, enhanced security, and demanding high availability, Enterprise Edition is the best choice. www.pragmaticworks.com PAGE 16