Point of View Overview on Key Enhancements and Updates Executive Overview Data and analytics strategies and solutions must be flexible and scalable to meet future needs. They must give users real-time access to data and provide a variety of visualization and graphing choices for examining data. They must securely integrate varied disparate data sources. They must work on public cloud, private cloud, traditional systems and third-party ecosystems in a world of hybrid technology at high performance. And, they must be capable of running on mobile devices of all shapes and sizes so that people can work when and where they need and want, with any type of data - big or small. Avanade Data and Analytics services and solutions help businesses access and analyze their vast stores of data for immediate insights into the dynamics of their business. We combine the power of SQL Server 2014 with our global experience and expertise with the latest Microsoft technologies. We build solutions that help executives and managers make confident, timely business decisions to guide their organizations to success today and into the future. makes it easier and more cost effective to build high-performance, mission-critical applications, enterpriseready data assets, and data and analytics solutions that help employees make better decisions, faster. These solutions have the flexibility to be deployed on premises, in the cloud or in a hybrid environment, and can be managed through a common and familiar tool set. April 1, 2014 brought the twelfth release of Microsoft SQL Server. brings enhancements to the database engine and management tools. Updates to the database engine significantly improve query performance for a wide range of SQL workloads including OLTP applications and data warehousing. New releases in data management provide features to integrate the use of Azure services as a first-class part of an organizations SQL ecosystem. While the updates to the database engine were significant, the rest of the SQL Server suite remained unchanged from SQL 2012. The release of was accompanied by the release of the Microsoft Analytics Platform System (APS), which couples the massively parallel processing (MPP) version of SQL Server PDW with HDInsight, Microsoft s 100 percent Apache Hadoop distribution. APS is a turnkey appliance that provides Microsoft s PolyBase technology, which enables the ability to query and integrate data using T-SQL syntax across the PDW and HDInsight instances with the appliance, plus Azure HDInsight, Hortonworks HDP and Cloudera. APS is the only current SQL SKU that provides the PolyBase capability. The focus of this document will be on SQL Server 2014, APS will be covered in the Avanade Data & Analytics Big Data Point of View 2014 document. This document is not intended to cover all of the features of the SQL Server suite and is focused on the significant updates in this release. 2014 Avanade Inc. All rights reserved.
Introduction enables customers to build mission-critical applications and data analysis solutions using highperformance, in-memory technology across OLTP, data warehousing, business intelligence and analytics workloads without having to buy expensive add-ons or high-end appliances. uses a common set of tools to deploy and manage databases both on premises and in the cloud, which makes it easier for businesses to take advantage of the cloud with existing skillsets. The updates to SQL Server are categorized and will be reviewed as follows: Performance Enhancements High-volume transaction processing (OLTP) Data warehousing Management Enhancements High availability Backup and restore Security Other Minor Updates Analysis services Business intelligence Definitions Azure is Microsoft s cloud services platform previously known as Windows Azure. It was renamed to Microsoft Azure on April 3, 2014. Azure provides data services, application services and networking services through a highly redundant series of data centers across the global that delivers a 99.95 percent monthly SLA. OLTP Online Transaction Processing OLAP Online Analytical Processing In-memory A DMS that primarily relies on main memory for computer data storage, which are faster than disk-optimized databases since internal optimization algorithms execute fewer CPU instructions. Hekaton The code name for the in-memory OLTP feature within the SQL Server database engine for OLTP workloads. xvelocity The code name for the ColumnStore feature within the SQL Server database engine for data warehousing workloads. ACID A transaction processing term that stands for Atomicity, Consistency, Isolation and Durability. Microsoft Mainstream Support Defined as five years from the release date or for two years after the successor product (N+1) is released, whichever is longer. Microsoft Extended Support Defined as 5 years following Mainstream support or for 2 years after the second successor product (N+2) is released, whichever is longer. Microsoft Online Support Defined as at least 10 years from product release. Updates Performance Enhancement High-Volume Transaction Processing (OLTP) Database performance has become an important subject for any database administrator, database analysts and IT directors. That is why Microsoft has focused on the performance factor within to achieve 10x-30x improvement without touching your code whatsoever. They did that through in-memory OLTP with no page buffer. In-Memory OLTP (also known as Memory-Optimized Tables, In-Memory Tables or by its code-name Hekaton) substantially improves transactional performance that is fully transactional, durable and does not require any special T-SQL syntax. 2014 Avanade Inc. All rights reserved. 2
For businesses, this means you can create a memory-optimized table and add it to a memory optimized file group, which is translated to.dll entry points. Moreover In-memory OLTP achieves significant performance and scalability gains by using: Algorithms that are optimized for accessing memory-resident data. Optimistic concurrency control that eliminates logical locks. Lock-free objects that eliminate all physical locks and latches. Threads that perform transactional work don t use locks or latches for concurrency control. Natively compiled stored procedures, which have significantly better performance than interpreted stored procedures when accessing a memoryoptimized table. The use of In-Memory Tables is not viable for all usage scenarios and the implementation of memoryoptimized tables should be preserved for implementations in scenarios that require: High data insertion to appendonly tables where contention is typically high. High read performance against tables that receive periodic batch inserts and updates. High business logic processing within the database with intensive usage of inserts, updates and deletes. represents the first release of In-Memory OLTP and does come with some limitations including a reduced set field types, row sizes, and foreign key and check constraints. Proper review of the limitations is highly recommended before starting an In- Memory OLTP implementation. Performance enhancements in SQL Server 2014 include In-Memory and highvolume transaction processing (OLTP), which both give businesses with large or many databases and a high volume of transactions the ability to provide realtime information in a timely manner. For example, these improvements allow companies with multiple warehouses to have the ability to look up inventory quickly and in real time with potential improvement on customer service and satisfaction. Data Warehousing The traditional data warehouse is under pressure from the growing weight of explosive volumes of data, the expansive variety of data types and the real-time processing velocity of how data is being used to grow and operate the business. Businesses need a logical architecture that can smoothly scale to meet these volume demands with real-time processing power and the ability to manage any data type to rapidly connect the business to valuable insights. Key benefits of Microsoft data warehousing capabilities include: Incorporate a wider variety of data sources that include mobile, social, scanners, photos, videos, sensors, devices, RFID, web logs, advanced analytics, click streams, machine learning and third-party data sources. Query both traditional relational data and these new data types with common T-SQL commands using PolyBase. Scale from tens of terabytes up to multi-petabytes by incrementally adding nodes to your existing infrastructure. Enable users to get results from their queries in near real-time. Queries that took hours can be reduced to minutes and seconds through inmemory and streaming technologies. Provide a trusted infrastructure that gives users confidence in the credibility and consistency of the data. Microsoft claims up to a 100x performance gain in query performance when using In-Memory ColumnStore found in Enterprise Edition. ColumnStore, also known by its code-name xvelocity, was introduced with SQL Server 2012 and is designed for use with data warehouse or data mart workloads. In-Memory ColumnStore stores and manages data by using column-based data storage and column-based query processing. A ColumnStore index works well for mostly read-only queries that perform analysis on large data sets. ColumnStore indexes provide high performance gains for queries that use full table scans. ColumnStore indexes should not be used for queries that perform searches for single values. 2014 Avanade Inc. All rights reserved. 3
Improvements in In-Memory ColumnStores in include allowing updates to tables that use a ColumnStore index, providing near realtime analysis and greater compatibility with standard SQL commands and table structures. Improvements in data compress provide up to 7x data compression over the uncompressed data size. Tables that use a clustered ColumnStore index can have no other indexes. Additionally, its columns can have no foreign key constraints. These limitations must be considered during the design of data models that will leverage a clustered ColumnStore index. In contrast, a non-clustered ColumnStore index can coexist with other indexes and do not have the same foreign key constraints. Advancements in data warehouse capabilities allow organizations to store the ever-growing amount of data as a result of big data and business intelligence trends. Features include PolyBase and ColumnStore indexes which give businesses the ability to combine data from any source or data type, at any velocity into actionable real-time insight. For example, PolyBase enables businesses to take advantage of nonrelational Hadoop data, like social media insights, that are stored in the Hadoop Distributed File System (HDFS), bypassing Hadoop s MapReduce distributed computing engine. As a result, you don t need to understand HDFS (typically written in Java) to take full advantage of non-relational data types. This means that businesses can leverage data warehousing features to bring unstructured data like social insights together. They can make the data more understandable and identify how it relates to other data - both structured and unstructured, such as sales in a retail store. Management Enhancements High Availability This section introduces SQL Server high-availability solutions that improve the availability of servers or databases. A high-availability solution masks the effects of a hardware or software failure and maintains the availability of applications so that the perceived downtime for businesses and users is minimized. Microsoft s customers tout the fact that they re able to achieve the 99.999 percent availability using AlwaysOn Availability Groups. AlwaysOn Availability Groups (AG) was a feature introduced in SQL Server 2012 to provide higher availability at a lower cost with an easier setup, as well as administration over traditional SQL Clustering and Replication (e.g. transactional, merge). AlwaysOn AG replicates databases to redundant servers and avoids the need for expensive shared data storage and identical hardware that was previously required for SQL Clustering. Improvements in AlwaysOn AG features include support for In-Memory Tables and In-Memory ColumnStore indexes, plus Microsoft Azure Virtual Machines may be used as a replica target. The number of replicas has been increased from four to eight, allowing for more options for creating disaster recovery fail-over replicas and geo-located read-only replicas. Replicas may be implemented in Synchronous mode, used for high availability where the replica is used as a fail-over target, and Asynchronous mode, used for disaster recovery and read-only replicas. Synchronous mode changes to the primary database are not marked as being complete until the secondary replica update is complete. This provides full ACID transaction compliance, but comes with performance degradation and care should be exercised to limit network latency between servers. Asynchronous replication mode removes the performance penalty as changes to the primary database are updated to replicas as soon as possible. But, this comes at the cost that a replica may be out of sync with the primary for some period of time or, possibly, if the primary has a failure. Improvements in high-availability give an organization s servers or databases a comprehensive suite of database protection tools. For example, database mirroring is a solution to increase database availability by supporting almost instantaneous failover. This feature has broad benefits to businesses as it protects data in the event of a disaster. For example, a retail business will benefit from highavailability in its point-of-sale registers, keeping them always up and running with no downtime. This way, the business can always complete a customer s transaction with no interruptions. 2014 Avanade Inc. All rights reserved. 4
Resource Governor Enhancements Resource Governor enables you to manage SQL Server workloads and resources by specifying limits on resource consumption by incoming requests. Resource Governor enables you to specify limits on the amount of CPU, physical IO and memory that incoming application requests can use. Resource limits can be reconfigured in real time with minimal impact on workloads that are executing. I/O has been added to Resource Governor, which lets I/O be pooled and tiered following an organization s criteria. This ensures greater scale and performance predictability for SQL Server workloads, especially when running applications in private clouds and environments managed by hosters. Resource Governor enhancements allow organizations with multiple lineof-business applications like CRM, customer portal or order transaction processing to maintain service levels. For example, hosting service providers can leverage Resource Governor as a way to maintain and manage compute usage. This allows them to maintain customer service levels so that the customer never experiences a level of performance lower than agreed upon. Backup and Restore The SQL Server backup and restore component provides an essential safeguard for protecting critical data stored in your SQL Server databases. To minimize the risk of catastrophic data loss, you need to back up your databases to preserve modifications to your data on a regular basis. A well-planned backup and restore strategy helps protect databases against data loss caused by a variety of failures. provides enhanced capabilities to leverage cloud-based storage options for backup and restore, as well as certificate or asymmetric key-based encryptions during backup operations. Cloud-based storage is currently limited to customers with a Microsoft Azure subscription. Introduced in SQL Server 2012 SP1 CU2, the Backup to URL capability in provides SQL Server Management Studio convenience for the setup of an Azure storage Blob for backup and restore. Additionally, SQL Server Managed Backup to Azure is a service that SQL Server provides to manage and schedule database and log backups. It can be configured for instances running on-premises and within Azure Virtual Machines. This means that if an organization experiences an event where there is catastrophic data loss, a well-planned backup and restore strategy has already been established. This enables the organization to be back up and running fast, right where they left off, with data integrity and responsiveness to service level events. Security introduces new permissions to facilitate server-level administration and auditing, including the ability to grant rights to all current and future databases on an instance. Additionally, the server-level permission to provide or restrict access to secured data as a server instance has been added with this release. This means that organizations can be assured they are using enterprise level security for their data assets. With advancements in security, organizations can know what data has been compromised, when it was compromised, and what information was taken. Analyst Perspective Analysts have communicated that Microsoft offers a competitive and expanding set of data and analytics capabilities, as well as packaging and pricing, that appeal to Microsoft developers, independent distributors and now to business users. It does so through a combination of enhanced BI and data discovery capabilities in Office (Excel) 2013 that integrate with data management capabilities in and collaboration, content and user and usage management capabilities in SharePoint with. Microsoft SQL Server continues to be ranked in the Leaders quadrant of the latest Gartner Magic Quadrant reports for Operational Database Management Systems (DBMS), Data Warehouse Database Management Systems and Business Intelligence and Analytics Platforms. 2014 Avanade Inc. All rights reserved. 5
Gartner notes in the Data Warehouse DBMS report that, Customers report a low count of software issues, aboveaverage customer experience and obvious interoperability with Excel (and Office). They also like the easyto-understand licensing and pricing. 2 Gartner gives Microsoft praise for its implementation of In-Memory technologies noted above. In the Operational DBMS report, the analysts state, Microsoft now has in-memory technology for transactions (code named Hekaton) with, ahead of both IBM DB2 and Oracle. 3 Gartner continued its praise of Microsoft in the Business Intelligence and Analytics platform report stating, Of the mega-vendors, Microsoft has made the most progress toward delivering a combination of business user capabilities with an enterprise-capable platform. 4 Forrester Research places Microsoft SQL Server as a Leader in the Forrester Wave : Agile Business Intelligence Platforms, Q3 2014. In the report, Microsoft SQL Server received high scores from clients for product vision, as well as for client feedback collected as part of a customer survey. Forrester notes that, Microsoft received high client feedback scores for its agile, business user selfservice and [advanced data visualization] ADV functionality. Clients also gave Microsoft BI a high score for its product vision. 5 Forrester Research, has also positioned Microsoft as a Leader in The Forrester Wave : Enterprise Business Intelligence Platforms, Q4 2013. Forrester analyst Boris Evelson notes in the report that [what] differentiates the Leaders [is] the completeness, comprehensiveness and integration of the entire BI architectural stack. In addition to increasingly popular features like analytics, data visualization, dashboards and data exploration, the Leaders also provide the pixel-perfect, industrial-strength report writers that companies still need. 6 IDC s research study, underwritten by Microsoft, Capturing the $1.6 Trillion Data Dividend, surveyed 2,020 large and mid-sized organizations in 20 countries across a range of public and public sector industries. IDC research showed a correlation between better outcomes from big data and business analytics projects and greater competitiveness of an organization in its industry or a better ability to fulfill its mission in the public sector. Although correlation does not equate to causation, a growing body of case-based research shows financial and productivity benefits directly linked to better data-driven decision making enabled by big data and business analytics solutions. Secondly, IDC research showed that the improved project outcomes accrue to the more innovative organizations that have embraced a data-driven culture, which has resulted in the introduction of: New Data Types and Sources: The number of data types and/or sources being analyzed is rapidly growing as part of the big data trend. Examples include supplementing transactional data with customer behavior and/or demographic data, data from machines or sensors, geolocation applications, mobile devices, social media or document management systems. New Analytics: This constitutes expanding the number and/or type of analytic techniques or methods used, such as using more predictive analytics or using techniques such as MapReduce to supplement SQL. New Metrics: This entails identifying and developing new key performance indicators, such as introducing new ways of measuring enterprise risk or employee performance, introducing a new performance management methodology or introducing new ways of looking at operations. New Users with Access to Big Data and Analytics Solutions: This consists of expanding the number and type of users (internal or external) who have access to the organization s data and analytics solutions and/or outputs generated from them. 7 Market Players, Vendor Comparisons, Competitive Overview Microsoft SQL Server continues to be a strong player compared to Oracle, IBM and SAP. Microsoft continues to grow the SQL Server business. On the Microsoft FY14 Q4 Earnings Review call, Satya Nadella was quoted, We also had another breakout year for SQL Server. With, we released industry leading in-memory technology across all database workloads of online transaction processing, data warehousing and business intelligence. and we grew our SQL business by more than 19 percent. 8 Microsoft continues to garner market share from Oracle and IBM, but we re seeing offerings like MySQL entering the market that was primarily dominated by SQL Server in the past. 2014 Avanade Inc. All rights reserved. 6
Customer/Project Summary Examples There are many benefits for adopting and upgrading an infrastructure to Microsoft. There are many included advancements or improvements in performance, security and management. But some of the most important reasons why an organization would invest in are to increase scalability and availability. As big data becomes an increasing concern and motive, allows organizations to utilize nonrelational Hadoop data, bring structure and unstructured data together for analysis and real time insights. Organizations also want to take advantage of cloud economics. allows organizations the ability to connect and establish a genuine hybrid cloud as well as lower operating cost. also paves the way for organizations to adopt data visualization and modeling tools like PowerView for increased insights into their business. Avanade Approach/Value Mainstream support for SQL Server 2008 ended on July 8, 2014, and extended support for SQL Server 2005 ends on April 12, 2016. Customers with business critical applications using SQL Server 2008 and SQL Server 2005 that rely on Microsoft product support should consider migrating to. The time to migrate and test business applications can be a considerable amount of time depending on the portfolio of applications. Beyond the support issues outlined in the previous paragraph, there are four main scenarios where customers should consider moving to : Improve Transactional Performance Customers with line of business transactional applications may want to look at Memory Optimized Tables as a way to improve transactional performance which is only possible with the in-memory capabilities in. Business applications whose performance is affected by engine-level concurrency, such as latch contention or blocking, will see significant improvements when their application moves to In-Memory OLTP. Businesses can leverage improved transactional performance among LOB applications like CRM or logistics tracking, giving users high performance computing to increase efficiency. High Performance Data Analysis - Customers with business intelligence systems that make heavy use of table scans would certainly benefit from the ColumnStore index technology. ColumnStore indexes give high performance gains for queries that use full table scans and benefits include high compression rates which improve query performance by using a smaller in-memory footprint. In turn, query performance can improve because SQL Server can perform more query and data operations in-memory with the in-memory advancements founds in. This can be an essential advancement to bring high performance to real-time insights for sales analytics and adopting mobile visualization tools. Enterprises investing in big data or business intelligence will find that SQL Server 2014 has the performance they need to understand their data or external data whether it is real-time sales data, understanding their possessive social media interactions or analyzing stock market events. 2014 Avanade Inc. All rights reserved. 7
Enhanced Disaster Recovery - Customers will find the AlwaysOn Availability Groups feature essential as a comprehensive disaster-recovery solution that provides an enterpriselevel alternative to database mirroring. Advancements in disaster recovery allow enterprises and organizations to minimize risk and automate data management. supports two different availability alternative modes: a. Asynchronous-commit mode. This availability mode is a disaster-recovery solution that works well when the availability replicas are distributed over considerable distances. b. Synchronous-commit mode. This availability mode emphasizes high availability and data protection over performance, at the cost of increased transaction latency. A given availability group can support up to three synchronous-commit availability replicas, including the current primary replica. Mobile and Data Visualization Tools - PowerView and PowerPivot make it easier to manipulate larger datasets with the familiarity and convenience of Excel. Data has the power to enrich and improve productivity for nearly every employee at an organization. Unfortunately data analysis is seen as intimidating and often reserved for those with strong statistical or mathematical backgrounds. Microsoft SQL Server 2014 combines the ease of use found in Office 365 to enable every employee the ability to analyze datasets and produce impressive graphics. For example, PowerView and PowerPivot allow an organization to model sales data as a great way to present data to decision makers and track KPIs. Conclusion Today s enterprises must leverage every asset, resource and opportunity to achieve its business objectives and remain flexible in a constantly changing business climate. Avanade Data and Analytics services and solutions help you retain and understand all of your data for immediate insights into how you can accelerate the growth of your business with the power of SQL Server 2014. Using our global experience and expertise with the latest Microsoft technologies, we build solutions that help your executives and managers stay ahead of the competition - by giving them the ability to make confident, timely business decisions to minimize risk and capitalize on real time opportunities. References 1 http://msdn.microsoft.com/en-us/ library/bb500435.aspx 2 3 4 Gartner: Magic Quadrant for Data Warehouse Database Management Systems - Mark Beyer, Roxane Edjlali, Mar 2104 Gartner: Magic Quadrant for Operational Database Management Systems - Donald Feinberg, Merv Adrian, Nick Heudecker, Oct 2014 Gartner: Magic Quadrant for Business Intelligence and Analytics Platforms - Rita L. Sallam, Joao Tapadinhas et al., Feb 2014 5 Forrester: Forrester Wave : Agile Business Intelligence Platforms: Q3 2014 - Boris Evelson, Jul 2014 6 7 Forrester: Forrester Wave : Enterprise Business Intelligence Platforms, Q4 2013 - Boris Evelson, Dec 2013 IDC Whitepaper: Capturing the $1.6 Trillion Data Dividend (Sponsored by Microsoft, Dan Vesset, Henry Morris etc al., May 2014 8 http://www.microsoft.com/investor Relations: Earnings Release FY14 Q4, Earnings Call Transcript - Jul 2014 iii Microsoft Developer Network http://tiny.cc/5hbtjx iv Datasheet http://tiny.cc/ve6rjx v Microsoft Developer Network http://tiny.cc/u9btjx vi Microsoft quick-notes poster Team Leader BMW http://tiny.cc/8rctjx About Avanade Avanade helps customers realize results in a digital world through business technology solutions, cloud and managed services that combine insight, innovation and expertise focused on Microsoft technologies. Our people have helped thousands of organizations in all industries improve business agility, employee productivity and customer loyalty. Avanade combines the collective business, technical and industry expertise of its worldwide network of experts with the rigor of an industrialized delivery model to provide high quality solutions using proven and emerging technologies with flexible deployment models on premises, cloud-based or outsourced. Avanade, which is majority owned by Accenture, was founded in 2000 by Accenture LLP and Microsoft Corporation and has 22,000 professionals in more than 20 countries. Additional information can be found at www.avanade.com. 2014 Avanade Inc. All rights reserved. The Avanade name and logo are registered trademarks in the US and other countries. Other brand and product names are trademarks of their of their respective owners.