Whitepaper: Solution Overview - Breakthrough Insight Published: March 7, 2012 Applies to: Microsoft SQL Server 2012 Summary: Today s Business Intelligence (BI) platform must adapt to a whole new scope, scale, and diversity of information and data and it must embrace the new ways that we discover and collaborate around information every day. Microsoft SQL Server 2012 empowers BI users through easy-to-use self-service capabilities for Office and SharePoint, while balancing the need to maintain credible, consistent data throughout the organization with robust data management capabilities. With the emergence of Big Data, it is all the more important to ensure that BI users are able to gain business insight from all available data, whether structured or unstructured. In this paper, we will also discuss Microsoft s capabilities for Big Data as an integral part of a BI Platform for the modern enterprise.
Copyright The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication. This white paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in, or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. 2011-2012 Microsoft Corporation. All rights reserved. Microsoft, SQL Server, Windows Server, SharePoint, Excel are trademarks of the Microsoft group of companies. All other trademarks are property of their respective owners. 2
Table of Contents The World of Data is Changing... 4 Breakthrough Insight with SQL Server 2012... 4 New with SQL Server 2012... 5 Rapid Data Exploration and Visualization... 5 Stunning Interactive Visualization with Power View... 5 Self-Service Analysis with PowerPivot... 7 Managed Self-Service BI... 8 Insight and Oversight... 8 User Created, IT Managed... 9 Ease of administration through SharePoint:... 9 Credible, Consistent Data... 9 BI Semantic Model... 9 Data Quality Services... 10 Master Data Services... 11 Integration Services... 11 Big Data Analytics and Data Warehousing... 11 Massive Data Warehousing:... 11 Next-Gen Performance with xvelocity In-Memory:... 12 Fast time to Solution:... 12 Microsoft Big Data Solution... 12 Conclusion... 12 3
The World of Data is changing A wave of change is transforming the world of data today. Businesses are facing an explosion in information volumes and new data types, generated from transactional growth and an increase in unstructured data, image files, and data from new sources, such as sensor networks. We re being asked new types of questions today as our businesses go online, we are interacting through new channels that provide a whole new set of data and a different level of urgency to move forward in real-time. There is a new scope, scale, and diversity to the types and shapes of information that we now need to process, manage, and analyze for business insight. The data explosion goes far beyond traditional data types. According to Gartner, the total worldwide volume of data is growing at a rate of 59% per year. Furthermore, Gartner estimates that 70-85% of data is unstructured 1. The dramatic shift from structured to unstructured and complex data types requires organizations to embrace back-end solutions that support complex data types and non-traditional data sources like Big Data. The way we work, and our expectations about how we work, are changing. With everything we are able to do on the Web, and the emergence of social media today, we are discovering, sharing, and collaborating on information in whole new ways. Business Intelligence is no different, in terms of how we access information, collaborate and work with others, and build on the work that other people have done. Breakthrough Insight with SQL Server 2012 Breakthrough Insight is about empowering everyone across the organization with self-service capabilities through the familiar Office productivity suite, embracing the way that we discover and collaborate around information every day. It is about providing IT with the tools they need to manage data of any size and type, across relational and non-relational, to harness the power of all available data and enable new business insights. SQL Server 2012 delivers Breakthrough Insight pervasively across the organization with the following enhancements: Rapid Data Exploration: Microsoft continues to deliver on the strategy of empowering users with self-service capabilities through the familiar productivity tools they use every day Office and SharePoint. With SQL Server 2008 R2, we first introduced the popular PowerPivot Add-In for Excel, which enables users of all levels to quickly access data from virtually any source, analyze and model that data in Excel, and then collaborate and share insights through SharePoint. With the release of SQL Server 2012 we are introducing a new tool in our self-service line up Power View. Power View is a highly interactive, browser-based tool for ease of data exploration and visualization. Power View is designed to enable users of all levels across the organization to gain rapid insight from data, and to collaborate and share that insight through SharePoint and PowerPoint. Managed Self-Service BI: This is about providing IT with the tools they need to manage and protect the data and content that end users create. Microsoft uniquely provides IT administrators with the insight and oversight they need via tools for monitoring and managing user-generated content and for transforming that content into corporate-grade solutions that are professionally 1 Source: Gartner Symposium Presentation Information Management Goes 'Extreme': The Biggest Challenges for 21st-Century CIOs, Mark Beyer, October 2011 4
managed by the IT department. This helps organizations simplify compliance without hampering user agility and creativity. Credible, Consistent Data: With a growing number of data types and sources, it is increasingly important for organizations to ensure that the data users are accessing is credible and consistent. With SQL Server 2012 we provide a complete and integrated suite of Enterprise Information Management tools. New to this release, we introduce Data Quality Services, which helps to improve the quality of data and delivers unique new capabilities, such as the ability to connect to the Windows Azure Marketplace and use 3 rd party reference data to reliably profile, cleanse, and match data. Along with productivity enhancements and a new Excel Add-In for maintenance of master data, these tools reduce time-to-solution and ensure ease of management going forward. Big Data Analytics & Scalable DW: Finally, organizations need to harness the power of all available data, across relational and non-relational, to drive the types of new insights that lead to innovation and new business opportunities. Microsoft customers are realizing the benefits of SQL Server in some of the largest analytical and Data Warehouse solutions today. New SQL Server 2012 capabilities add complete fault tolerance and disaster recovery with AlwaysOn, as well as next generation performance improvements of 10X-100X with the new xvelocity in-memory technologies. SQL Server 2012 also activates non-relational data types, bridging beyond the DataWarehouse to embrace Big Data through Hadoop. New with SQL Server 2012 Rapid Data Exploration SQL Server 2012 enhances the data exploration capabilities of organizations by empowering users to access and integrate data from virtually any source. It allows users to create compelling reports and analytical applications, and easily collaborate and share insights using familiar tools. With this new version, users can create interactive, tabular, graphical, or free-form reports using data from a wide variety of data sources, including on-premises, syndicated, or unstructured data sources. SQL Server 2012 also includes customization capabilities with several programmable features. Stunning Interactive Visualization with Power View Microsoft makes self-service reporting a reality by providing a highly interactive and responsive webbased data exploration, visualization, and presentation experience for users of all types from business executives to information workers. Now anyone can easily create a report in just seconds, transform the shape of data with a single click, add powerful timed animation sequences to quickly identify trends or anomalies, and make a more convincing case through rich presentation of discovered insights. 5
Figure 1. Power View Report Power View includes the following main benefits: Visual Design Experience: Users can access and analyze information in a completely interactive, web-based authoring environment that has the familiar look and feel of Microsoft Office. Users can create and manipulate data with a variety of tables, charts, and views to visualize data in a way that best suits their purpose. Filtering and Highlighting of Data: Power View can identify the relationships between different tables in the underlying BI Semantic Model, enabling users to interact with the data to gain insights. This common metadata layer allows users to apply various kinds of visualization filters and highlight capabilities across the entire report. Presentation-ready: Power View enables users to share information quickly and easily, and lets users browse and present data at virtually any time, without having to preview it on any other platform to see how it looks. Users publish the reports to SharePoint Server, allowing other users to easily view and interact with the information. Performance: Power View retrieves only that part of the data which is needed for visualization at any given time. This enhances performance by saving the large amount of time and resources 6
that would be needed to load all of the data and by providing users with a highly interactive experience from the start. Based on the BI Semantic Models: Users can launch Power View over a tabular model (a PowerPivot model or tabular model running on SSAS) directly from a browser, without having to download anything on their local machines. Self-Service Analysis with PowerPivot With SQL Server 2012, Microsoft empowers users to access and integrate data from virtually any source, to create compelling self-service reports and analytical applications. These reports can be published to SharePoint Server and then shared across the organization easily with enhanced collaboration features, IT organizations can manage these solutions with complete oversight through the PowerPivot Management Dashboards available in the SharePoint Central Administration Console, allowing IT to manage and protects the data without hampering user creativity. Figure 2. PowerPivot workbook With this release, PowerPivot provides new advanced analytic capabilities, such as the ability to create KPIs, Advanced Sorting, Perspectives, Hierarchies and sophisticated business logic. New features also include extended DAX (Data Analysis Expressions) functionality, giving end users the power to build sophisticated analytical solutions with the familiarity of Excel-like functions. 7
Managed Self-Service BI Insight and Oversight for IT The PowerPivot Management Dashboard, available as part of the SharePoint 2010 Central Administration console, helps IT monitor data and workbook usage and gather performance metrics from servers. This diagnostic information provides more visibility into how data is used throughout the organization, helping to gain a deeper understanding of usage patterns so that IT can decide where best to invest time and resources. Figure 3. PowerPivot Management Dashboard End user created, IT managed: SQL Server 2012 now bridges the gap between end user created Self-Service content and IT managed corporate solutions by providing the importing of PowerPivot models directly into Analysis Services. IT Professional can identifies workbooks of interest through the PowerPivot Management Dashboard, and actively engage in managing that workbook for the user. This is made possible with the extended functionality available in the professional BI development tools in Visual Studio. IT professionals can add role-based security or other needed enhancements and deploy the model back out to the user community. 8
Figure 4. Import from PowerPivot feature in Visual Studio Ease of administration through SharePoint: SQL Server 2012 delivers new capabilities that provide an improved SharePoint integration platform for Reporting Services and self-service BI features, like Power View. This helps to reduce TCO for SharePoint administrators by consolidating Reporting Services administration, configuration, and management capabilities into SharePoint 2010 Central Administration Portal and PowerShell scripting. IT can now enable reporting capabilities to all information workers in the organization by using a single switch, and they can manage it like any other SharePoint feature. Credible, Consistent Data BI Semantic Model SQL Server 2012 introduces the BI Semantic Model, a single model for users with multiple ways of building business intelligence (BI) solutions. This includes: Continued support for the powerful online analytical-processing (OLAP) technologies that make SQL Server Analysis Services indispensable to BI specialists. Tools for IT professionals and developers accustomed to dealing with data in rows and columns. Support for a spectrum of BI solutions that span personal, team, and corporate contexts. 9
Figure 5. BI Semantic Model With the BI Sematic Model, organizations can scale from small personal BI solutions to the largest organizational BI needs. The BI Semantic Model is one model for all end user experiences: reporting, analytics, scorecards, dashboards, and custom applications. It offers developers flexibility of modeling experiences, richness to build sophisticated business logic, and scalability for the most demanding enterprise needs. Enterprise Information Management (EIM) Data Quality Services A new service to SQL Server, Data Quality Services (DQS) helps round out end-to-end data management for organizations. DQS provides knowledge-driven tools customers can use to allow data stewards to create and maintain a Data Quality Knowledge Base that helps improve data quality and ease data management. Specifically, customers can gain confidence in data quality by using organizational knowledge to profile, cleanse, and match data. Data Quality Services can be run as a standalone tool or integrated with Integration Services (SSIS). With SQL Server 2012, customers will be able to access the Windows Azure Marketplace DataMarket as a source of 3rd party data to help validate and cleanse data in a data quality project. 10
Master Data Services Master Data Services (MDS) continues to make it easier to manage master data structures (object mapping, reference data, and dimensions and hierarchies) used in data integration operations. With new Entity Based Staging, all members and attribute values for an entity can be loaded at one time. Additionally, the Explorer and Integration Management functional areas of the Master Data Manager Web application have been updated with a new Silverlight look and feel. Members can be added and deleted more quickly, and they can be moved into a hierarchy more easily. The new MDS Add-in for Excel democratizes data management, so information workers have the ability to build data management applications directly in Excel. Information workers can use this add-in to load a filtered set of data from the MDS database, work with the data in Excel, and then publish the changes back to the MDS database. Administrators can also use the add-in to create new entities and attributes. Integration Services SQL Server Integration Services (SSIS) helps lower the barriers to getting started with data integration. SSIS enables organizations of all sizes to operationalize their commitment to information with capabilities that increase efficiency and improve productivity related to information management efforts. SQL Server 2012 Integration Services delivers comprehensive support to help run your data integration capabilities as a mission critical application. SSIS includes rich DBA and IT Implementer support for the deployment and administration of ETL tasks. SQL Server 2012 also includes the ability to run administration and other capabilities as a separate SQL Server instance. Big Data Analytics and Data Warehousing SQL Server 2012 supports the largest of enterprise needs for Analytic and Data Warehouse solutions, as well as complete fault tolerance and disaster recovery with AlwaysOn, providing the required 9s of up time. Massive Data Warehousing: Microsoft provides a range of solutions to help customers address Big Data challenges. Our family of data warehouse solutions from Microsoft SQL Server 2012, SQL Server Fast Track Data Warehouse, and SQL Server 2008 R2 Parallel Data Warehouse offer a robust and scalable platform for storing and analyzing data in a traditional data warehouse. SQL Server 2012 provides enhanced features such as Remote Blob Storage and partitioned tables that scale to 15,000 partitions to support large sliding window scenarios. Also available is increased support for up to 256 logical cores to enable high-performance for very large workloads and consolidation scenarios. 11
Next-Gen Performance with xvelocity In-Memory: Microsoft SQL Server 2012 provides a generational leap in performance with the new xvelocity inmemory technologies allowing typical improvements of 10X-100X for Analytics and Data Warehousing. With scan rates of 10s of Billions of rows per second on typical industry hardware, xvelocity gives users the ability to actively interact and explore on an unprecedented amount of data at the speed of thought. Fast time to Solution: Fast time to solution is made available through reference architectures, appliances that offer pre-tested, pre-configured, and pre-tuned integration with open tier-one industry hardware, and software that installs and configures 4X faster than other leading vendors. SQL Server offers one of the lowest costs of acquisition and ownership, where real-life scenarios show savings of hundreds of thousands of dollars after migrating to SQL Server. Microsoft Big Data Solution Microsoft s Big Data strategy embraces Hadoop for activating ambient data being born outside the traditional data platform. Hadoop is the open source implementation of MapReduce parallel computation engine and environment, and it is used for processing streams of data that go well beyond the size of even the largest enterprise data sets. Whether it s sensor, clickstream, social media, location-based or other data that is generated and collected in large gobs, Hadoop is often on the scene in the service of processing and analyzing it. Microsoft is delivering an enterprise-class implementation, or distribution, of Hadoop for Windows Server and Windows Azure, integrated with SQL Server 2012, Active Directory, and System Center to make it dramatically easier, efficient, and cost effective for organizations to capitalize on the opportunity Big Data can bring. Microsoft s Big Data solution enables customers to augment their analysis with publicly available data from social media sites such as Twitter and Facebook, and hundreds of trusted data providers on Windows Azure Marketplace. Azure Marketplace also exposes hundreds of applications and data mining algorithms to help unlock new insights for customers. Also, through deep integration with PowerPivot, PowerView, and EDW tools, Microsoft s Big Data solution offers customers deep insights on all their structured and unstructured data with the tools they use every day. Conclusion Organizations today are challenged with managing an explosion in information volume and new data types. To address this, they need BI solutions to manage increased scale and a diversity of information and data. At the same time, they need to empower end users with tools for gaining and sharing business insight. Microsoft SQL Server 2012 offers a balance of user-powered BI and data management capabilities by both empowering end users with easy-to-use Self-Service capabilities for Office and SharePoint, and providing the tools to maintain credible, consistent data throughout the organization. 12
With Power View, Microsoft delivers powerful self-service reporting to any user with a highly interactive, web-based data exploration, visualization, and presentation experience. Power View provides an interactive data exploration and visual presentation tool to users who don t have technical knowledge in queries and reporting. With xvelocity in-memory technologies, this allows end users to analyze billions of rows of data at the speed of thought. In addition, SQL Server 2012 offers enhancements to PowerPivot that deliver more advanced analytic capabilities and give end users the power to build sophisticated analytical solutions with the familiarity of Excel-like functions. SQL Server 2012 offers capabilities for managed self-service BI, helping IT to provide end users with powerful BI functionality while balancing the need to monitor and manage end user generated content. IT can gain improved insight and oversight with features like the PowerPivot IT Management Dashboard that increases visibility into how data is used throughout the organization. With the ability to import PowerPivot models into Analysis Services, SQL Server 2012 helps close the gap between end user created BI applications and IT managed corporate solutions. SQL Server 2012 also offers improved SharePoint integration, helping to ease administration. With a single switch, IT can enable reporting capabilities to end users throughout the organization and manage it like any other SharePoint feature. With SQL Server 2012, organizations have the tools to help ensure that they have more credible, consistent data. The new BI Semantic Model provides a single model for all end user experiences, enabling organizations to scale from small personal BI solutions to the largest organizational BI needs. The new Data Quality Services (DQS) enables organizations to create and maintain a Data Quality Knowledge Base that improves data quality and eases data management helping to leverage organizational knowledge to profile, cleanse, and match data. Master Data Services (MDS) continues to make it easier to manage master data structures and offers new enhancements, such as Entity Based Staging, updates to the Master Data Manager Web application, and the new MDS Add-in for Excel. SQL Server Integration Services enhancements, and SSIS as a Server, help organizations increase efficiency and improve productivity in information management efforts. Lastly, Microsoft continues its ongoing commitment to Big Data Analytics and Data Warehousing. Microsoft s range of data warehouse offerings provides robust and scalable solutions for storing and analyzing data in a traditional environment. Now, Microsoft is embracing Apache Hadoop as part of delivering on its vision to provide business insights to all users from any data. Microsoft will ship an Apache Hadoop-based distribution for Windows Server and Windows Azure, enabling customers to derive business insights from structured and unstructured data. By leveraging familiar Microsoft tools, and maintaining compatibility with existing Hadoop tools, such as Pig, Hive, and Java, the new Hadoop-based distribution for Windows offers IT professionals a simplified acquisition, installation, and configuration experience. With this new distribution, rich insights from Hadoop can be combined seamlessly with the Microsoft Business Intelligence Platform, enabling a range of users to unlock business insights from any data. For more information: http://www.microsoft.com/sqlserver/: SQL Server Web site http://www.microsoft.com/bigdata/: Big Data Solutions 13